Comments (8)
Great work. Not that it matters but I was on linux and windows. I think your benchmarking negates the need for me to test. Great stuff! FYI @GitTFJ
from sentimentr.
@AugustT What OS are you using?
Related: trinker/textclean#51
from sentimentr.
I have made changes to address this @AugustT. Could you try it out and give feedback?
from sentimentr.
On windows I get the following using this code but I am wondering what mac users will get:
gsub_reg <- function(x) gsub("\\d:\\d|\\d\\s|[^\\p{L}',;: ]", '<<>>', x)
gsub_perl <- function(x) gsub("\\d:\\d|\\d\\s|[^\\p{L}',;: ]", '<<>>', x, perl = TRUE)
library(microbenchmark)
library(sentimentr)
y <- hotel_reviews$text
r <- microbenchmark::microbenchmark(
gsub_reg = gsub_reg(y),
gsub_perl = gsub_perl(y),
times = 100
)
plot(r)
from sentimentr.
Similar results from Mac:
from sentimentr.
I get similar on Windows with \\p{L}
swapped for [:alpha:]
:
from sentimentr.
And with \\p{L}
swapped for [:alpha:]
on Mac:
from sentimentr.
One final related:
gsub_reg_p <- function(x) gsub("\\d:\\d|\\d\\s|[^\\p{L}',;: ]", '<<>>', x)
gsub_reg_a <- function(x) gsub("\\d:\\d|\\d\\s|[^[:alpha:]',;: ]", '<<>>', x)
gsub_perl_p <- function(x) gsub("\\d:\\d|\\d\\s|[^\\p{L}',;: ]", '<<>>', x, perl = TRUE)
library(microbenchmark)
library(sentimentr)
y <- hotel_reviews$text
r <- microbenchmark::microbenchmark(
gsub_reg_p = gsub_reg_p(y),
gsub_reg_alpha = gsub_reg_a(y),
gsub_perl_p = gsub_perl_p(y),
times = 100
)
plot(r)
x<-c("Jeg hader dårlige mennesker.", "Jeg kan godt lide lækker is.",
"Jeg elsker dig ikke mere; undskyld.")
Inside of make_sentence_df2
I do not use perl = TRUE
(see d24e6af) because in this case the alpha is faster than \\p{L}
and setting it perl = TRUE
for [:alpha:]
does not retain what we want in the alphabetic nonascii chars:
gsub("\\d:\\d|\\d\\s|[^['alpha:]',;: ]", '<<>>', x)
## [1] "Jeg hader dårlige mennesker." "Jeg kan godt lide lækker is."
## [3] "Jeg elsker dig ikke mere; undskyld."
gsub("\\d:\\d|\\d\\s|[^['alpha:]',;: ]", '<<>>', x, perl = TRUE)
## [1] "Jeg hader dårlige mennesker." "Jeg kan godt lide lækker is."
## [3] "Jeg elsker dig ikke mere; undskyld."
from sentimentr.
Related Issues (20)
- Some words in Hu&Liu dict have off polarity values HOT 1
- Any hint why the two emoji approaches are different and in what circumstance which one is better?! HOT 1
- Amplifiers
- polarity_dt requires words with spaces to work HOT 1
- Sentimentr split words when there is an accented character HOT 5
- Totally different results in Qdap polarity and sentiment r HOT 4
- Changing polarity of words in the dictionary HOT 2
- Question about 'highlight' HOT 1
- how to calculate sum of sentiment? HOT 1
- Question about how the dictionary was built HOT 1
- Transparency for lexicon::hash_sentiment_jockers_rinker HOT 1
- What is the theoretical range for the "sentiment" variable? HOT 1
- I wonder why this sentence is recognized as positive HOT 2
- Polarity categorization? HOT 1
- relabel "black" and "white" as neutral sentiments (black is currently negative; white is labeled positive)
- Overall positive and negative scores given a piece of text
- Positive and Negative Sentiment based on German Language text HOT 1
- how to get pos, neg, neu other than ave_sentiment ?
- plot.sentiment_by ordered fix?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sentimentr.