SENTIMENT ANALYSIS CSE-634: DATA MINING

Transcription

1 SENTIMENT ANALYSIS CSE-634: DATA MINING TEAM 4 PROF. ANITA WASILEWSKA

2 REFERENCES Johan Bollen, Huina Mao, Xiaojun Zeng Twitter mood predicts the stock market, Journal of Computational Science 2:1, /j.jocs

3 OVERVIEW What is Sentiment analysis? Why Sentiment analysis? How does it work? Demo Challenges Application Future of sentiment analysis Few Tools and Concepts of Sentiment Analysis Paper presentation

4 SENTIMENT ANALYSIS It allows us to track attitudes and feelings on the web. People write blog posts, comments, reviews and tweets about all sorts of different topics. We can track products, brands and people for example and determine whether they are viewed positively or negatively on the web. Source: Source:

5 OTHER NAMES FOR SENTIMENT ANALYSIS Opinion extraction Opinion mining Sentiment mining Subjectivity analysis Source:

6 SCHERER TYPOLOGY OF AFFECTIVE STATES Emotion: brief organically synchronized evaluation of a major event. angry, sad, joyful, fearful, ashamed, proud, elated Mood: diffuse non-caused low-intensity long-duration change in subjective feeling cheerful, gloomy, irritable, listless, depressed, buoyant Interpersonal stances: affective stance toward another person in a specific interaction friendly, flirta1ous, distant, cold, warm, supportive, contemptuous Attitudes: enduring, affectively colored beliefs, dispositions towards objects or persons liking, loving, valuing, desiring Personality traits: stable personality dispositions and typical behavior tendencies nervous, anxious, reckless, morose, hostile, jealous Source:

7 Source:

8 ELEMENTS OF SENTIMENT 1. Holder (source) of attitude 2. Target (aspect) of attitude 3. Text containing the attitude (Sentence or entire document) 4. Type of attitude From a set of types (Like, love, hate, value, desire, etc. ) Simple more commonly used weighted polarity: positive, negative, neutral, together with strength

9 POSITIVE SENTIMENTS

10 NEUTRAL SENTIMENTS

11 NEGATIVE SENTIMENTS

12 WHY SENTIMENT ANALYSIS? Movie: is this review positive or negative? Products: what do people think about the newly released product? Politics: what do people think about this candidate or issue? Prediction: predict election outcomes or market trends from sentiment Source:

13 METHODS OF DOING SENTIMENT ANALYSIS Lexical approach 1. Dictionary based approach 2. Corpus based approach Machine learning 1. Supervised learning 2. Unsupervised learning Source:

14 HOW DOES IT WORK? THE MACHINE LEARNING WAY Input text Transformation into feature vector Initialise models such as NN Tokenize Dictionary creation Train the model Stopword filtering Stemming Predict the sentiment

15 HOW DOES IT WORK? THE LEXICAL WAY Preprocess input text Polarity calculation Output the sentiment score Tokenization Sentiment detection by score Negation handling POS tagging Source:

16 DEMO This is a demonstration of sentiment analysis using a NLTK powered text classification process. It can tell you whether it thinks the text you enter below expresses positive sentiment, negative sentiment, or if it's neutral.

17 CHALLENGES LEADING TO LATEST DEVELOPMENTS

18 DETECTING SENTIMENT OF THE WRITER, READER, AND OTHER ENTITIES For example, James: The pop star suffered a fatal overdose of heroine People can react differently to the same utterance, for e.g. people on the opposite sides of a debate or rival sports fans. Modeling listener sentiment requires modeling listener profiles. This is an area of research not explored much by the community. Source:

19 SENTIMENT TOWARDS ASPECTS OF AN ENTITY Source:

20 SENTIMENT TOWARDS ASPECTS OF AN ENTITY A review of a product or service can express sentiment towards various aspects. For example, I really liked the pizza at the restaurant but their service was disappointing. The best performing systems for detecting aspect sentiment adds localization features so that more weight is given to sentiment features close to the mention of the aspect. Source:

21 STANCE DETECTION Task of automatically determining from text whether the author of the text is in favor of, against, or neutral towards a proposition or target. For example, Target of interest: women have the right to abortion Text: A foetus has rights too! Automatically detecting stance has widespread applications in information retrieval, text summarization, and textual entailment. Source:

22 CHALLENGES IN SPECIFIC TYPES OF SENTIMENT COMPOSITION Negated Expressions Earlier works on negation handling employ simple heuristics such as flipping the polarity of the words in a negator s scope or changing the degree of sentiment of the modified word by a fixed constant. Several Proposals for embeddings-based recursive neural network models to capture the impact of negators more precisely Still Unexplored: In what contexts does the same negator impact the sentiment of its scope more and in what contexts is the impact less; how do people in different communities and cultures use negations differently Source:

23 CHALLENGES IN SPECIFIC TYPES OF SENTIMENT COMPOSITION Sentiment of Sentences, Tweets, and SMS messages creatively spelled words (e.g. happpeee) slang (e.g. bling) abbreviations (e.g. lol) hashtags and conjoined words (e.g. #loveumom) tweets with emoticons. Source:

24 CHALLENGES IN SPECIFIC TYPES OF SENTIMENT COMPOSITION Phrases with Degree Adverbs, Intensifiers, and Modals For example, cannot work seems less positive than work or will work (cannot and will are modals). Several areas remain unexplored, such as identifying patterns and regularities in how different kinds of degree adverbs and intensifiers impact sentiment, ranking degree adverbs and intensifiers in terms of how they impact sentiment, and determining when (in what contexts) the same modifier will impact sentiment differently than its usual behavior. I m pretty sure we ve met before [quite, but not extremely] He has a pretty strident rant about how important it is. Source:

25 CHALLENGES IN ANNOTATING FOR SENTIMENT IN DIFFERENT INSTANCES Success or failure of one side w.r.t. another Yay! France beat Germany 3 1, Supreme court judges in favor of gay marriage, and the coalition captured the rebels. In the 2014 Sochi Winter Olympics when Finland beat Russia in ice hockey, the event was tweeted around the world predominantly as Russia lost to Finland as opposed to Finland beat Russia. Source:

26 CHALLENGES IN ANNOTATING FOR SENTIMENT IN DIFFERENT INSTANCES Sarcasm and ridicule Source:

27 MORE CHALLENGING EXAMPLES Disliking swimming is not really my thing. (Negation, inverted word order) Sometimes I really hate the place (Adverbial modifies the sentiment) I'd really truly love going out in this cold weather! (Possibly sarcastic) Chris Craft is better looking than Limestone. (Two brand names, identifying target of attitude is difficult). The movie is surprising with plenty of unsettling plot twists. (Negative term used in a positive sense in certain domains). I love my mobile but would not recommend it to any of my colleagues. (Qualified positive sentiment, difficult to categorize) The actors are good, the music is brilliant and appealing. Yet, the movie fails to strike a chord (Contains words that contradict the overall sentiment of the sentence) Source:

28 INTERESTING STORY Expedia Canada noticed that there was a steady increase in negative feedback to the music used in one of their television adverts. Sentiment analysis conducted by the brand revealed that the music played on the commercial had become incredibly irritating after multiple airings, and consumers were flocking to social media to vent their frustrations. A couple of weeks after the advert first aired, over half of online conversation about the campaign was negative. Expedia then reacted by airing a new version of the advert. Source:

29 CAVEATS OF SENTIMENT ANALYSIS Knowing sentiment is a very natural ability of a human being. Can a machine be trained to do it as good as a human? Sentiment analysis has its limitations and is not to be used as a 100% accurate marker. As with any automated process, it is prone to error. Beyond reliability, it s important to acknowledge that human s expression doesn t fit into just three buckets; not all sentiment can be categorized as simply as positive, negative or neutral. Source:

30 TWITTER SENTIMENT APP Twitter Sentiment App Alec Go, Richa Bhayani, Lei Huang Twitter Sentiment Classification using Distant Supervision

31 RESEARCH PAPER

32 Tools and Concepts used in the paper 1. OpinionFinder: It is a publicly available software package for sentiment analysis that can be applied to determine sentence-level subjectivity, i.e. to identify the emotional polarity (positive or negative) of sentences. 2. GPOMS: Unlike OF, which is a unidimensional model of mood, Google-Profile of Mood States, a proprietary tool, can measure human mood states in terms of 6 different mood dimensions, namely Calm, Alert, Sure, Vital, Kind and Happy. GPOMS mood dimensions and lexicon are derived from an existing and well-vetted psychometric instrument, namely the Profile of Mood States (POMS-bi).

33 Daily time series of public mood fluctuations OF and GPOMS were applied to tweets posted in a 2-month period (Oct - Dec, 2008) This period was chosen specifically because it includes several socio-cultural events that may have had a unique, significant and complex effect on public mood namely- 1. U.S presidential election and 2. Thanksgiving Correlation between the trend obtained from OF lexicon and the six dimensions of GPOMS using multiple regression Y OF = α + i=1 6 β i X i + ε t, Img Source:

34 3. Granger causality analysis: It is a statistical hypothesis test for determining whether one time series is useful in forecasting another. While regressions reflect "mere" correlations, this has the ability to predict the future values of a time series using prior values of another time series. 4. MAPE: The mean absolute percentage error is a measure of prediction accuracy of a forecasting method in statistics, for example in trend estimation. 5. MDA: Mean Direction Accuracy is a measure of prediction accuracy of a forecasting method in statistics. It compares the forecast direction (upward or downward) to the actual realized direction

35 6. Self-Organizing Fuzzy Neural Network model A Self-organizing learning algorithm: An evolving learning algorithm which will automatically determine the number of neurons or basis functions and learn model parameters Convergence to real systems has been proved if there are enough training data High accuracy in comparing many other methods A fuzzy neural network is a learning machine that finds the parameters of a fuzzy system (i.e., fuzzy sets, fuzzy rules) by exploiting approximation techniques from neural networks. Fuzzy sets is mathematical model of vague qualitative or quantitative data i.e somewhat like sets whose elements have degrees of membership. Img Source:

36 Algorithm Steps: Start from a network with one neuron Adding a neural (or basis function) when the existing model is not complicated enough to catch up the nonlinear behavior. Merger neurons if they are highly similar to avoid overfitting. Pruning a neuron if it makes little contribution to the forecasting accuracy Recursive least square parameter learning to catch the time varying factor. The output of a neuron in the layer 5 is: Img Source:

37 Twitter Mood Predicts the Stock Market Johan Bollen, Huina Mao, Xiao-Jun Zeng School of Informatics and Computing, Indiana University, USA School of Computer Science, University of Manchester, UK Publication: Journal of Computational Science, Volume 2, Issue I, March 2011 Citations:

38 "Sooo does anyone else not open Snapchat anymore? Or is it just me... ugh this is so sad," Jenner tweeted on Wednesday.

39 Why this paper? The stock market is an important part of the economy of a country as it plays pivotal role in the growth of its industries and commerce, eventually affects the economy of the country. Ever-fluctuating - prices are largely driven by new info. News is unpredictable and follow a random walk pattern. Cannot be predicted with more than 50% accuracy. New research suggests that mining social media data can provide valuable information to predict changes in various economic/commercial fields. (e.g; A paper in AAAI 2006 Spring Symposium showed assessments of blog sentiment to predict future movie sales)

40 Why this paper? (Cont.) Behavioral finance has provided proof that financial decisions are significantly driven by emotion and mood. Hence => Sentiment and mood has significant impact to investors behavior and stock move We need reliable, scalable and early assessments of the public mood at a time for practical prediction. Limitations: i) Large surveys are expensive and time-consuming ii) The accuracy is limited Goal: Investigate whether public analyzing sentiment, as large-scale collections of daily Twitter posts, can indeed be used to predict the stock market or not!!

41 Data Overview Timeline: Public tweets: Feb 28 - Dec 19th, 2008 (9,853,498 tweets posted by approximately 2.7 M users). Data type : Tweets that contain explicit statements of their author's mood states, i feel, i am feeling, i m feeling, i dont feel, I m, Im, I am, and makes me. The Dow Jones Industrial Average (DJIA) is a stock market index that shows how 30 large publicly owned companies based in the US have traded during a standard trading session. e.g; 3M, Amex, Apple, Boeing Extract a time series of daily DJIA closing-values from Yahoo! Finance. (Feb 28 to Dec 19th, 2008)

42 Approach 3 phases: 1) Creation and validation of OF and GPOMS public mood time series Oct-Dec '08 (2) Use of Granger causality analysis to determine correlation between DJIA, OF and GPOMS public mood Aug-Dec '08 (3) Training of a SOFNN to predict DJIA v a l u e s o n t h e b a s i s o f v a r i o u s combinations of past DJIA values and OF and GPOMS public mood data March-Dec 08. Img Source:

43 Method Overview 1. Granger causality analysis to correlate the daily time series values produced by GPOMS and OpinionFinder vs. the DJIA of past n days using 2 linear models: L 1: D t = α + i=1 N β i D t 1 + ε t L 2: D t = α + i=1 N β i D t 1 + i=1 N γ i X t 1 + ε t Results (P values) reject the null hypothesis with high level of confidence, meaning, mood time series do predict DJIA values, however this result only applies to 1 GPOMS mood dimension (Calm) 2. To better address the non-linear effects, compare the performance of a SOFNN model that predicts DJIA values on the basis of two sets of inputs: (1) the past 3 days of DJIA values, and (2) the same combined with permutations of our mood time series. SOFNN model deployed to test the hypothesis that the prediction accuracy of DJIA prediction models can be improved by including measurements of public mood.

44 Experiment Result DJIA daily prediction using SOFNN. I 0 = {DJIA t-3,2,1 } -input vars I 1 = {DJIA t-3,2,1, X 1,t-3,2,1 } I 1,2 = {DJIA t-3,2,1, X 1,t-3,2,1,, X 2,t-3,2,1 } I 1,3 =. I OF = {DJIA t-3,2,1, X OF,t-3,2,1 } Img Source:

45 Experiment Result To assess the statistical significance of the SOFNN achieving the above mentioned accuracy in predicting the up and down movement of the DJIA, the odds of this result occurring by chance was calculated. The SOFNN direction accuracy is most likely not the result of chance nor selecting a specifically favorable test period Since the SOFNN prediction is more accurate when using a combination of X 1 and X 6, this confirms a nonlinear relation among the different dimensions of moods so is between public mood and stock market values. Certain dimensions of public mood states, in particular Calm, increase the accuracy of a Self Organizing Fuzzy Neural Network model in predicting up and down changes in DJIA closing values to 87.6%.

46 Conclusion and Future Research Social media data include some useful mood and sentiment information related to stock market. Including such mood and sentiment information can improve the accuracy of stock prediction. More comprehensive prediction system combines: o Online news mining from web data and social media data, as big news are often accumulated from many small news and such online mining will provide the early indicators for the current unpredictable news. o Online social media data mining to get early mood and sentiment indicators o Historical stock price data and fundamental financial data

47