Volume 119 No. 12 2018, 15395-15405 ISSN: 1314-3395 (on-line version) url: http://www.ijpam.eu ijpam.eu Stock Market Predictor and Analyser using Sentimental Analysis and Machine Learning Algorithms 1 K.M. Umamaheswari, 2 Ayush Sharma and 3 Shivam Khandelwal 1 Department of Computer Science and Engineering, SRM Insititute of Science and Technology, Chennai, India. umamaheswari.km@ktr.srmuniv.ac.in 2 Department of Computer Science and Engineering, SRM Insititute of Science and Technology, Chennai, India. ayush_rajesh@srmuniv.edu.in 3 Department of Computer Science and Engineering, SRM Insititute of Science and Technology, Chennai, India. shivam_khandelwal@srmuniv.edu.in Abstract Stock market plays a very important role in fast economic growth of the developing country like India. From the last twenty years, the application of Internet based technologies had brought a significant impact on the Indian stock market. Use of the Internet has eliminated the barriers of brokers and geographical location because now investors can buy and sell their shares by accessing the stock market status from anywhere at any time. Before investing money, it is very important for investors to predict the stock market. In today s digital world Internet based technologies such as Cloud Computing, Big Data analytics, and Sentiment analysis have changed the way we do business. Sentiment analysis or opinion mining makes use of text mining, natural 15395
language processing (NLP), in order to identify and extract the subjective content by analyzing user s opinion. In this project, with the use of Sentimental Analysis and Machine learning algorithms, we make it easy for the stock market investors to invest their money and buy shares if they think investing in that could result into they making profit. We provide the user with two options to predict the same using either sentimental analysis or through machine learning algorithms Keywords: Sentimental Analysis, Machine Learning, Stock Market. 15396
1. Introduction The websites which have record of all the news of a particular company that User wants to invest his money in can be used as a database for sentimental analysis. Sentiment analysis is used to extract such opinion and remarks of users by classifying them as positive, negative and natural sentiment 2. Although there are a number of definitions about sentiment analysis in the literature, but in simple terms sentiment analysis is a technique used to extract intelligent information based on the person s opinion from raw data available on the internet. In this definition, the term opinion means a person s perspective about an object or issue. There are some challenges related to sentiment analysis, the first challenge is a word that is used to express an opinion; it can be positive as well as negative depending upon the type of sentiment 14. For example: if a word large is used for size of mobile device, then it is considered as negative, whereas if the statement contains large word for the height of a person then it is considered as a positive statement. The second challenge is related to the opinion holder as opinion holder is always changing its statement, according to his state of mind, it is very difficult to understand such type of statement by the machine. For example: I like the picture quality, but the battery life is poor. This statement is a combination of both positive and negative statements. Also, there is a problem when the statement is too short to understand even by human being. Indian stock market has gained the interest of investors investing in two main stock market named as Bombay Stock Exchange (BSE) and National Stock Exchange (NSE). There is high risk involved for investors because of more complexity of the stock market. The Sensex and NIFTY are two such prominent market indices that function within the Indian stock market. These two market indexes represent the stocks for BSE (Bombay Stock Exchange) and NSE (National Stock Exchange) respectively. Intelligent data analysis tools produce a data base to search for hidden information that may be missed due to beyond expert s predictions. Extraction which was previously unknown, implicit and potentially useful information from data in databases, is an effective way of data mining. It is commonly known as knowledge discovery in databases (KDD). Although data mining and knowledge discovery in databases (or KDD) both are used as similar often, Data mining is actually part of knowledge discovery. Data mining techniques play important role in stock market which can search uncover and hidden patterns and increasing the certain level of accuracy, where traditional and statistical methods 15397
2. Live Stock Market Values Python script is run in the backend where user can input the details of the company in the UI created, for which he needs the live stock market price. The user needs to input the name of the company, the keyword of that company And whether we want NSE or BSE. The result is displayed in the interface. The python script scrapes out the relevant information from a particular Financing website and the this is displayed to the user in the User Interface. 3. Role of Sentimental Analysis In determining how a particular company will do in the upcoming weeks would depend not only on the previous trends but on the current finance news also. For example:- The CEO of a XYZ company has resigned from the post due to some internal affair within the company. This would definitely effect the stock market prices of that company in the near future. This is where sentimental analysis comes into play. Let us continue this further for a given company XYZ. So we want to do sentimental analysis on the recent news article of this company. We use money control.com as the financial website for scraping out the news. 15398
The user needs to input the name of the enterprise and the python script takes the input and displays the universal link where all the news database is stored for XYZ. All the html links for the news articles for XYZ are stored in the list. After this we remove all the duplicate links for avoiding any recurrence of them. BeautifulSoup is a python library which is used for parsing the scraped data which helps in extracting the data from HTML And hence it is useful in web scraping. It makes the useful data easily distinguishable from the data which is of no use. With this we have all the scraped news on which sentimental analysis is to be done. Textblob is a blobber, which is a python library which is used for sentimental analysis. This is used for giving positive and negative scores for a sentence. This contains crores of words which are used for the score evaluation. Pattern Analyser is used in this textblob library which has a sentimental analysis approach. Polarity and Subjectivity are the two major outcomes or result of the sentimental analysis. Polarity of the given text can be determined at different levels, whether the expressed opinion, a sentence or an aspect is positive, negative or neutral. For achieving polarity classification, one can see the whole process as a pipeline including different stages that can lead to the accuracy of ending results. Polarity is closely related to emoticons which express the user s opinion. Subjectivity can be looked as the stage where a sentence is regarded as either objective or subjective. These sentences express user s beliefs, views or feelings. These do not have any emoticons attached to it. They do not express positivity or negativity of the sentence. For example:- I think he went to school is neither expresses a positive nor negative statement. So textblob is responsible to give Polarity and Subjectivity to the scraped news statement and this hence completes the Sentimental analysis part of the project. 4. Role of Machine Learning Machine learning is a method used to devise complex models and algorithms that lend themselves to prediction which is also known as predictive analysis. It 15399
involves with the study and making of the algorithms which is then used to make predictions of future from the available data. Pandas which stands for Python Data Analysis Library is a python library which is used for data manipulation and analysis. This is used to convert data set obtained from the below mentioned Yahoo API into data frames as the model used for learning takes input only as data frames. Yahoo finance api is part of Yahoo family network which is used for obtaining financial news, data and commentary including stock quotes, press releases, financial reports, and original content. We use this obtains all the records of XYZ from year 1986 till the present date which include all the stock prices (opening, closing, average). This data is easily available on Yahoo finance API. Prophet model is basically a library to build forecasting models for time series data, but instead of using the traditional way of building the model such as using ARIMA, etc., it is fitting additive regression models or known as curve fitting. They have implemented the core part of the procedure in Stan s probabilistic programming language. Prophet has been a key piece to improving Facebook s ability to create a large number of trustworthy forecasts used for decision-making and even in product features. 15400
Working of prophet This is how machine is trained using prophet model which analyses the factors such as Standard Deviation, Mean, Slope etc. This model is then used to predict data of XYZ for future and thus making it easy for the user to decide whether to invest in XYZ or not. Graphical analysis is used in this model for predicting the future values of stocks. Thus after performing this approach, we can obtain the predicted increase and predicted decrease of XYZ in the upcoming days. The important thing about the Prophet algorithm is that, even without a prior knowledge or experience in the time series data specific data preparation or the configuration of the model parameters, you will get a very reasonable result that is good enough to start the time series data exploration with useful insights. 15401
5. Applications 1. Minimising the loss in investment or maximising the profit of an investor. 2. Taking prior measures for restricting the company from going into loss. 3. Providing both sentimental and machine learning approaches and investing in the enterprise which has high score in both the approaches. 15402
4. There is no need to search the web for the live stock prices for both BSE and NSE of a particular company. 5. One can directly extract out only positive or only negative news without actually digging deep in it. 6. Conclusion This paper has presented an overview of various techniques involving different algorithms used for Machine Learning and Sentimental Analysis. Though it is not a completely blended process as it done with a mixture of processes like creating a good User Interface, Displaying of live stock prices, Doing sentimental analysis on the scraped news articles and prediction of future stock prices obtained after process of Machine Learning. The user interface include Google account login and a discussion form for clarification of the queries from users. This can be used for real time process by many investors who invest in the stock market for making more profit by not Investing in a enterprises whose both algorithm values are low. 15403
Despite the vast researching, many challenges would still be faced such as combining the algorithms of both Machine Learning and Sentimental Analysis into a single process rather then doing both of them discretely. Acknowledgement We would like to thank our project guide i.e. Assistant Proffessor K.M.Umamaheswari, who had been a constant inspiration for the entire duration of the research, without whom this project would not have been possible. We are also grateful to the Head of the Department Dr. B. Amutha for giving us this opportunity. References [1] Youngsub Han, Kwangmi Ko Kim : Sentiment Analysis on Social Media Using Morphological Sentence Pattern Model [2] B.Siddhartha Reddy : Prediction of Stock Market Indices Using SAS [3] Bhat, A.A.; Kamath, S.S., "Automated stock price prediction and trading framework for Nifty intraday trading," Computing, Communications and Networking Technologies (ICCCNT),2013 Fourth International Conference on, vol., no., pp.1,6, 4-6 July 2013. [4] Asif Ullah Khan; T.K. Bandopadhyaya; Sudhir Sharma;, Comparisons of stock rated prediction Accuracy using Different Technical Indicators with Back Propagation Neural Network and genetic Algorithm Based back propagation Neural Network., First International Conference on Emerging Trends in Engineering and Technology, IEEE [5] Vincent Martin, Predicting the French Stock Market using Social Media Analysis, 8th International Workshop on Semantic and social media adaption and personalisation, IEE, pp.3-7. 15404
15405
15406