Public sentiment analysis using Natural language processing on social media to determine Indian stock index movement

Abstract

Sentiment analysis is detection & extraction implicit sentiment and opinion of a semantic orientation. There are some challenges for using sentiment analysis which still has net been solved, are regarding inconsistent structure of texts, data processing & transmission bottlenecks in real time [1] as per behavioral finance, information and emotion affect individual decision making and there is ingrained irrationality in financial market. But how information affect society at a large in decision making and thus stock market movement is still a field not understood clearly. At the same time, Efficient Market Hypothesis says that investors always use all available information rationally & financial market index reflects the optimum price level. Andrew Lio’s [2] Adaptive market Hypothesis indicates that there is always some inefficiency in practical market due to cognitive biases of investors, presence of short sale constraints and uninformed demand shocks. Hence, there exist an opportunity to arbitrage until all information got disseminate among public. But a major challenge for investors is to utilize an active portfolio management to gain a better price before the stock attain the saturation level. So, if the investor can predict the movement in stock price before any actual movement happens than it helps to gain a better return. Hence, there is a gap in study to determine stock price index movement through analyzing public sentiment, live media news on continuous basis. With rapid growth of online news portals, today investors heavily depend on these sources to mould their investment decision. Effect of media can be two?fold related to investment decision. Media sentiment can either mould the public investment decision and investment decision also affect media output. Contents of media news can create either a positive, a neutral or a negative sentiment among the investors and it gets continuously updated as new news get published. Here investors investment strategy affects market output, hence the media news which in turn further affect investors in decision making. Now with increasing breadth & width of media there is a huge influx of market news. It is near to impossible for a human brain to process all the media information and determine the best investment strategy for highest return for each and every asset because of high volume & speed of data. To resolve this issue & to help investors to make optimum investor strategy, text miming & sentiment analysis can be used to pre?determine affects of every media content in terms of positive or negative on the company’s stock market index.