Using The NewsInn Algorithm To Predict Stock Market Movements Two Days In Advance

ABOUT NEWSINN

Abbildung in dieser Leseprobe nicht enthalten

Description

NewsInn is an Artificial Intelligence Algorithm that parses the major news publications for news articles, groups them based on the main story, and presents them in order of importance. ^[1] It was design to become a news aggregator at first, but since important stories are covered be most publications, an Artificial Intelligence algorithm was implemented in order to group such articles.

Opinion Extraction

To do a sentiment analysis on each article, the algorithm used a derivative of SentiWord ^[2], but it quickly became clear that the library was not created to be used in a news Environment. So a new library was developed, that would better suit the case it was being used in. It manages to provide a good estimate of the impact that the text of an article has (see image 1).

Abbildung in dieser Leseprobe nicht enthalten

Image 1. An example of an article’s Sentiment Analysis Result. Partial screenshot of http://newsinn.org/display/mostnegativeweek/ on 2015/10/18

ABOUT STOCK INDEXES

Definition

A stock index or stock market index is a measurement of the value of a section of the stock market. It is computed from the prices of selected stocks (typically a weighted average). It is a tool used by investors and financial managers to describe the market, and to compare the return on specific investments. ^[3]

Theory

Since the stock market is heavily affected by world problems, and therefore, world news, it is highly likely that a general sentiment analysis of the news would produce a result directly related to the general sentiment of the public on the world affairs. This would theoretically, in time, affect the stock market. The time-to market of world news we estimated to be of a few days, since only huge global events have the power to change the stock price in-day.

Therefore we started testing at a one-day offset, then went on for two, three, and even four days offset between the news data and the stock data. We only kept the data that provided the highest accuracy in predicting the stock market indexes flow.

Data Collection

Taking into account the fact that the final version of the NewsInn A.I. Algorithm started gathering information, on the 14th of September, the NewsInn algorithm checked more than 24,000 articles against the Opinion Extraction library. We used this data to gather an average opinion score of the all the news written that day.

The opinion, however, was extracted from the conglomerated articles. We found no resemblance between the raw article opinion and the indexes. The explanation relies probably in the idea that people only register one event, even if, in some cases, it is detailed in more than 20 articles.

The Data extracted can be found below. Please note that week-end data was removed, as it is not highly accurate (contains less than half the number of the articles that a week day has), and the fact that Stock Markets are closed during week-ends.

Abbildung in dieser Leseprobe nicht enthalten

Table 1. Result of the Average Opinion of Conglomerated Articles (excluding week-ends)

We used the historical prices of Dow Jones Index , the S&P 500 Index and the NASDAQ Index for analysis inside of the graphs, while offsetting them by two days in the case of Dow Jones and NASDAQ, and one day for S&P 500, to accommodate the prediction time of the algorithm. ^[4]

We arrived at the following data, normalized for ease of use:

Abbildung in dieser Leseprobe nicht enthalten

Table 2. Table used to plot the graphs and extract data

Were data could not be found, we duplicated the last known data. That is, the last row of the table. But this row was not taken into account when doing the calculations. Please see below in blue (on the right).

RESULTS

Dow Jones Index

The prediction, two days in advance , for the Dow Jones Index had the highest degree of accuracy, with an average prediction rate of 69.44%. For 4 of the days, for unknown reasons, the prediction was offset by a day, so it could only predict the moving of the index in a one day interval. This was probably due to the refugee crisis.

For the offsetted case we divided the number of days correctly predicted by a factor of two.

Abbildung in dieser Leseprobe nicht enthalten

Image 2. Results of Dow Jones Prediction Two Days Ahead

Abbildung in dieser Leseprobe nicht enthalten

NASDAQ Index

The smallest amount of success in predicting the Stock Market Indexes was for NASDAQ. this index as well had an almost 100% rate of prediction for the last week. Since no data was available for the 23rd of October, the last day of the graph was not taken into consideration.

Abbildung in dieser Leseprobe nicht enthalten

Image 3. Results of NASDAQ Prediction Two Days Ahead

Abbildung in dieser Leseprobe nicht enthalten

S&P 500 Index

The S&P 500 index was predicted with an accuracy rate of 66.66%. That is 2 out of 3 predictions made by the algorithm were correct. As with the previous indexes, the last two weeks showed a very high accuracy, missing only two out of 16 days. The issue with this was that it could only predict the Stock Market fluctuation only a day ahead.

Abbildung in dieser Leseprobe nicht enthalten

Image 4. Results of Dow Jones Prediction Two Days Ahead

Abbildung in dieser Leseprobe nicht enthalten

CONCLUSIONS

Abbildung in dieser Leseprobe nicht enthalten

As predicted, Stock Markets are heavily influenced by the news, and therefore, a sentiment analysis on them can indeed predict the fluctuation with a medium degree of accuracy. For our experiment, the medium prediction was the following

Abbildung in dieser Leseprobe nicht enthalten

All indexes had an almost 100% rate of prediction for the last week. Since the algorithm is not yet in its final form, we can expect these numbers to go higher. It also helps the fact that the more data goes into the system, the more the NewsInn Algorithm learns, and the better associations it can do.

FURTHER RESEARCH

Abbildung in dieser Leseprobe nicht enthalten

In order to thoroughly test the Artificial Intelligence Algorithm, and its prediction on the stock market, we will need to wait some more time, so that it can collect more news data. Even if it was tested against 24,000 articles, we do not feel this has enough significance. We will keep monitoring and hope that, in six months’ time, NewsInn will have collected enough information to properly test it.

SOURCES

Abbildung in dieser Leseprobe nicht enthalten

[...]

^[1] NewsInn Website http://newsinn.org/

^[2] SentiWordNet http://sentiwordnet.isti.cnr.it/

^[3] Wikipedia - Stock market index page https://en.wikipedia.org/wiki/Stock_market_index

^[4] Yahoo Finance Historical Quotes https://uk.finance.yahoo.com/q/hp?s=%5EDJI