Improving the Power of Predicted

Transcription

1 Improving the Power of Predicted Surprise with TRNA News Sentiment StarMine Research Note Nathan Meixler The StarMine SmartEstimate and Predicted Surprise have long been reliable indicators for anticipating future analyst revisions and predicting the direction of actual surprises at report time. In this research note we use the Thomson Reuters News Analytics (TRNA) dataset to further that predictive ability. Filtering on Predicted Surprise, we analyzed the direction of future consensus revisions in reaction to significantly positive or negative news events. We found that the combination of trailing news sentiment and Predicted Surprise gives us a powerful tool to accurately predict the direction of both future consensus revisions and actual earnings surprises. Our main findings are as follows: Combining news sentiment with Predicted Surprise significantly improves upon our ability to predict the direction of consensus revisions from either news sentiment or Predicted Surprise alone. The combination of Predicted Surprise and trailing news sentiment helps us accurately predict the direction of quarterly earnings actual surprises. The combination is more accurate than Predicted Surprise or trailing news sentiment alone. These findings are exploitable to create profitable trading strategies. In one such example, we implement a long/short strategy which achieves 4% annualized returns over a 0 year period. REUTERS/Stephen Hird MAY 04

2 Introduction The StarMine SmartEstimate has a long and proven track record of being a more accurate prediction of actual earnings than the oft-cited consensus. Similarly, the Predicted Surprise, defined as the percent difference between the Smart Estimate and I/B/E/S consensus estimate, has shown to be a reliable leading indicator of future analyst revisions and actual earnings surprises. Past research indicates that when the magnitude of Predicted Surprise is greater than %, the Predicted Surprise forecasts the direction of FY actual surprise to nearly 70% accuracy (Stauth & Bonne 009). Much of the predictive power of The SmartEstimate comes from two main differences between it and the I/B/E/S consensus. First, by detecting clusters of recent analyst revisions, the SmartEstimate excludes stale estimates that occurred prior to a revision cluster. Second, the SmartEstimate incorporates the analyst s track record such that historically more accurate analysts bear more weight in the final aggregation (Bonne et al. 007). For this research note we investigate if we can improve upon the accuracy of the SmartEstimate by incorporating knowledge about the information to which analysts have reacted. We identify positive and negative news events using the Thomson Reuters News Analytics (TRNA) data set and examine how the consensus reacts to that news when the magnitude of Predicted Surprise is significantly high or low. We find that when conditioned on trailing news sentiment, the Predicted Surprise is able to more accurately anticipate the direction of consensus revisions and actual surprises. Finally we use this knowledge to construct a simple trading signal and present our results. Thomson Reuters News Analytics Thomson Reuters News Analytics (TRNA) is a uniquely powerful natural language processing technology capable of automatically analyzing news within milliseconds and scoring it across multiple dimensions such as sentiment, relevance, and novelty (TRNA Revolutionizing News Analysis Brochure). The sentiment engine analyzes the text of any news article and individually scores the mentioned companies based on the sentiment of the related text. In this fashion, the sentiment engine outputs a 0- probability that the story is positive (sent_pos), a 0- probability that the story is neutral (sent_neut), and a 0- probability that the story is negative (sent_neg) where sent_pos + sent_neut + sent_neg = (TRNA Output Image Format Users Guide). Our study makes use of a net sentiment value constructed as sent_pos sent_neg where a net sentiment of - represents extreme negative news and a value of + represents extreme positive news. The TRNA data set also includes a relevance data item scored on a zero to one scale with higher values indicating the news article is more relevant to a particular company and lower values indicating that the news item is only minimally relevant to that company. For our analysis, we chose to only look at highly relevant novel Reuters news items. Methodology Instead of examining the intraday impact of individual news items, we constructed news events by aggregating the net sentiment of an individual company to the daily level. This aggregation is an average of (sent_pos sent_neg) for the relevant articles on each day. We then ranked all net sentiment values across the entire news universe on a -00 scale, with the highest values representing the most positive sentiment on that given day. For this research note, we refer to any event with a daily score 9 (top decile) simply as positive news. Similarly, we refer to any event with a score 0 (bottom decile) as negative news. Our study looked solely at FQ EPS estimates for S&P 500 index constituents between 003 and the third calendar quarter of 03. Here, FQ refers to the fact that the estimates under consideration pertained to the most recent unreported quarter. We obtained consensus estimates and company actuals from the I/B/E/S data set (Methodology for Estimates). Excluded from analysis were any news events for companies that underwent a stock split in the same quarter; less than % of the news events were omitted for this reason. To examine the reaction in the EPS consensus (i.e. the mean of all analysts estimates for the quarter), we measured the percent of cases experiencing an upward revision, downward revision, or no revision after positive or negative news. For this study, we only included news events that occurred after the previous quarter s report date and prior to the current quarter s period end date. To avoid counting small fluctuations in the consensus, we set a revision threshold of ± $.0 ( cent). We also measured the percent of cases revising when the Predicted Surprise was significantly positive or negative, specifically, when the Predicted Surprise was % or -%. For the remainder of this research note, we simply refer to these regimes as positive Predicted Surprise and negative Predicted Surprise with the inherent notion of this ±% threshold. Note that the events we analyzed are fairly rare. On average, less than 5% of S&P 500 companies have any news events which meet our criteria on any given day. When we consider only those events which occurred in the presence of significantly positive or negative Predicted Surprise we are left with only a smattering of stocks. On average, there are approximately 9 negative news events each month when Predicted Surprise is also negative and 5 positive news events each month when Predicted Surprise is also positive. Although, these events are infrequent in any given month, our study was conducted over a 0 year time period leaving us with a hearty population of events to examine. For our second analysis, we tested the Predicted Surprise hit rate when conditioned on trailing news sentiment. That is, we counted the number of times a positive Predicted Surprise preceded a positive actual surprise (actual > consensus) and the number of times a negative Predicted Surprise preceded a negative actual surprise (actual < consensus). In this section, our news signal was aggregated from the trailing net sentiment of varying periods between and 360 market days. Specifically, we took news items where Relevance.9, Item_type!= Alert, Attribution= RTRS and lnkd_cnt5.

3 In order to measure the hit rate of the Predicted Surprise throughout the span of a quarter the values of Predicted Surprise and actual surprise were sampled from the last trading day of every month. That is, the Predicted Surprise is calculated as: SmartEstimate Month End Consensus Month End MAX(.,ABS(Consensus Month End)) And the actual surprise is calculated as: Actual Report Date Consensus Month End MAX(.,ABS(Consensus Month End)) In the above two equations the absolute value of the consensus is capped at 0 cents to avoid undo influence to small denominators. Included as hits were cases where Predicted Surprise % preceded actual surprises > 0 and cases where predicted Surprise -% preceded actual surprises < 0. The hit rate was then aggregated from all month-end samples as: Total Number of Hits Total Number of Observations Reaction to Positive News Figure shows the aggregate effect of,70 positive news events on the EPS consensus given no other conditions. After 30 days the percent of cases revising up (7%) was almost identical to the percent of cases revising down (8%). When we conditioned on positive Predicted Surprise we were left with,360 total events. In these cases, upward revisions were much more likely. We see upward revisions occur 40% of the time after 30 days and downward revisions only 3% of the time. Upward revisions are 3 times more likely in this scenario than downward revisions. Reaction to Negative News In contrast, Figure 3 shows the impact of,659 negative news events on the consensus without incorporating any Predicted Surprise information. In these examples 33% of cases had revised downward after 30 days compared to only % revising upward. In Figure 4, we see the analyst s reaction to the,80 negative news events when Predicted Surprise was also negative. Here downward revisions occurred 64% of the time and upward revisions only 7% of the time. In this situation, a downward consensus revision is 9 times more likely than an upward revision. Consensus Revisions After Positive News Upward Downward No Revision Consensus Revisions After Positive News When Predicted Surprise is Positive Upward Downward No Revision Figure. Upward and downward revisions are equally likely after positive news. Figure. Upward revisions are more likely after positive news when Predicted Surprise is also positive. Consensus Revisions after Negative News Upward Downward No Revision Consensus Revisions after Negative News When Predicted Surprise is Negative Upward Downward No Revision Figure 3. Downward revisions are more likely after negative news. Figure 4. Likelihood of downward revisions increases to 64% when Predicted Surprise is also negative. 3

4 Testing the Combination of News Sentiment and Predicted Surprise As we have seen, when the Predicted Surprise is in the same direction as the news sentiment, the analysts are much more likely to revise in line with the news. This is not entirely unexpected, though, as we already know that Predicted Surprise alone is a good indicator of future consensus revisions. To test whether the news sentiment is additive we examined the reaction to different types of news when the Predicted Surprise is held positive or negative and compare these results to the unconditioned news case. Here the unconditioned news case refers to the analysts reaction to any news when the Predicted Surprise is held positive or negative. Figures 5 and 6 show clear stratification in analysts reactions to the various types of news. We see that when the Predicted Surprise is positive we are most likely to see upward revisions in the presence of positive news, and are least likely to experience upward revisions in the presence of negative news. Similarly when the Predicted Surprise is negative we are most likely to experience downward revisions in the presence of negative news and least likely to see downward revisions in the presence of positive news. Though these results are not altogether surprising, the differing reactions to varying news sentiment provides evidence we are seeing a real effect and not just the revisions expected from the value of the Predicted Surprise alone. Downward Consensus Revisions After News When Predicted Surprise is Negative Positive News Negative News No Conditioning Upward Consensus Revisions After News When Predicted Surprise is Positive Positive News Negative News No Conditioning Figure 5. When Predicted Surprise is negative, negative news leads to the most downward revisions. Figure 6. When Predicted Surprise is positive, positive news leads to the most upward Revisions. Predicting Actual Surprise As we observed in the previous section, consensus reaction to positive or negative news is easier to anticipate when the Predicted Surprise is also positive or negative. That is, when the Predicted Surprise and news sentiment agree in direction, we are most likely to see consensus revisions in the same direction. We also were curious to see if there is an increased ability to predict the direction of quarterly EPS actual surprise when the trailing news sentiment and Predicted Surprise agree. To test this idea, we constructed six different news sentiment signals by varying the trailing aggregation periods from to 360 market days. We then calculated a hit rate when filtering to the top/bottom deciles, quintiles, and halves of trailing news sentiment. For comparison, we also included the hit rates for cases where there was any news and for cases where there was no news. We break out performance for positive Predicted Surprise, negative Predicted Surprise and the combination of the two in the following tables. Unconditioned hit rates are provided in the table headings for comparison. News Sentiment Decile New Sentiment Quintile News Sentiment Half ANY News Sentiment NO News Sentiment Positive Predicted Surprise Only (Unconditioned HitRate =.67 ) Trailing News Aggregation Period Table. Variations of trailing news sentiment help improve hit rates of predicting actual surprise. For example, when 7 day trailing news sentiment is in the top quintile, a positive Predicted Surprise precedes a positive actual surprise 74% of the time, as compared to an unconditioned hit rate of 67%. 4

5 thomson reuters financial and risk ManageMent in general, we see that predicted Surprise combined with trailing news sentiment does a better job of anticipating actual surprises than the predicted Surprise alone. for example, with a 7 day lookback and conditioning on top/bottom sentiment deciles, we see that our positive surprise hit rate improves from 67% to 76% and our negative surprise hit rate improves from 54% to 64%. note that the magnitude of improvement over the unconditioned case is similar for both the positive and negative predicted Surprise. We also see that the unconditioned hit rate (60%) is well under the previously mentioned 70% hit rate reported in the Stauth study. this discrepancy is entirely caused by the fiscal period under examination. Whereas the Stauth study considered the hit rate for annual fy eps predicted Surprise, we analyzed hit rates for a quarterly fq eps predicted Surprise. two other noteworthy findings are that:. conditioning on trna quintiles seems to perform almost as well as conditioning on trna deciles and. there seems to be at least some value conditioning on trailing news sentiment as far back as 360 market days. News Sentiment Decile New Sentiment Quintile News Sentiment Half ANY News Sentiment NO News Sentiment Negative Predicted Surprise Only (Unconditioned Hit Rate =.54) Trailing News Aggregation Period News Sentiment Decile New Sentiment Quintile News Sentiment Half ANY News Sentiment NO News Sentiment Positive and Negative Predicted Surprise (Unconditioned Hite Rate =.60) Trailing News Aggregation Period table. Variations of trailing news sentiment help improve hit rates of predicting actual surprise. for example, when 7 day trailing news sentiment is in the bottom quintile, a negative predicted Surprise precedes a negative actual surprise 64% of the time, as compared to an unconditioned hit rate of 54%. table 3. hit rates of predicting actual surprise when predicted Surprise is positive or negative. Many variations of trailing news aggregation period and quantile help improve the hit rate over the unconditioned 60%. case example: Jc penney And the bad news bears Unfortunately for Jc penney shareholders, the last two years have not been very kind. the company had been plagued by decreasing consumer traffic, debt issues, and a carousel of executives unable to spark a robust turnaround. over this period, the stock price declined nearly 75% as the company had been generating more bad news than sales growth. here we take a closer look at Q4 fy0 and Q fy03. for both of these periods the predicted Surprise was significantly negative, averaging -95% and -6% throughout the January and april quarters respectively. in the figure, we see the Smartestimate significantly lower than the consensus and the consensus steadily revising down throughout the quarter. We also see that when earnings were reported, the misses were severely negative (-000% and -55%). Price Change (%) Consensus, SmartEstimate, and Actuals JC Penney November 0 May 03 Q4 FY 0 Q FY 03 D J F M A M EPS ($) Mean SmartEstimate Price S&P 500 Reported Actuals figure 7. Jcp experienced significant downward revisions and negative actual surprises in the quarters ending January 3st and april 30th 03. Quarters ending on January 3st 03 and April 30th 03. 5

6 What this plot doesn t show is the abysmal amount of negative company news released over these same two quarters. Throughout this time period JCP s daily news sentiment was in the bottom decile 7 times, in the bottom quintile 8 times and in the lower half of news sentiment on 35 occasions. In the chart below, we see a selection of this extreme negative sentiment in the news throughout these two quarters. A Selection of Negative News JC Penney November 0 May 03 Date Headline Net Sentiment /9/ JC Penney options volume swells as shares drop /3/3 JC Penney down.6 percent in premarket trading //3 JC Penney falls in premarket after UBS downgrade /5/3 JC Penney bonds, CDS shrug off default notice -0.6 /7/3 JC Penney 4th-quarter same-store sales fall 3.7 percent /8/3 JC Penney shares fall after sales plunge /5/3 Vornado sale of JC Penney shares adds to growing pressure on CEO /8/3 York s Dinan squares off against Ackman over JC Penney //3 JC Penney shares shoot up on talk CEO may leave /0/3 JC Penney says turnaround could take longer - 3/8/3 Put play on JC Penney suggests more downside /9/3 No quick fixes expected at JC Penney from returning CEO /9/3 JC Penney exposes inefficiency valuing CEOs /0/3 JCP change in CEO will not solve near term earnings pressure /0/3 Three more top executives leave JC Penney //3 Martha Stewart loses bid to dismiss Macy s contract claim /5/3 Macy s appeals against ruling on Martha Stewart goods in JC Penney /7/3 JC Penney says st quarter same-store sales down 6.6 percent Although the negative news produced subsequent consensus revisions, the analysts severely under-reacted to the quantity of negative information hitting the wire. Incorporating the news sentiment would have given us a powerful confirmatory signal to limit any upside exposure to this stock. Trading Strategy and Performance Calculations As we have seen, the combination of news sentiment and Predicted Surprise gives us a powerful tool for determining the direction of future consensus revisions and the direction of the actual surprise. With the combination of news sentiment and Predicted Surprise shown to be synergistic, we test two simple low frequency long-short strategies designed to capitalize on this effect. For our benchmark we create a long portfolio of stocks with high Predicted Surprise and a short portfolio of stocks with low Predicted Surprise. To build our TRNA conditioned portfolios we incorporate a TRNA Signal with a 30 day lookback. That is, we construct a long portfolio of stocks with a positive Predicted Surprise and trailing 30 day TRNA net sentiment in the top decile/quintile with the idea that these stocks will have a tendency to experience positive upward revisions and positive actual surprises. We take short positions in stocks with a negative Predicted Surprise and trailing 30 day sentiment in the bottom decile/quintile with the idea that these stocks will be more likely to experience negative consensus revisions and be more prone to earnings misses. We rebalance monthly on the st of the month. Our results are presented in Table 5. Table 4. A selection of negative news for JC Penney. Summary of Backtest Results for Long-Short Strategies Spread Long Portfolio Short Portfolio Predicted Surprise Only Predicted Surprise + TRNA Quintile Predicted Surprise + TRNA Decile Sharpe Ratio Annualized Return Sharpe Ratio Annualized Return Portfolio Size (Avg) Sharpe Ratio Annualized Return Portfolio Size (Avg) % % % % % % % % % 4 Table 5. A Summary of Backtest Results 003 Q3 03 6

7 Over the year period our Predicted Surprise + TRNA decile portfolio was able to generate an annualized return of 4.6%. This dramatically beats the.5% return of the benchmark Predicted Surprise only portfolio. During this time period we see that the majority of our performance can be attributed to the long portfolios, with our short portfolio actually generating a positive annualized return. Most readers will notice the small portfolio sizes for these strategies. As previously mentioned, stocks with such extreme values of news sentiment and Predicted Surprise are rare. As such, we are trading an average of stocks every month when conditioned on deciles of trailing news sentiment and 4 stocks every month when conditioned on quintiles of news sentiment. Due to the relatively small portfolio sizes this signal may be more useful as a tilt/weighting factor rather than as a standalone signal. Equal Weighted S&P500 Predicted Surprise + TRNA Weighted S&P 500 Long Only S&P 500 Strategies Sharpe Ratio Annualized Return % % TABLE 6. A simple news sentiment + Predicted Surprise weighting scheme increases the Sharpe Ratio and annualized returns over the equal weighted S&P 500. In Table 6 and Figure 8, we present results from one such long-only strategy differentially weighting S&P 500 stocks based on our news sentiment + Predicted Surprise signal. Starting with an equal weighted S&P 500, we remove any stock from our portfolio which had a negative Predicted Surprise and trailing 30 day news sentiment in the bottom quintile. Any stock which exhibited a positive Predicted Surprise and trailing 30 day news sentiment in the top quintile we overweighed by a factor of 00. From this simple strategy, we are able to show outperformance over the equal weighted S&P 500 in both annualized return and Sharpe ratio. Cumulative Returns (%) Cumulative Returns S&P 500 Using Two Different Weighting Schemes Equal Weighted S&P 500 News Sentiment + Predicted Surprise Weighted S&P 500 //03 //04 //05 //06 //07 //08 //09 //0 // // //3 Figure 8. A simple news sentiment + Predicted Surprise weighting scheme outperforms the equal weighted S&P 500. Conclusions We began our Research Note by expounding the accuracy of the StarMine SmartEstimate and Predicted Surprise. Still, we questioned whether News Sentiment and the SmartEstimate would naturally enhance each other. By examining the sentiment of recent news stories and conditioning on regimes of Predicted Surprise, we found the value of these two data sets to be additive. Although Predicted Surprise has been a solid indicator in the past, we find that conditioning on trailing news sentiment leads us to more accurately predict the direction of consensus revisions and the directions of actual surprise. Further, we were able to use this information to develop simple, yet profitable low frequency trading strategies. These data are available through Thomson Reuters via ftp, data feeds, or in our Eikon desktop solution. We are confident that these data can add value to current investment research process of our readers and encourage them to contact StarMine quantitative consulting for more information (starmine.quantconsulting@thomsonreuters.com). 7

8 Citations Bonne, G., et al. 007, StarMine Analyst Revisions Model (ARM), StarMine Whitepaper. Stauth, J. and Bonne G., 009, SmartEstimates and the Predicted Surprise: Construction and Accuracy Thomson Reuters News Analytics Revolutionizing News Analysis, 00, Marketing Brochure. Available at customers.reuters.com Methodology For Estimates, A guide to Understanding Thomson Reuters Methodologies, Terms and Policies for I/B/E/S Estimated Databases, June 03. Available at customers.reuters.com ABOUT THOMSON REUTERS FINANCIAL AND RISK Financial and risk management solutions from Thomson Reuters help investors, traders and compliance professionals overcome their three most pressing challenges: to drive profits; to connect and collaborate via open, secure networks; and to efficiently manage risk and regulation. about thomson reuters Thomson Reuters is the world s leading source of intelligent information for businesses and professionals. We combine industry expertise with innovative technology to deliver critical information to leading decision makers in the financial and risk, legal, tax and accounting, intellectual property and science and media markets, powered by the world s most trusted news organization. With headquarters in New York and major operations in London and Eagan, Minnesota, Thomson Reuters employs approximately 60,000 people and operates in over 00 countries. Thomson Reuters shares are listed on the Toronto and New York Stock Exchanges. For more information about Thomson Reuters, please go to Visit thomsonreuters.com For more information, please contact your Thomson Reuters representative or StarMine Quantitative Consulting at: starmine.quantconsulting@thomsonreuters.com Visit us at: Thomson Reuters 04. All rights reserved /4