Can internet search queries help to predict stock market volatility?

Size: px

Start display at page:

Download "Can internet search queries help to predict stock market volatility?"

Lesley Bradley
5 years ago
Views:

1 Can internet search queries help to predict stock market volatility? Thomas Dimpfl and Stephan Jank Eberhard Karls Universität Tübingen BFS Society Vortragsreihe Tübingen, 4 December 2017 Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 1 / 40

2 Measuring interest Literature Large stock market movements capture investors attention Why should searches be helpful to predict volatility? Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 2 / 40

3 Christmas is coming Internet searches are a measure for individuals interests Measuring interest Literature Large stock market movements capture investors attention Why should searches be helpful to predict volatility? Interest Over Time (in %) Source: Google Trends Plätzchen Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 3 / 40

4 After Christmas Internet searches reflect timing of individuals actions Measuring interest Literature Large stock market movements capture investors attention Why should searches be helpful to predict volatility? Interest Over Time (in %) Plätzchen Diät Source: Google Trends Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 4 / 40

5 The first study that used google trends data predicted influenza epidemics Measuring interest Literature Large stock market movements capture investors attention Why should searches be helpful to predict volatility? Ginsberg, Mohebbi, Patel, Brammer, Smolinski and Brilliant (2009, Nature) Influenza like illnesses (ILI) Reported (red), prediction(black) Source: Ginsberg et al. (2009), Figure 2 Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 5 / 40

6 Google search volume data seem to carry information about what people are interested in Measuring interest Literature Large stock market movements capture investors attention Why should searches be helpful to predict volatility? Prediction of unemployment rates (Choi and Varian, 2009a) Prediction of retail sales (Choi and Varian, 2009b) Source: Choi & Varian (2009b), Figure 2.3 Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 6 / 40

7 There are numerous applications of Google search volume data in financial research Measuring interest Literature Large stock market movements capture investors attention Why should searches be helpful to predict volatility? Measure of retail investors attention: individual stocks (Da, Engelberg, Gao, 2011, Journal of Finance) Weekly stock market volatility (Vlastakis and Markellos, 2012, Journal of Banking & Finance) The sum of all FEARS: Investor sentiment and asset prices (Da, Engelberg, Gao, 2014, The Review of Financial Studies) Investor Pessimism and the German Stock Market: Exploring Google Search Queries (Dimpfl and Kleiman, forthcoming, German Economic Review) Googling Gold and Mining Bad News (Dimpfl and Baur, 2016, Resources Policy) Can we predict the financial markets based on Google s search queries? (Perlin, Caldeira, Santos and Pontuschka, 2017, Journal of Forecasting) Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 7 / 40

8 Large stock market movements capture investors attention Measuring interest Literature Large stock market movements capture investors attention Why should searches be helpful to predict volatility? Realized volatility Realized volatility of the Dow Jones Search queries for the index name Search Volume Index Contemporaneous correlation (RV-SQ):0.82 Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 8 / 40

9 Why should searches be helpful to predict volatility? Measuring interest Literature Large stock market movements capture investors attention Why should searches be helpful to predict volatility? Search queries proxy retail investors attention/interest Agent-based models of stock market volatility Lux & Marchesi (1999, Nature): two agents: fundamentalists and noise traders fundamental price shock noise trading volatility Recent evidence by Foucault et al. (2011, Journal of Finance): Noise traders contribute to volatility (approx. 23%) Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 9 / 40

10 What is Realized volatility The dataset Search term Dow or S&P 500 Autocorrelation of realized volatilities and search queries Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 10 / 40

11 What is Realized volatility What is Realized volatility The dataset Search term Dow or S&P 500 Autocorrelation simply put: RV is the daily variation of the price of a product, e.g. of a stock formally: RV is the (daily) standard deviation of the log returns of a stock Realized variance: RVar t = withn t the number of intervals per day n t i=0 r 2 t,i Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 11 / 40

12 Squared returns as a variance estimator typical model for returns: What is Realized volatility The dataset Search term Dow or S&P 500 Autocorrelation r t = h t η t withη t i.i.d. N(0,1) On every day, prices are observed at every point in time τ i,τ i {τ 0,...,τ nt } during the day thenp t,i (i = 1,...,n t ) is thei-th observation on dayt return between two intradaily points of timeiandi 1: r t,i = h t,i η t,i withη t,i N(0, 1 n t ) r t,i = p t,i p t,i 1 Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 12 / 40

13 Squared returns as a variance estimator What is Realized volatility The dataset Search term Dow or S&P 500 Autocorrelation Hence, we get daily values as r t = n t i=0 r t,i h t = 1 n t n t Aim: show thate [ r 2 t F t,0 ] = ht holds i=1 h t,i Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 13 / 40

14 Squared returns as a variance estimator What is Realized volatility The dataset Search term Dow or S&P 500 Autocorrelation r 2 t = = ( nt i=0 n t i=0 E [ [ rt F 2 ] nt 0 = E r t,i ) 2 r 2 t,i +2 i=0 n t 1 i=0 r 2 t,i F 0 ] }{{} RVar +2E n t 1 i=0 n t j=t+1 n t j=t+1 r t,i r t,j r t,i r t,j F 0 Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 14 / 40

15 Squared returns as a variance estimator What is Realized volatility The dataset Under the assumption that returns are uncorrelated, we get E [ r 2 t F t,0 ] = E[RVar Ft,0 ] = h t Search term Dow or S&P 500 Autocorrelation and finally RV t = RVar t Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 15 / 40

16 The dataset What is Realized volatility The dataset Search term Dow or S&P 500 Autocorrelation Stock market index: Dow Jones Sample: daily data (trading days) 5 1/2 years, July December 2011 Realized volatility (estimated on 10 min intervals) Search queries for index name in the US: dow Timing: searches measured from 12 pm to 12 pm Pacific Standard Time corresponds to 3 am Eastern Standard time, i.e. 6.5 hours before opening of the NYSE Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 16 / 40

17 Dow is the appropriate search term Search terms correlated with dow Rank Correlation Scale Term dow jones djia dow stock dow close current dow dow jones industrial current dow jones google dow stock market now industrial average Search queries Search queries comparison: Dow Dow Dow Jones Dow Jones Industrial Average DJIA DJI Issues with low search volume: Lower accuracy Missing values at daily frequency (threshold) Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 17 / 40

18 Why we use the Dow and not the S&P 500 What is Realized volatility The dataset Search term Dow or S&P 500 Autocorrelation Search queries Search queries comparison: Dow vs. S&P Dow S&P 500 Weekly data, correlation ( Dow - S&P 500 )0.76 Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 18 / 40

19 Autocorrelations of realized volatility and search queries What is Realized volatility The dataset Search term Dow or S&P 500 Autocorrelation Autocorrelations ACF: DJIA Realized volatility Autocorrelations ACF: DJIA Search queries Lag Lag Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 19 / 40

20 Dynamics of RV and SQ Dynamics of SQ and trading volume Analysis of the joint dynamics in a vector autoregressive model Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 20 / 40

21 Dynamics of realized volatility and search queries VAR estimation: log-rv t log-sq t Dynamics of RV and SQ Dynamics of SQ and trading volume log-rv t (0.000) (0.057) log-rv t (0.000) (0.582) log-rv t (0.005) (0.384) log-rv t (0.000) (0.352) log-sq t (0.000) (0.000) log-sq t (0.347) (0.275) log-sq t (0.299) (0.021) log-sq t (0.993) (0.002) Constant (0.000) (0.139) Granger causality test: log-rv log-sq log-rv 5.97 (0.201) log-sq (0.000) Granger-causality: SQ predicts RV Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 21 / 40

22 Dynamics of search queries and trading volume Dynamics of RV and SQ Dynamics of SQ and trading volume VAR estimation: log-sq t log-vo t log-sq t (0.000) (0.000) log-sq t (0.320) (0.144) log-sq t (0.004) (0.793) log-sq t (0.002) (0.066) log-vo t (0.185) (0.000) log-vo t (0.400) (0.000) log-vo t (0.013) (0.000) log-vo t (0.587) (0.005) Constant (0.060) (0.000) Granger causality test: log-sq log-vo log-sq (0.000) log-vo (0.029) Granger-causality: SQ predicts Volume Volume predicts SQ Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 22 / 40

23 Modeling RV and SQ criteria In-sample Out-of-sample Out-of-sample forecast: Robustness across different times of volatility Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 23 / 40

24 Modeling realized volatility and searches Modeling RV and SQ criteria In-sample Out-of-sample Out-of-sample forecast: Robustness across different times of volatility AR and VAR model of realized volatility (RV) and search queries (SQ): log-rv t = c 1 + p β 1,j log-rv t j +γ 1,1 log-sq t 1 +ε 1,t j=1 log-sq t = c 2 +β 2,1 log-rv t 1 + q γ 2,j log-sq t j +ε 2,t. HAR and V-HAR model of realized volatility (RV) and search queries (SQ): log-rv t = c 1 +β d log-rv t 1 +β w log-rvt 1 w +β m log-rvt 1 m +γ 1,1 log-sq t 1 +ε 1,t q log-sq t = c 2 +β 2,1 log-rv t 1 + γ 2,j log-sq t j +ε 2,t. j=1 One-step ahead predictions: inclusion of SQ as an explanatory variable sufficient. Multi-step predictions: It is necessary to model and forecast SQ as well. j=1 Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 24 / 40

25 criteria Modeling RV and SQ criteria In-sample Out-of-sample Out-of-sample forecast: Robustness across different times of volatility Mean squared error and quasi likelihood loss function: MSE = (RV t+1 RV t+1 t ) 2, QL = RV t+1 RV t+1 t log RV t+1 RV t+1 t 1, (robust to possible noise in the volatility measure) Minzer-Zarnowitz RegressionR 2 : RV t+1 = b 0 +b 1 RV t+1 t +e t+1. Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 25 / 40

26 In-sample forecast evaluation Modeling RV and SQ criteria In-sample Out-of-sample Out-of-sample forecast: Robustness across different times of volatility Model: MSE QL R 2 AR(1) AR(1) + SQ 0.160** AR(4) AR(4) + SQ 0.145* HAR HAR + SQ 0.146* HAR model: best among the univariate models Including SQ improves forecast for each univariate model HAR + SQ: best performing model Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 26 / 40

27 Out-of-sample forecast evaluation Initial estimation window: 500 trading days (July 2006 to June 2008) Out-of-sample period: high volatility phase of financial crisis 1 day 1 week 2 weeks Model: MSE QL R 2 MSE QL R 2 MSE QL R 2 AR(1) RV VAR(1) RV, SQ 0.224** ** ** AR(4) RV VAR(4) RV, SQ 0.191** ** ** HAR RV VHAR RV, SQ 0.194* * * Intuition: Shock of searches on volatility quite persistent. Good fit of time-series model for searches allows to iterate the system forward. Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 27 / 40

28 Out-of-sample forecast: Robustness Modeling RV and SQ criteria In-sample Out-of-sample Out-of-sample forecast: Robustness across different times of volatility Delayed publication of search query data (1 day) Focus of this paper: dynamics of volatility and attention Technically possible to publish search volume even faster (Google Hot Trends) Model: MSE QL R 2 AR(1) RV VAR(1) RV, SQ 0.214** AR(4) RV VAR(4) RV, SQ 0.196* HAR RV VHAR RV, SQ Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 28 / 40

29 across different times of volatility: in-sample prediction Modeling RV and SQ criteria In-sample Out-of-sample Out-of-sample forecast: Robustness across different times of volatility Realized Volatility Bottom Quantiles 1% 5% 10% 25% 50% MSE: HAR MSE: HAR + SQ Difference in MSE QL HAR QL HAR + SQ Difference in QL Realized Volatility Top Quantiles 50% 25% 10% 5% 1% MSE: HAR MSE: HAR + SQ Difference in MSE QL HAR QL HAR + SQ Difference in QL Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 29 / 40

30 across different times of volatility: Out-of-sample forecast Modeling RV and SQ criteria In-sample Out-of-sample Out-of-sample forecast: Robustness across different times of volatility Realized Volatility Bottom Quantiles 1% 5% 10% 25% 50% MSE: HAR MSE: VHAR (RV,SQ) Difference in MSE QL HAR QL VHAR (RV,SQ) Difference in QL Realized Volatility Top Quantiles 50% 25% 10% 5% 1% MSE: HAR MSE: VHAR (RV,SQ) Difference in MSE QL HAR QL VHAR (RV,SQ) Difference in QL Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 30 / 40

31 of Volatility Timing of Volatility Timing of Volatility Timing of Volatility Timing or: can you earn money with this stuff? Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 31 / 40

32 of Volatility Timing Quadratic utility of an investor: U(W t+1 ) = W t r p,t+1 aw2 t 2 r 2 p,t+1 of Volatility Timing of Volatility Timing of Volatility Timing r p,t+1 : return of the investor s portfolio a absolute risk aversion Portfolio consists of a risky asset (market portfolio) and a risk-free asset Investor implements a variance targeting strategy (σ 2 p = 12%): Int: investor derives weights Int+1: portfolio gains and losses are realized Variance prediction based on either HAR model of realized volatility or HAR model including Google search queries Compare managed investment strategies to static buy and hold Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 32 / 40

33 of Volatility Timing of Volatility Timing of Volatility Timing of Volatility Timing Assessment of economic gains: r A,t andr B,t : returns of two alternative portfolios : maximum fee investor is willing to pay to switch from portfolio A to portfolio B Compare utility of the two investments to find : T 1 t=0 T 1 = [ (r A,t+1 ) t=0 [ r B,t+1 ] γ 2(1+γ) (r A,t+1 ) 2 ] γ 2(1+γ) r2 B,t+1 Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 33 / 40

34 of Volatility Timing of Volatility Timing of Volatility Timing of Volatility Timing Additional gain from including Google search queries: Different levels of relative risk aversionγ Performance fee paid for search query based portfolio is robust with respect to risk aversion relative risk aversionγ = (50-HAR) (50-HAR+SQ) (HAR-HAR+SQ) Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 34 / 40

35 Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 35 / 40

36 VAR analysis: Results are consistent with an agent-based/noise trader theory of (additional) volatility: Co-movement of volatility and retail investors attention Search queries predict volatility Search queries predict volume Search queries are a valuable source of information for future volatility. Forecast improvements... in-sample out-of-sample longer forecast horizons esp. in high volatility phases (e.g. the financial crisis of 2008) Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 36 / 40

37 Thank you. Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 37 / 40

38 Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 38 / 40

39 (1) Baur, D. G., Dimpfl, T., Googling gold and mining bad news. Resources Policy 50, Choi, H., Varian, H., 2009a. Predicting initial claims for unemployment benefits. Working paper Choi, H., Varian, H., 2009b. Predicting the present with Google trends. Working paper Da, Z., Engelberg, J., Gao, P., In Search of Attention. The Journal of Finance 66, Da, Z., Engelberg, J., Gao, P., The Sum of All FEARS Investor Sentiment and Asset Prices. Review of Financial Studies 28, 1 32 Dimpfl, T., Kleiman, V., Investor Pessimism and the German Stock Market: Exploring Google Search Queries. German Economic Review doi: /geer Foucault, T., Sraer, D., Thesmar, D. J., Individual Investors and Volatility. The Journal of Finance 66, Ginsberg, J., Mohebbi, M. H., Patel, R. S., Brammer, L., Smolinski, M. S., Brilliant, L., Detecting influenza epidemics using search engine query data. Nature 457, Lux, T., Marchesi, M., Scaling and criticality in a stochastic multi-agent model of a financial market. Nature 397, Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 39 / 40

40 (2) Perlin, M. S., Caldeira, J. a. F., Santos, A. A. P., Pontuschka, M., Can We Predict the Financial Markets Based on Google s Search Queries? Journal of Forecasting 36, Vlastakis, N., Markellos, R. N., Information demand and stock market volatility. Journal of Banking & Finance 36, Thomas Dimpfl and Stephan Jank Can internet search queries help to predict stock market volatility? 40 / 40

CFR Working Paper NO Can Internet search Queries help to predict stock market volatility? T. Dimpfl S.Jank

CFR Working Paper NO Can Internet search Queries help to predict stock market volatility? T. Dimpfl S.Jank CFR Working Paper NO. 11-15 Can Internet search Queries help to predict stock market volatility? T. Dimpfl S.Jank Can internet search queries help to predict stock market volatility? Thomas Dimpfl and