Decision 411: Class 2

Transcription

1 Decision 411: Class 2 Explanation of lags & differences Random walk model How to identify a random walk Examples of random walks Forecasting from the random walk model Log transformation & geometric random walk Linear trend model Model comparison & validation

2 Explanation of lags & differences A lagged series is just the original series shifted down by one or more periods, so that it lags behind. LAG(Y,1) in row k is the same as Y in row k-1 You can also lag a series by 2 or more periods. Lagged series can be used as autoregressors and leading indicators in regression models. Note that a lagged series has missing values at the beginning, but it extends farther into the future than the original series. A differenced series is the change in the original series from last period to this period (i.e., delta-y ). DIFF(Y) is logically the same as Y LAG(Y,1). It also has a missing value at the beginning.

3 The random walk model A time series is a random walk if its period-to to- period changes are statistically independent & identically distributed ( i.i.d i.i.d. ). ) In each period it takes an independent random step away from its last position If the mean step size is non-zero, it is a random walk with drift (i.e., trend)

4 Analogies Random walk: a drunk person staggering left and right with equal probability while moving forward Random walk with drift: a drunk person with one shoe Continuous random walk ( Brownian motion ): a drunk snail

5 Walking the random walk

6 A random walk with little or no drift often does not look random! It may appear to have trends, cycles, head and shoulders patterns and other interesting features by sheer chance It Ain't the Things You Don't Know That Hurt You, It's the Things You Know... That Ain't So!

7 A related statistical illusion: the hot hand in basketball Many basketball players are perceived as streaky shooters (e.g., Allen Iverson), but statistical analysis shows that the chance of making a given field goal or free throw is roughly independent of what happened on the last few shots: Chance is a very powerful force in creating streaks" See the Hot Hand in Sports website at thehothand.blogspot.com/

8 How to identify a random walk Plot the first difference, i.e., the period-to to-period changes, of the original time series ( delta-y ) If the first difference has constant variance and no significant autocorrelations*,, the original series is a random walk at at least approximately. If the series is logically bounded above or below or has a stable long-run average, then it is not a true random walk, but the random walk model may still be appropriate for short-term term forecasting (at least as a benchmark for comparing fancier models). *to be explained shortly

9 Relation to mean model If a time series is a random walk, then its first difference is described by the mean model. Thus, you should predict that the next change will equal the average change. The average change may or may not be zero, depending on whether there is drift.

10 What s an autocorrelation? A correlation is a number r xy between -11 and +1 that measures the strength of the linear relationship between two variables X and Y The correlation is zero if X and Y are statistically independent If you regress Y on X,, the percent of variance explained ( R squared ) is just the square of the correlation coefficient (r( xy squared). The autocorrelation of Y at lag k,, denoted r k, is the correlation between Y and LAG(Y,k), i.e., the correlation between Y and itself lagged by k periods.

11 Computing & plotting autocorrelations Formula: r k = n t= k+ 1 ( y )( ) t Y yt k Y t= 1 ( y ) t Y An autocorrelation plot is a bar chart of the autocorrelation vs. the lag, i.e., r k vs. k n 2 1 Estimated autocorrelations for adjusted MortgageRate Autocorrelations lag This plot shows a single significant autocorrelation at lag 1.

12 Computing autocorrelations in Excel Statgraphics automatically calculates and plots autocorrelations in its time series procedures, but you can also do this in Excel using the CORREL function For example, suppose you have 100 observations of a time series stored in the range A1:A100 in Excel, then r 1 = CORREL(A1:A99, A2:A100) r 2 = CORREL(A1:A98, A3:A100) r 3 = CORREL(A1:A97, A4:A100) Note that the ranges are offset by 1 period, 2 periods, 3 periods, etc.

13 What does autocorrelation mean? Strong positive [negative] autocorrelation at lag k means that if one observation is above the mean then the k th following observation is also likely to be above [below] the mean. Thus, positive [or negative] autocorrelation measures the tendency for successive observations to fall on the same [or opposite] sides of the mean. Series with consistent trends always have strong positive autocorrelation at small lags (1, 2, ) Series with seasonality have strong positive autocorrelation at the seasonal period (e.g., lag 12)

14 How we use autocorrelations It s usually good to find significant autocorrelations in your original series. This means the past contains clues to the future. It s bad to find significant autocorrelations in your residuals (forecast errors). This means that there is some pattern in the data that your model has not explained. Rough rule of thumb for significant autocorrelation: 95% confidence interval 2 ± n k 1 2 i= 1 i Exact SE( rk) = (1 + r ) / n 1/ n for small k and/or low autocorrelation

15 Autocorrelation & the RW model In a true random walk, there is strong autocorrelation in the original series, but no significant autocorrelation in the first difference of the series at any lags. This means there is no pattern in the data except the change next period will equal the average change Hence the random walk model is sometimes called the naïve model..but it s not really naïve! It says you can t do better than this,, and it has precise implications for the uncertainty (i.e., widths of confidence intervals) surrounding the forecasts at horizons of more than one period ahead.

16 Example: daily dollar/euro FX rate 1999-present: random walk without drift? Time Series Plot for DollarEuroFXday Original series shows erratic behavior, strong positive autocorrelation, peaks and valleys Estimated Autocorrelations for DollarEuroFXday 1 Autocorrelations lag The heights of the bars are the autocorrelations. Here the autocorrelations are all close to 1 because the series tends to remain on the same side of its sample mean for long periods.

17 After a first-difference transformation, autocorrelations are all insignificant, the signature of a random walk 0.001) 25 Time Series Plot for adjusted DollarEuroFXday The variance of the differences also appears to be roughly constant over time. 1 Estimated Autocorrelations for adjusted DollarEuroFXday Autocorrelations lag Right-mouse button options in Describe/Time Series/Descriptive Methods procedure: one order of nonseasonal differencing has been chosen

18 Forecasting from the RW model Random walk without drift: y ˆn+1 = y n i.e., the next forecast equals the last observation. Standard error: n t = SE fcst = 2 Δy 2 t /( n 1) Δy = y y where t t t 1 is the change in Y at period t, i.e., delta-y at time t. Thus, SE SE fcst is is the RMS value of delta-y.

19 Forecasting from RW-with with-drift model Random walk with drift: y y Y ˆn + 1 = n + Δ where n Δ Y = Δy t 2 t n = yn y1 n = /( 1) ( ) /( 1) is the average change between periods. Thus, the next forecast equals the last observation plus the average change.

20 Forecast standard error for RWD model For the RW with drift model, SE fcst is the same as SE fcst for the mean model applied to delta-y: SE fcst fcst Δy Δy ( 1) ΔY 1 SE = s + s / n = s + n 1 where n 2 Δ Y = ( Δ ) /( 2) t= 2 t Δ s y Y n is the sample standard deviation of delta-y Note that the sample size for delta-y is only n 1, hence the denominator in the sample standard deviation calculation is n 2 and a t-test for non-zero drift will have n 2 degrees of freedom.

21 Forecasting more than 1 period ahead RW without drift: (zero trend) y ˆ = n+ k y n RW with drift: (non-zero trend) yˆ n+ k = y n + k Δy In both cases: SE = k SE fcst( k) fcst(1) i.e., the k-period-ahead forecast standard error is larger than the 1-period 1 ahead standard error by a factor of k

22 In plain English... The forecasts from the random walk model are extrapolated as a straight line extending from the last observed data point. In a RW without drift, the line is horizontal. In a RW with drift, it has a non-zero slope equal to the average trend over the whole sample. The standard error of the k-step ahead forecast is the sample standard deviation of delta-y times. Hence confidence intervals widen in proportion to the square root of time ( sideways parabola shape). k

23

24 Why the square root of time rule? In a random walk, the k-step ahead forecast error is the sum of k independent random variables ( steps ) The variance of a sum of independent random variables is the sum of the variances,, hence the variance of the sum of k steps is just k times the variance of one step. The standard deviation is the square root of variance, hence the standard deviation of the k-step- ahead forecast error goes up in proportion to square root of k.

25 SG s User-specified specified forecasting procedure 1. Applies inflation adjustment, math transformation, and seasonal adjustment (if any), in that order. 2. Fits a forecasting model of the specified type to the adjusted data, produces residual plots and computes forecasts in adjusted units. 3. Untransforms the forecasts by undoing the adjustment operations (in reverse order), to obtain & plot forecasts in original units. 4. Computes forecast errors and their statistics in original units. 5. Compares up to 5 models side-by by-side!

26 Model specification options

27 Fitting RW models in Statgraphics The Forecasting/User-Specified Specified-Model procedure includes a Random Walk model type If the Constant box is checked, you get a random walk with drift. If the input variable is logged, or if the Natural log box is checked, you get a geometric random walk ( more about that later )

28 RW forecasts for FX rate DollarEuroFXday Time Sequence Plot for DollarEuroFXday Random Walk Without Drift Confidence intervals for forecasts have parabolic shape actual forecast 95.0% limits Analysis options in Forecasting procedure: RW model with no constant term

29 Updating of RW forecasts Y Random walk with drift Random walk with drift actual actual forecast forecast 95.0% limits 95.0% limits Forecasts into the future are a trend line re-anchored on the last observed data point. Past forecasts look like a plot of the data shifted to the right and slightly up.

30 Here s a RW model without drift for housing starts SAAR. Note that the forecasts extend horizontally, and the confidence interval widen parabolically. (The longer-horizon CI s are probably somewhat too wide: the series seems to have a stable long-run average.) The residual autocorrelations are almost insignificant, indicating that this series is close to a random walk.

31 Here s a random walk model for deflated, seasonally adjusted retail sales data. Note that the fitted values track the historical data closely (just lagging behind by one month), and the trend line is extrapolated from the last observed data point. This report also shows head-to-head comparisons against 2 other models we ll come back to that later

32 How to tell if drift (trend) is non-zero? The drift term often does not matter much to 1-period-ahead forecasts; it shows up only as the trend in longer-horizon forecasts. Ask whether it makes sense that the series should continue to trend upward or downward indefinitely: if not, then assume no drift. It may be difficult to test the hypothesis of zero drift by purely statistical methods (e.g., by looking at the t-statistic of the sample mean of delta-y) ) unless the sample is large.

33 Looking ahead Some of the more sophisticated forecasting models we will meet later (e.g., simple and linear exponential smoothing) are just fancied-up random walk models. The forecast line is extrapolated from the average position of the last few points. Its slope is equal to the average trend of the recent data,, not the whole sample.

34 The logarithm transformation The natural logarithm function looks like this: Y=LOG(X) Y=X Note that the line y = x 1 is tangent to y = LN(x) at x=1. Hence LN(x) x 1 for x 1, and LN(1+r) r when r is a small percentage (e.g., a return)

35 Properties of the natural logarithm transformation The defining property of any logarithm is that LOG(xy xy) ) = LOG(x) ) + LOG(y) Because the natural log has a slope of 1 at x = 1 it converts percentage changes into absolute changes: LN((1+r)x) ) = LN(x) ) + LN(1+r) LN(x) ) + r By the same token, it converts an exponential (compound) growth curve into a linear growth curve LN((1+r) k ) = k LN(1+r) k r

36 Geometric random walk If the log of a series is a random walk, the original series is a geometric random walk. The change in the natural log is (approximately) the percentage change between periods : LN(y t ) LN(y t-1 ) = LN(y t / y t-1 ) (y t / y t-1 ) 1 1 = (y( t y t-1 )/ y t-1! Hence, in a geometric random walk, the series takes random steps in (roughly) percentage terms Percentage change is a more familiar and easy-to-think-about concept, but change-in-the-natural-log is theoretically the right way to measure relative changes when exponential growth is occurring or when small changes are compounded over many periods.

37 Best example: stock prices The geometric random walk is the default model for stock prices* & many other financial assets for which speculative markets exist This means it is hard to beat the market by technical analysis ( charting )... or by fitting regression models to monthly, weekly, or even daily data. *First proposed by Louis Bachelier in 1900, 70 years later it became the basis for the Black-Scholes options pricing model

38 Why should stock prices behave like a geometric random walk? If everyone could predict that the stock market will go up more than average tomorrow, it would have already gone up today,, hence future returns are independent of past returns (and other public information) Investors generally think in terms of percentage changes in stock values when responding to informational events (earnings announcements, interest hikes, etc.), hence volatility is fairly constant in percentage terms.

39 Three forms of random walk ( efficient markets ) hypotheses 1. Weak form: future returns can t be profitably predicted from past histories of returns (e.g., by chartists or data-miners or we in this class) 2. Semi-strong form: future returns can t be profitably predicted from any public information (e.g. by mutual fund managers) 3. Strong form: future returns can t be profitably predicted from any available information (even by insiders) Conventional wisdom: the truth is probably somewhere between semi-strong and strong. Better to buy index funds or throw darts!

40 The market is not a perfect random walk. But any systematic relationships that exist are so small that they are not useful for an investor The history of stock price movements contains no useful information that will enable an investor to outperform a buy-and-hold strategy in managing a portfolio. (p. 151)

41 Example: S&P 500 monthly close Original series shows exponential growth up to crash Differenced series has no autocorrelation but variance is increasing ( heteroscedasticity )

42 Logged S&P 500 monthly close Logged series shows linear growth with bubble in late 90 s Right-mouse button options in Time Series/Descriptive Methods procedure: first let s add a log transformation. Next we will also add a first-difference transformation.

43 Slope of trend in logged data average percentage increase The slope of a trend line fitted to the natural log of the data is (approximately) the average percentage change per unit time. Here the natural log increased by roughly = 3.0 over 25 years, so the average annual increase in the S&P500 index was 3/25 12%

44 Logged and differenced S&P 500 monthly close The first difference of the logged data is (approximately) the percentage change from period to period. Here we see that the percentage changes have no significant autocorrelations and (almost) constant variance over time

45 S&P 500 daily close since ,000 data points!

46 Logged S&P 500 daily close Not a perfect geometric random walk! The lag-1 autocorrelation is about Also, very-long time scale shows periods of greater & lesser volatility.

47 Logged S&P 500 daily close Excluding Black Monday (& Tues. & Wed.), an even clearer pattern emerges. (Actually, the lag-1 daily autocorrelation has faded out in the last 20 years.)

48 Logged DJIA daily close Same pattern!

49 Daily 3-mo. 3 T-bill T rates

50 Logged daily 3-mo. 3 T-bill T rates Here we also see autocorrelation at lag 1, as well as 5 and 10: a dayof-week effect? Also, note the pattern of changing volatility.

51 Another example: weekly gas prices Time Series Plot for GasPrice GasPrice Original series shows erratic behavior, strong positive autocorrelation, peaks and valleys

52 Logged weekly gas prices 5.9 Time Series Plot for adjusted GasPrice adjusted GasPrice Natural log transformation linearizes growth, stabilizes size of fluctuations at different points in time

53 Differenced and logged weekly gas prices 0.17 Time Series Plot for adjusted GasPrice adjusted GasPrice First difference of logged prices (i.e. weekly % change) looks much more like noise Autocorrelations Estimated Autocorrelations for adjusted GasPrice lag However, there is a significant wave pattern of autocorrelation in the differences, so this series is not a random walk. (This pattern suggests an exponentially weighted moving average would be a better model, as we will see later.)

54 Forecasting from the GRW model First apply the standard RW model to the logged series to obtain forecasts and confidence intervals in logged units Then unlog unlog them (apply the EXP function) to obtain forecasts and CI s in original units. Statgraphics does all this automatically when you specify RW with a log transformation.

55 Direct calculation of GRW forecast One period ahead: y ˆ 1 = (1 + r) n+ y n k periods ahead: y ˆ = (1 + r) n+ k k y n Exponential growth curve where r is the average percentage growth per period Example: forecast for next month s S&P500 closing value x this month s closing value, based on estimated drift of * in logged units Forecast standard error x this month s closing value, based on RMSE of 0.042* in logged units *See output on later slide for RW model fitted to log(sp500)

56 Linear trend model The linear trend model is a straight line fitted to a plot of data versus time: y t = a+bt. Mathematically, it is a simple regression of the data variable Y on the time index T. It may be visually helpful to slap a trend line on a time series plot (Excel can do this)... but this is often a poor way to extrapolate a trend for purposes of forecasting the future!

57 Estimated trend = 2.68, compared with 2.45 in RWD model Here s the linear trend model for the retail data. Notice that its fit to the historical data is poor (especially during the bubble period of the late 90 s). Coincidentally, the trend line happens to pass almost through the last observed data point, although this need not be true in general. The confidence intervals for more distant horizons are unrealistically constant in width.

58 Linear trend vs. RW with drift A linear trend model assumes that there is a trend line fixed somewhere in space around which the data varies in an i.i.d.. manner. The fitted trend line always passes through the center of the data, i.e., through the point ( TY, ) A RW-with with-drift model also assumes that there is a constant trend, but it continually re-anchors the trend line on the last observed data point. CI s for the RW model widen parabolically parabolically as the forecast horizon increases, CI s for the linear trend model hardly widen at all. Which is more realistic for your data set?

59 Linear trend vs. RWD, continued In practice a RW-with with-drift model works better for smooth data; the linear trend may be appropriate for very noisy data. If the growth pattern in the data is irregular or not perfectly linear, the linear trend model may fit badly near the end of the series which is where the forecasting action occurs! Because the linear trend model anchors the trend line in the exact center of the data, its goodness of fit near the end of the series is very sensitive to the amount of past history used.

60 Importance of model assumptions Every forecasting model is based on particular assumptions about the nature of the true pattern in the data Example: RW model assumes that period-to to- period changes are i.i.d.,., while LT model assumes that deviations from a fixed trend line are i.i.d. The validity of the forecasts and confidence intervals depends on whether the assumptions are correct Assumptions should be tested statistically as well as by appealing to intuition or relevant theories.

61 How to test assumptions Statistical output not only shows goodness of fit but also provides diagnostic tests of assumptions Violations of model assumptions are indicated by non-random patterns in the errors (residuals): Autocorrelation Heteroscedasticity (non-constant variance) Non-normality normality and/or by poor out-of of-sample performance

62 Out-of of-sample validation It is always good practice to hold out some data during model-fitting and then validate the model by testing on the hold-out out data. This is also called out-of of-sample testing or (in the investment business) backtesting backtesting To be completely honest, you should hold out data while selecting the model,, not just during the final parameter estimation.

63 Estimation vs. validation vs. forecasting

64 Model comparisons: which model is better, and why? Models should be compared on the basis of the size of the errors they are expected to make in the future,, as well as on simplicity & intuitive reasonableness. The key error measures in the estimation & validation period (RMSE, MAE, MAPE) are indicators of the size of the errors the model is likely to make if they can be trusted!* *Depends on whether the model passes tests of its assumptions!

65 Keeping score: error measures root-mean-squared error, i.e., the square root of the average of the squared errors* RMSE: root MAE: mean absolute error, i.e., the average of the absolute values of the errors** MAE: MAPE: mean absolute percentage error** *Usually adjusted for # coefficients estimated, and it equals the sample standard deviation of the errors if if the mean error is zero (i.e., if forecasts are unbiased ) **NOT adjusted for # coefficients estimated

66 Which measure is best to focus on? RMSE is what your software is trying to optimize, because it always estimates the coefficients of the model by least squares, and it heavily penalizes very large errors MAE is a bit easier to understand and does not give as much relative weight to large errors MAPE is also easy to understand and is unit-free (robust against effects of inflation, compound growth, and multiplicative seasonality)

67 Here the geometric RW model for SP500 has been implemented as a RW model fitted to log(sp500) so that output statistics and plots are in log units. The last 50 points were held out for validation. (Note: log is the natural log function in Statgraphics.) Estimated drift is = 0.91% per month = 11.5% per year All three error stats are a bit larger in the validation period than in the estimation period, but in the same ballpark. This is probably because the volatility is a bit higher at the end of the series. RMSE of in estimation period means that the standard deviation of monthly changes is roughly 4.2%

68 Here is the same geometric RW model, constructed by using SP500 as the input variable and with the log transformation performed inside the procedure as a model option. The plots and statistics are now in unlogged terms, so we can t compare RMSE and MAE between estimation and validation periods, although we can still compare MAPE. Forecast plot is now unlogged Same estimated drift Error stats are now in unlogged terms, so RMSE and MAE are much smaller in estimation period than in validation period Residual plot is still logged but MAPE is still relatively similar in both periods (essentially the same as the MAE in log units on the previous slide)

69 Comparisons between models Ideally, error measures should be compared between models in the same (e.g., original) units and fitted to the same sample of data. The User-Specified Forecasting procedure in Statgraphics makes this easy: it produces a Model Comparison report that gives side-by by- side comparisons in original units.

70 What to look for Compare key error measures between models in the estimation period. Also check to see that validation period results are roughly consistent with estimation period results, particularly on the MAPE measure. RMSE and MAE are not comparable between estimation & validation periods if a log transform has been used as a model option inside SG s forecasting procedure or if inflation or multiplicative patterns are present stick stick with MAPE in those cases.

71 If one model Which model to choose? has clearly smaller errors than its rivals, passes the residual diagnostic and validation tests so that its assumptions are credible, yields sensible-looking looking plots of forecasts & CI s, and is supported by theory and intuition, then you have good reasons for choosing it. But don t pick models based on hair-splitting differences in error stats. When in doubt, choose the model that is simpler and more intuitive.

72 Here, model A (random walk with drift) is best on all measures in the estimation period Residual diagnostics OK MAPE in the validation period is consistent with estimation period results

73 Class 2 Recap Explanation of lags & differences Random walk model How to identify a random walk Examples of random walks Forecasting from the random walk model Log transformation & geometric random walk Linear trend model Model comparison & validation