Introduction to Time Series Analysis. Madrid, Spain September Case study: exploring a time series and achieving stationarity

Transcription

1 Introduction to Series Analysis Madrid, Spain -4 September 27 Case study: exploring a time series and achieving stationarity Objectives: at the end of the case study, the participant should Understand principles and characteristics of time series Manipulate statistical tools to explore a time series Characterize a time series in term of trend, variance, seasonality Achieve stationarity in a time series Denis Coulombier, Fernando Simón, Philippe Quénel

2 Descriptive analysis of time series Page 2 Present at i on This case study includes 2 parts. The first part covers the descriptive analysis of a time series while the second part illustrates principles of converting a crude time series in a stationary time series. Part : descriptive analysis of time series Part 2: achieving stationarity in the series Programs needed on the computer: Microsoft Excel Example files used by the case study: TIMESERIES.XLS STATIOANRITY.XLS Text style used in the case-study Commands to type in the computer. The text between ' and ' is the text you actually need to type Additional information about the programs Reference to cells in Excel uses the following syntax: A refers to the first cell of the first row. When a formula is copied over a range, it is important to keep in mind that Excel considers cells entered in this format as relative references. If the formula: =A + B is copied over a range, the output cells will have their references incremented: =A2 + B2 =A3 + B3 and so on. Using $ sign in front of one of the coordinates makes it an absolute reference. If the formula: =$A$ + $B is copied over, it gives: =$A$ + $B2 =$A$ + $B3 A cell can be named rather than expressed as coordinates. To name a cell, place the cursor in the cell, click on the left cell of the line immediately above the header line and enter the desired name. Subsequently, you can use this name to refer to this cell. This allows for more meaningful names of cells in formulas. In the same way, a range can be named. Select the range to name (it appears against a black background) and indicate the appropriate name in the left cell above the top row. In this case study, most cells have been renamed.

3 Descriptive analysis of time series Page 3 Most formulas and labels have been already entered in the spreadsheet. Macros are used to carry out the various tasks required for the exploration of the series. However, all tasks can be carried out manually by entering the relevant formulas in the corresponding cells. The protection of the spreadsheet is activated in order to avoid erasing a cell. However the protection of cells in which you need to enter, change the value, or copy cells has been disabled. This case study uses Excel version 5 or later. Excel uses different names for formulas in the various European languages. This case study uses English denomination. If you are using non-english version of Excel, you should use the following table to adapt formulas. Lexicon of formulas in European languages Formula French English German Italian p PI() PI() PI() PI.GRECO() sum of a range SOMME() SUM() SUMME() SOMMA() Variance VAR() VAR() VARIANZ() VAR() Covariance COVARIANCE() COVAR() KOVAR() COVARIANZA() Standard deviation STD() STDEV() MITELABW() DEV.ST() Mean MOYENNE() AVERAGE() MITTELWERT() MEDIA() This case study uses some Excel add-ins. You need to check that these add-ins have been activated before proceeding. In the Tools menu of the main menu bar, you should see options for Solver and Analysis tool pack. If not, Click on add-ins, and activate the Solver and the Analysis tool pack If you do not see the analysis tool pack option, it means that you did not carry out a complete installation of Excel. When a complete installation is carried out, you should see an Analysis directory under EXCEL\LIBRARY on your hard disk. If not, reinstall Excel.

4 Descriptive analysis of time series Page 4 Part : descri pt i ve anal ysi s of t i me seri es Load the file TIMESERIES.XLS in Excel. Introduction This Microsoft Excel folder includes 8 spreadsheets, plus an introduction page, each of them covering one aspect of descriptive analysis of time series. Spreadsheet Introduction The button Adjust screen display will optimise the layout of the various spreadsheets for the resolution of your screen. Click on the Adjust screen display button Click on the Parameters spreadsheet Spreadsheet : Parameters This spreadsheet allows you to set various parameters contributing to generate a time series. These parameters determine a linear slope (slope2), an exponential slope (Slope), a constant term (Constant), amplitude of seasonal variations (Amplitude), starting point for seasons (Phase), periodicity of seasons (Period), a random factor (Alea), and an exponential factor (Exponential). Setting these parameters will create a different time series each time. The random factor (ALEA in the formula below) is a Gaussian white noise defined in Excel by the Muller formula: (-2*LN(RAND()))^.5*COS(2*PI()*RAND()) A B C D E F G 2 Parameter Slope 4 Slope2,, 5 Constant 6 Amplitude 7 Phase Period Alea Exponential, 2 3 The objective of the case study is to explore each of the eight predefined time series. Each series does not correspond to actual data extracted from a surveillance system, but results from an equation defined as follows: Value=Slope^2*+Slope2*+Constant+Amplitude*(Cos(2pi*(-Phase)/period)+Alea The current time series is the one defined by the parameters appearing in cells C3 to C. To view all the series available in the case study, click on one of the view series option button. The corresponding plot appears on the spreadsheet. Activate the first spreadsheet called 'parameter'. Review all the series available by clicking on the corresponding option buttons labelled Series to Custom series View series Series Series 5 Series 2 Series 6 Series 3 Series 7 Series 4 Series 8 Custom series To load a new time series in the spreadsheet, for example series, click on the button Series in the above option button list, and then on Load series. The macro command associated with this button copies the cells of the range E2 to E to the range C2 to C. We start with series. Click on the option button Series Click on the button Load series

5 Descriptive analysis of time series Page 5 The plot of the series appears. Crude data Series Tim e Comment the plot in term of trend, cycles, and variance over time It shows: A clear positive trend over time 52 weeks seasonal variations of increasing amplitude A variance that seems to increase over time and with values Spreadsheet 3: Data This sheet stores the data corresponding to the series that was selected in the "parameter" sheet. Column C contains the formula for the equation generating the series. Column D contains a copy of column C, but the actual values and not the formula have been pasted in order to freeze the random factor. If data in column C were used, the random factor would be recalculated whenever an action is performed, and the series would not be stable. This explains why the 2 series are not equal. Columns E and F contain standard description of the values of the series. The data values on your screen may differ slightly from the one data value because of the random factor. A B C D E F 2 Data Series Indicator Value Observations Minimum Maximum Range Mean Median Variance Standard deviation Skewness Kurtosis Skewness characterizes the degree of asymmetry of a distribution around its mean. Positive skewness indicates a distribution with an asymmetric tail extending toward more positive values. Negative skewness indicates a distribution with an asymmetric tail extending toward more negative values.

6 Descriptive analysis of time series Page 6 Kurtosis characterizes the relative peakedness or flatness of a distribution compared with the normal distribution. Positive kurtosis indicates a relatively peaked distribution. Negative kurtosis indicates a relatively flat distribution. At this junction summarize the information you get from the indicator table. Summary findings, in short: Contains 256 data point ranging from 2.26 to The mean is about twice the median, suggesting a skewed distribution This is confirmed by a positive skewness of 2.79 and a kurtosis of 9.6 suggesting a distribution much broader than a normal distribution. The standard deviation is greater than the mean confirming wide amplitude of variations in the series. Activate spreadsheet 4 called 'Distribution' Spreadsheet 4: Distribution The interval for representing the distribution of values is set by the spreadsheet by the formula =(MAX(DATA)- MIN(DATA))/2 entered in cell C2. This creates 2 classes of equal size and covering the entire range of the data set. A B C D E F 2 Interval Class Frequency Distribution of values 4 2 Number Class Describe the shape of the distribution curve in term of skewness.

7 Descriptive analysis of time series Page 7 It confirms visually the impression gotten by looking at the descriptive statistics: a very POSITIVELY skewed, non normal distribution of values. Positively skewed distribution usually indicates an unstable variance, corresponding to an exponential factor in the series. This is an indication to perform a log transform to stabilise the variance. Infectious disease surveillance time series distribution of values is usually positively skewed. Activate spreadsheet 6 called 'Trend' This sheet lets you explore the trend in the working series. A moving average window of 52 weeks is used to remove (hide) the seasonal component of the series. In order to do so, the size of the moving average window should include a whole cycle, 52 weeks in our example. As a consequence, the first and the last 26 data points are missing. Excel comes with a built-in function for calculating moving averages that is not relevant in epidemiology. In order to smooth a series by moving average, the average should be plotted against the middle point of the window. In business, moving averages are plotted against le last point of the window, allowing brokers to study stock values at a given time, taking into account average past values over windows of increasing sizes. But such a display translates the series of a factor (window size/2), which is not appropriate for describing the series in epidemiology. Look at the formulas in cells D3 to D258 for centring the moving average window over the series. 5. Moving average y =.467x -.78 R 2 = Clicking on the Show trend line button will display the linear trend line and its equation. Parameter Value Value P value Status m se(m) R²/standard error F/DDL/p E-2 Significant LINEEST(DATA) Excel formula has been used to describe the regression line. This is an array formula which requires to select the range for output (5 rows by 2 columns) before typing the formula, and use Ctrl-Shift-Enter to enter the formula. Use Excel help on "Array formulas" to learn more about this tricky kind of formulas. This function returns values (see Excel help for details). As indicated on he graphic, is the slope and -.78 the intercept of the regression line. A F test can be carried out to test the significance of the slope of the regression line. A F statistics of , with 254 degrees of freedom returns a p value of 9.9E -2, which is extremely significant.

8 Descriptive analysis of time series Page 8 A linear trend will tend to be parallel to the moving average series. It will cross it several times, without displaying any particular pattern. On the other hand, if a quadratic trend is present, the 52-week moving average series will tend to be above the regression line on both extremities, and below the regression line in the middle of the series. Indicators describing the regression line are shown in cells G28 to J 33. Click on the various cells to see the meaning of each parameter. Significant in cell J32 means that the slope is "significant", not the result of random variations. However, there will be time when it will be significant, even though no trend parameter exists in the data, but just as a result of randomness. You can click on the "Show trend" button to display the linear regression line of the series. This will help you to describe the 52 week moving average. Comment further the trend The trend seems a bit exponential, with the extremities of the moving average line above the linear trend, and the centre part under. Using moving averages is a very good way to describe the shape of the regression line. An exponential trend leads to a parabolic moving average line. Activate spreadsheet 7 called 'Seasonality' This sheet lets you explore the seasonal component in the time series, using a moving average window in order to remove (hide) the "accidental/random" component of the series. Unlike for removing the trend, the optimal size of the moving average window cannot be preset. It will depend on the amount of "randomness" in the series. The more randomness, the larger needs to be the moving average window to smooth the series. Seasonality can be assessed by moving average. For assessing cycles the size of the moving average window should be adjusted visually, in order to remove random variations, without averaging too broadly the cycles. In our example, a window size of 2 reaches a good compromise. The up and down arrows of the Spin button located under the Window label can be used to enlarge and reduce the size of the moving average window. The Guess button will screen the series to identify the best window size to emphasise seasons. Optimisation is based on the function curve of the correlation coefficient between the original series and the moving average series. The optimal window size, when seasons are present is the one for which the decrease of the correlation coefficient is minimal when the size of the window is enlarged by one week. Look at the graphic located in cell F49. It shows the distribution of the correlation A B C D E F 2 Data MAVG Window #N/A #N/A #N/A #N/A #N/A #N/A coefficient as a function of the moving average window size. For a value of the MAVG window of, the 2 series are identical, and thus, the correlation coefficient is equal to one. As the window increases, the correlation coefficient drops until it becomes almost horizontal. This area corresponds to the optimal size for the MAVG window, in order to emphasize seasonal variations. As the MAVG window enlarges beyond this point, the correlation coefficient drops again.

9 Descriptive analysis of time series Page 9 R² as a function ofmoving average window width R² Optimal size of the MAVG window Window width Change the value in cell E3 to 6 and then to 36 and see how it affects the display. Comment. 5 Moving average The presence of cycles is quite obvious here. However, to further characterise the cycles, we should look whether the cycles are similar each year. To do so, we split the data set in 52-week periods that we plot in different series to compare their shapes. Seasonality 25 Series Series2 Series3 Series4 Series5 Series Comment the yearly plot

10 Descriptive analysis of time series Page On the first display, the sense of the trend is very strong, yet cycles could have varied in length over time unnoticed. On the second display, we loose the sense of the trend over time, but we are convinced that the cyclical pattern is similar in shape over time. Thus these 2 representations are complementary and should be selected for a paper or a presentation according to the message being delivered. Activate spreadsheet 8 called 'Variance' Finally the stability of the variance is checked. A variance is not stable if it varies according to time or according to the mean value over periods. The descriptive tool to assess this is called the 'mean-variance plot'. Two series are generated: - A moving average mean - A moving average variance The graph plots the mean (X-axis) against the variance (Y-axis). A positive regression line indicates a positive relation ship between the mean and the variance. The graph uses a moving average mean and a moving average variance for showing the relationship. This is done in order to avoid splitting arbitrarily the series. However, more often you will find the mean and the variance plotted for predefined periods (such as years). This works well if there is a trend in the series, but may hide the relationship if there is seasonality without a trend, and the period used corresponds to the seasonality. The two series are plotted against each other. In order to optimise the display, click on the button Set the axis scale. This reset the 2 series to minimum and maximum values. y = 57.34x R 2 = Variance Mean Mention that this is used to pose the indication to a log transform. Should we assess the variance first since it is controlled in this order. Click on Set the axis scale Comment the mean variance plot There is an obvious positive slope in the plot. The higher the mean, the higher the variance. This indicates a nonstable variance. Other alternatives to check this consist of plotting the mean against the range, or slicing the series in equal size periods and plotting the mean against the variance. The moving average mean variance plot is a less used, easy to implement alternative. The F test in cell K 3 is significant, confirming the visual exploration of the plot.

11 Descriptive analysis of time series Page Explain the F test Activate spreadsheet 9 called 'ACF' Autocorrelations measure how strongly time-series values at a specified number of periods apart (lags) are correlated. Thus an autocorrelation at lag is a measure of how successive values (one period apart) are related to each other throughout the series, and correlations at lag 2 measures how series values two periods apart are correlated. Correlations are computed as values between - and +. A value close to + indicates strong positive correlation and a value close to - indicates strong negative correlation. Values close to zero indicate no correlation or independence. Highly correlated values at lag (adjacent series values) indicate a trend in the data (differencing) and highly correlated values at seasonal lags (lag s) indicate seasonality or periodicity in the data (seasonal differencing). The autocorrelation function has been calculated up to Explain LAGS lag 6, in order to explore the trend, and for lags 52, 4,56, 28 to explore seasonality. Click on the Show significant lags to highlight significant lags on the graph. Regular ACF. Regular ACF Lag The graph indicates that values until lag 3 are significantly correlated. This indicates a trend in the data. Seasonal ACF Seasonal ACF Lag Values at seasonal lags are significantly correlated. It means that values situated 52, 4, 56 and 28 weeks apart are correlated, indicating a significant seasonal component in the time series. Repeat part for all eight series in the parameter sections

12 Achieving stationarity in a time series Page 2 Part 2: achi evi ng st at i onari t y i n t he seri es Load the file STATIONARITY.XLS in Excel. Definition of stationarity A series is stationary if it has: - A constant mean - A constant variance - Cycles have been removed It is a prerequisite for modelling the series. This second part uses the tools described in part one for assessing whether the above conditions are met. Techniques for achieving stationarity Several techniques can be used to make a series stationary: Stabilizing the variance - By applying various transforms to the series: log, square root, reciprocal, Removing the trend: - Subtract a regression line, either linear or quadratic - Differencing or first derivative consisting in plotting difference between subsequent values rather than absolute values. Removing Cycles - Subtract a cosine regression line - Differencing of order one at lag=period The variance is always controlled by applying transforms to the series. In epidemiology, most, if not all the times, a log transformed is indicated. For tend and cycles, two approaches are possible: regression and differentiation. We will use both in this part, as an introduction to Spectral analysis which uses regression and Box & Jenkins SARIMA modelling which uses differentiation. These techniques should be applied in an ordered mode: variance, then trend, finally cycles. Refer to the lecture for further considerations on these techniques. Activate spreadsheet called Intro. As in part, we will reset various parameters before starting. Click on Adjust screen display Click on Reset spreadsheet to default Activate spreadsheet 2 called parameter. This part functions in a similar way as for part spreadsheet parameters. Review all the series available by clicking on the corresponding option buttons labelled Series to Custom series

13 Achieving stationarity in a time series Page 3 We start with series. Click on the option button Series Click on the button Load series Describe the series, by looking at the plot. Working series Activate Spreadsheet 3, called 'data' A B C D E F G H 2 Data Values Variance Trend Seasonality Working , , , , , , , , , , ,85 The series Value, Variance, Trend, Seasonality, and Working hold the same values on each line so far. The Working series is the one that is used by all the descriptive tools in the subsequent worksheets. It is our working series. First, explore the crude series to describe it. Click on distribution and comment Click on plot and comment Click on trend and comment Click on seasonality and comment Click on variance and comment

14 Achieving stationarity in a time series Page 4 This series has a non-stable variance, a linear trend, and seasonal variations. Stabilizing the variance Our descriptive analysis showed a variance increasing with the mean. This is an indication to apply a log transform. Click on Data spreadsheet Click on the button Cells E3 to E258 contains the formula =LN(D3) to =LN(D258) which means Neper (Natural?) logarithm of the cell immediately on the left of the current cell. Go through all exploratory tools to check the effect of the transform The distribution of values has become a normal distribution, indicating the pertinence of the transform The plot shows how the variance has been stabilized: the amplitude of variations look constant over time and according to the mean. Working series The effect of the transformation was to make the trend clearly linear and no longer parabolic.

15 Achieving stationarity in a time series Page 5 7. Moving average 6. y =.7x R 2 = At the same time the amplitude of seasonal variations are more constant. The mean variance plot shows an inversion of the regression line, indicating a slight 'over transformation'. However, the visual inspection of the plot clearly shows the necessity to apply this transform y = -.3x R 2 = Variance Mean Removing the trend by regression Cells J3 and K3 in the data sheet use the regression line Excel function to get respectively the slope and the intercept of the regression line. Slope Intercept The regression line is removed by subtraction: Copy cell F265 of the data spreadsheet over the range F3 to F258 or click on the Remove linear trend by regression button. Explore other pages to see how this affected the checks.

16 Achieving stationarity in a time series Page 6 Moving average y = -3E-7x + 5E-5 R 2 = 6E The distribution was not much affected The plot is centred around The trend has effectively been removed Seasonality are quite identical The variance was further stabilized Removing the cycles by regression In this part, we will attempt to fit a cosine curve to the working series. A cosine curve is defined by its amplitude of oscillations, its period (or wavelength) and its phase (its starting point). A cosine curve can be written as follows: Y t = R cos(ωt + θ) Where: ω is the frequency of the periodic variation = 2π/period R is the amplitude of the variation θ is the phase Typically, the translation of such an equation in Excel uses a formula like this: =$K$5*COS(2*PI()*(TIME-$K$6)/PERIOD) Where $K$5 is the value of the amplitude TIME is the range B3 to B258, representing the weeks $K$6 is the phase And PERIOD the value of the period, appearing in cell C8 of the parameter spreadsheet. In order to get the best fit, we need to use the solver to find the most suited values for the amplitude and the phase. These values will the one minimizing the sum of the square of the difference between the cosine curve and the data. Click on the button Confirm the optimisation of parameters returned by the solver Clicking on the button performs the following actions: It copies the formula =$K$4*COS(2*PI()*(TIME-$K$5)/PERIOD) in the range M3 to M258

17 Achieving stationarity in a time series Page 7 It calls the solver (Tools menu) with the following parameters o $K$6 for the target cell o Min in the options Equal to o $K$4:$K$5 for the changing cells The following values are returned by the function: Amplitude Phase S² These values may be slightly different on your computer because of the random factor Check all diagnostic tools to comment the effect The distribution of values is close to a random gaussian distribution The plot looks more random as well Working series There still seems to be cycles on the plot using moving averages

18 Achieving stationarity in a time series Page 8 2 Moving average But 'seasons' are not correlated Seasonality Series Series2 Series3 Series4 Series5 Series The variance remains stable The ACF functions shows only 7 significant peaks, no trend and no seasons.. Regular ACF Lag Thus, we have achieved stationarity. Summary of regression approach Regression woks by subtracting from the crude data various regression curves for removing pattern in the signal. A finer approach will be developed in tomorrow case study as we explore the Fourier transform which permits to refine the characteristics of the cyclical regression curve by adding curves of various periods.

19 Achieving stationarity in a time series Page 9 Removing the trend by differentiation This is the second approach to remove trend and seasons in time series. Activate the spreadsheet Data Click on the button Reset removing trend and seasons This will reset the trend and seasonality series to their default values. Differentiation works by plotting the difference between 2 subsequent data point rather than their absolute values. Click on the button Note that the first data point, cell F3 is reset to since we use the difference between 2 subsequent data points. Explore the pages and comment the achievements. The plot shows that the trend has been completely removed Working series Yet, we still have clear seasonality, looking at the moving average 2 Moving average

20 Achieving stationarity in a time series Page 2 Removing the cycles by differentiation To remove the seasonality, we do a second differentiation of order at 52 weeks. Click on the button We loose the first 52 additional data point in this process. Comment the various pages after this second differentiation 3 Moving average The trend and cycles have been removed during the process. The series looks stationary. Summary of differentiation approach Differentiation works by subtracting subsequent values at certain lags. A finer approach will be developed in a coming case study as we further explore the autocorrelation function that permits to refine the evaluation of stationarity, as a first step of the Box & Jenkins SARIMA modelling approach.