General-to-Specific (GETS) Modelling and Indicator Saturation with the R Package gets (version 0.4)

Transcription

1 General-to-Specific (GETS) Modelling and Indicator Saturation with the R Package gets (version 0.4) Genaro Sucarrat Department of Economics BI Norwegian Business School Forskermøtet 2016 (NTNU, Trondheim) 4 January 2016 Joint with: Felix Pretis (Oxford) and James Reade (Univ. of Reading)

2 Prelude: Homepage: The paper is available there ++ (slides, forum and more) How to use R? Default IDE: Comes with R RStudio IDE (very popular): R Commander (a GUI): In this presentation I will use RStudio Why R? Tremendously powerful, versatile and democratic Enormous user-community (7700+ CRAN packages, 1100 on Bioconductor, on Github, ++) All CRAN and Bioconductor packages are checked and have user-manuals Open source and free Available on all operating systems, and via the cloud

3 The R package gets provides GETS modelling of the mean of a regression: y t = β 1x 1t + + β k x kt + ɛ t, ɛ t = σ tz t, z t iid(0, 1) 2...GETS modelling of the (log-)variance of a regression: ln σ 2 t = α 1w 1t + + α l w lt 3...detection and tests for breaks w/indicator Saturation (IS) methods The main functions of the gets package: 1 arx: Estimation of AR-X model with (optionally) log-arch-x errors 2 getsm: Automated GETS modelling of mean specification 3 getsv: Automated GETS modelling of variance specification 4 isat: Automated GETS modelling of an indicator saturated mean specification

4 Automated multi-path GETS modelling software: Sucarrat (November 2011): AutoSEARCH. An R package available from the CRAN. Multi-path, single-round, GETS modelling of both the mean and variance specifications. Based on code developed for Sucarrat and Escribano (2012): Automated Model Selection in Finance..., Ox.Bull.Econ.Stat. 74, pp Sucarrat (October 2014): gets. An R package available on the CRAN. More user-friendly and faster than AutoSEARCH, and contains more features (e.g. indicator saturation methods) Hoover and Perez (1999): MATLAB code. Only 10 paths, not user-friendly and no help-system OxMetrics (commercial): Hendry and Krolzig (2001): PcGets. Multi-path, multi-round, additional features Doornik (2009): Autometrics. Multi-branch, multi-round, additional features

5 Why AutoSEARCH/gets?: PcGets/Autometrics models the mean: y t = φ 0 + r φr yt r + s ηsx m s,t + ɛ t, ɛ t = σ tz t, z t iid(0, 1) In my research, I was interested in GETS modelling of the log-variance: ln σ 2 t = α 0 + p αp ln ɛ2 t p + d δ dx v d,t PcGets/Autometrics achieves this by modelling the AR-X representation: ln ɛ 2 t = φ 0 + P p=1 αp ln ɛ2 t p + D d=1 δ dx v d,t + u t, u t iid(0, σ 2 u), where φ 0 = α 0 + E(ln z 2 t ) and u t = ln z 2 t E(ln z 2 t ), see e.g. Bauwens and Sucarrat (2010): General to Specific Modelling of Exchange Rate Volatility: A Forecast Evaluation, Int.J.Forecasting 26, pp Problems: Uncorrelated and homoscedastic residuals û t does not imply uncorrelated and homoscedastic standardised residuals ẑ t, and likelihood-based comparisons with other models should preferrably be undertaken in terms of the likelihood of ɛ t rather than of û t This led to Sucarrat and Escribano (2012): Automated Model Selection in Finance..., Ox.Bull.Econ.Stat. 74, pp

6 Does anyone use gets/autosearch? AutoSEARCH: Downloads per week from 0 cloud (RStudio)

7 Four main ingredients of GETS modelling: Backwards elimination (along multiple paths) Regressor significance testing (individual and joint) Diagnostics testing Information criteria GETS modelling in 3 steps: 1: Formulate a General Unrestricted Model (GUM) that passes the chosen diagnostics tests 2: Backwards eliminiation of insignificant regressors along multiple paths while at each regressor removal: a) Test for joint insignificance and b) Check the diagnostics 3: Choose the best terminal model according to an information criterion

8 Example: The starting model (i.e. the GUM): y t = β 1 x 1t + β 2 x 2t + β 3 x 3t + ɛ t [p val] [0.07] [0.02] [0.26] P-values of two-sided t-tests in square brackets If we choose a 5% significance level, then deletion along two paths (i.e. a maximum of two distinct terminal models) If we start by deleting x 1t, then the second model in the first path becomes y t = β 2 x 2t + β 3 x 3t + ɛ t [p val] [0.00] [0.22] Next, deleting x 3t gives y t = β 2 x 2t + ɛ t, [p val] [0.00] i.e. the terminal model of path 1 (the deletion path is given by x 1t, x 3t)

9 Example (cont.): Recall the starting model (i.e. the GUM): y t = β 1 x 1t + β 2 x 2t + β 3 x 3t + ɛ t [p val] [0.07] [0.02] [0.26] If we now start by deleting x 3t, then we obtain i.e. the terminal model of path 2 Summarised: y t = β 1 x 1t + β 2 x 2t + ɛ t [0.03] [0.00] Path 1 = {x 1t, x 3t} with Terminal model = {x 2t} Path 2 = {x 3t} with Terminal model = {x 1t, x 2t} The final model: The one with smallest value on the chosen information criterion, e.g. Schwarz (1978)

10 Model selection properties of GETS modelling: k rel : Number of relevant variables k irr : Number of irrelevant variables E( k irr /k irr ) α: The irrelevance proportion or gauge equals the significance level α E( k rel /k rel ) 1: The relevance proportion or potency The (L)DGP is contained in the final model with probability 1 The irrelevance proportion (i.e. gauge) is closely related to the Per Comparison Error Rate (PCER). Recall: PCER = E [ kirr /(k rel + k irr )] The gauge and potency can be viewed as a more detailed (and more intuitive) characterisation of the False Discovery Rate (FDR). Recall: FDR = E [ kirr /( k irr + k rel )] How well does GETS modelling do compared with other model selection algorithms in terms of the relevance and irrelevance proportions? Very well! Studies show that it generally does better than step-wise methods Studies show that it generally beats the LASSO/shrinkage methods

11 Selected reading: Hendry and Richard (1982): On the Formulation of Empirical Models in Dynamic Econometrics, Journal of Econometrics Mizon (1995): Progressive Modeling of Macroeconomic Time Series: The LSE Methodology, in Hoover (ed.) Macroeconometrics. Developments, Tensions and Prospects, Kluwer Academic Publishers Hoover and Perez (1999): Data Mining Reconsidered: Encompassing and the General-to-Specific Approach to Specification Search, Econometrics Journal Hendry and Krolzig (1999): Improving on Data Mining Reconsidered by K.D. Hoover and S.J. Perez, Econometrics Journal Campos, Ericsson and Hendry (eds.) (2005): General-to-Specific Modeling. Volumes 1 and 2. Edward Elgar Publishing Hendry, Johansen and Santos Hendry et al. (2007): Automatic selection of indicators in a fully saturated regression, Computational Statistics Sucarrat and Escribano (2012): Automated Model Selection in Finance: General-to-Specific Modelling of the Mean and Volatility Specifications, Oxford Bulletin of Economics and Statistics Hendry and Doornik (2014): Empirical Model Discovery and Theory Evaluation. The MIT Press

12 Outline: arx: Estimation of AR-X model with (optionally) log-arch-x errors getsm: GETS modelling of mean specification getsv: GETS modelling of (log)variance specification isat: GETS modelling of an indicator saturated mean specification LaTeX: Generating LaTeX code with the xtable package Exporting results to EViews and Stata Future versions

13 arx: Estimation

14 The AR-X model with log-arch-x errors is given by y t = φ 0 + r φ r y t r + s η sx m s,t + ɛ t, ɛ t = σ tz t, z t iid(0, 1) ln σ 2 t = α 0 + p α p ln ɛ 2 t p + d δ d x v d,t Example of arx: set.seed(123) y <- arima.sim(list(ar=0.4), 100) mod01 <- arx(y, ar=1) Let us make things more interesting... mx <- matrix(rnorm(100*5), 100, 5) mod02 <- arx(y, mc=true, ar=1:2, mxreg=mx) mod03 <- arx(y, mc=true, ar=1:2, mxreg=mx, arch=1:3, asym=1, vxreg=log(mx ˆ2), vcov.type="white") Extraction functions: coef, fitted, loglik, plot, print, recursive, residuals, summary, vcov

15 getsm: Modelling the mean

16 Usage of getsm: Apply on arx object Examples: getsm02 <- getsm(mod02) getsm02b <- getsm(mod02, t.pval=0.01, wald.pval=0.01) getsm02c <- getsm(mod02, keep=1) All arguments of getsm function (w/defaults): t.pval = 0.05, wald.pval = 0.05, do.pet = TRUE, ar.ljungb = list(lag = NULL, pval = 0.025), arch.ljungb = list(lag = NULL, pval = 0.025), normality.jarqueb = NULL, info.method = c("sc", "aic", "hq"), keep = NULL, include.gum = FALSE, include.empty = FALSE, max.regs = NULL, zero.adj = NULL, vc.adj = NULL, verbose = TRUE, print.searchinfo = TRUE, estimate.specific = TRUE, plot = TRUE, alarm = FALSE Extraction functions: coef, fitted, loglik, paths, plot, print, recursive, residuals, summary, terminals, vcov

17 getsv: Modelling the variance

18 Usage of getsv: Apply on arx object Examples: getsv03 <- getsv(mod03) getsv03b <- getsv(mod03, t.pval=0.1, wald.pval=0.1) getsv03c <- getsv(mod03, keep=1:4) All arguments of getsv function (w/defaults): keep = c(1), t.pval = 0.05, do.pet = TRUE, wald.pval = 0.05, ar.ljungb = list(lag = NULL, pval = 0.025), arch.ljungb = list(lag = NULL, pval = 0.025), info.method = c("sc", "aic", "hq"), include.empty = FALSE, zero.adj = NULL, vc.adj = NULL, tol = NULL, LAPACK = NULL, max.regs = 1e+05, verbose = TRUE, alarm = FALSE Extraction functions (same as those of getsm): coef, fitted, loglik, paths, plot, print, recursive, residuals, summary, terminals, vcov

19 isat: Indicator Saturation

20 Prelude to Saturation: Recall: A step-indicator for observation t {1, 2,..., T } is given by 1 {t t } = 1 if t t and 0 if t < t, t = 1, 2,..., T Advantages with the step-indicator approach to location-shifts: Computationally simple Empirically, many shifts are of a step-nature/non-smooth Logistic Smooth Transition Term (LSTT) terms can be written as a sum of step-shifts: LSTT = = t exp [ b(t t )], b > 0, a t1 {t t }, a t (0, 1) A series {y t} with a break in the deterministic trend can be represented by step-shifts in its difference, i.e. in y t Outliers or impulses can be represented by a pair of subsequent step-indicators. That is: 1 {t=t } = 1 {t t } 1 {t t +1}

21 Prelude to Saturation (cont.): Consider the following DGP: y t = ɛ t for t = 1,..., 30 and y t = 4 + ɛ t for t = 31,..., 100, ɛ t N(0, 1): y <- rnorm(100) y[1:30] <- y[1:30] + 4 Let us do GETS modelling with 20 step-indicators as regressors, covering the period t = 20,..., 40: mysteps <- sim(y) blockofsteps <- mysteps[,19:39] mod04 <- arx(y, mc=true, mxreg=blockofsteps) getsm(mod04) Saturation: The whole sample T is divided into blocks/subsets of indicators, and GETS modelling is undertaken on each block and/or block combination Divide the T indicators into blocks and do GETS on each block Take the union of the final models and do GETS anew

22 isat function: GETS modelling of an indicator saturated mean specification Indicators: Impulses, steps (default), trends Joint with Felix Pretis, Univ. of Oxford, and James Reade, Univ. of Reading Specification (an AR-X w/indicators): y t = φ 0 + r φ r y t r + s η sx m s,t + indicators + ɛ t, Example of isat w/step Indicator Saturation (SIS): The annual flow of the river Nile at Ashwan data(nile) plot(nile, xlab="year", ylab="annual flow") nilemod <- isat(nile, ar=1:2, sis=true, t.pval=0.01) Extraction functions (same as those of getsm): coef, fitted, loglik, paths, plot, print, recursive, residuals, summary, terminals, vcov

23 LaTeX

24 Generating LaTeX code: The objects returned by arx, getsm, getsv and isat are lists To obtain the names of the entries of these lists, use the summary command (or names). Example: summary(mod04) The estimation results are stored in data frames named, respectively: mean.results, variance.results and diagnostics These can readily be converted into LaTeX code with the package xtable. Example: library(xtable) print( xtable(mod04$mean.results) ) Similarly for the log-variance results and the diagnostics: print( xtable(mod04$variance.results) ) print( xtable(mod04$diagnostics.results) ) To control the number of printed digits, use the digits argument of the xtable function

25 EViews and Stata

26 Most popular commercial econometric softwares: EViews and Stata None of these provide GETS modelling capabilities To facilitate GETS modelling for EViews and Stata users, we have created two functions: eviews and stata These can be applied on arx, gets and isat objects. Example: eviews(nilemod) Similarly for Stata: stata(nilemod) The data-export feature is most likely to be useful in concjunction with isat

27 Future versions

28 Future versions: More options and features Further speed improvements

29 Thanks!

30 These slides are available at:

31 References: Bauwens, L. and G. Sucarrat (2010). General to Specific Modelling of Exchange Rate Volatility: A Forecast Evaluation. International Journal of Forecasting 26, Campos, J., D. F. Hendry, and N. R. Ericsson (Eds.) (2005). General-to-Specific Modeling. Volumes 1 and 2. Cheltenham: Edward Elgar Publishing. Doornik, J. (2009). Autometrics. In J. L. Castle and N. Shephard (Eds.), The Methodology and Practice of Econometrics: A Festschrift in Honour of David F. Hendry, pp Oxford: Oxford University Press. Hendry, D. F. and J. Doornik (2014). Empirical Model Discovery and Theory Evaluation. London: The MIT Press. Hendry, D. F., S. Johansen, and C. Santos (2007). Automatic selection of indicators in a fully saturated regression. Computational Statistics 20, DOI /s z. Hendry, D. F. and H.-M. Krolzig (1999). Improving on Data Mining Reconsidered by K.D. Hoover and S.J. Perez. Econometrics Journal 2, Hendry, D. F. and H.-M. Krolzig (2001). Automatic Econometric Model Selection using PcGets. London: Timberlake Consultants Press. Hendry, D. F. and J.-F. Richard (1982). On the Formulation of Empirical Models in Dynamic Econometrics. Journal of Econometrics 20, Hoover, K. D. and S. J. Perez (1999). Data Mining Reconsidered: Encompassing and the General-to-Specific Approach to Specification Search. Econometrics Journal 2, Dataset and code: Mizon, G. (1995). Progressive Modeling of Macroeconomic Time Series: The LSE Methodology. In K. D. Hoover (Ed.), Macroeconometrics. Developments, Tensions and Prospects, pp Kluwer Academic Publishers. Schwarz, G. (1978). Estimating the Dimension of a Model. The Annals of Statistics 6, Sucarrat, G. (2011). AutoSEARCH: An R Package for Automated Financial Modelling. Sucarrat, G. (2014). gets: General-to-Specific (GETS) Model Selection. R package version Sucarrat, G. and Á. Escribano (2012). Automated Model Selection in Finance: General-to-Specific Modelling of the Mean and Volatility Specifications. Oxford Bulletin of Economics and Statistics 74,