Financial Time Series Volatility Analysis Using Gaussian Process State-Space Models

15 IEEE Global Conference on Signal and Information Processing (GlobalSIP) Financial Time Series Volatility Analysis Using Gaussian Process State-Space Models Jianan Han, Xiao-Ping Zhang Department of Electrical and Computer Engineering, Ryerson University 35 Victoria Street, Toronto, ON, Canada M5B K3 jhan, xzhang@ee.ryerson.ca Abstract In this paper, we propose a novel nonparametric modeling framework for financial time series data analysis, and apply it to the problem of time varying volatility modeling. Existing parametric models have a rigid-form transition function and they often have over-fitting problems when model parameters are estimated using maximum likelihood methods. These drawbacks effect the models prediction performance. To solve this problem, we take Bayesian nonparametric modeling approach. By adding Gaussian process prior to the hidden state transition process, we extend the standard state-space model to a Gaussian process state-space model. We introduce the Gaussian process regression stochastic volatility (GPRSV) model and instead of using maximum likelihood methods, we use Monte Carlo inference algorithms. We demonstrate performance of our model and inference methods with both simulated and empirical financial data. I. INTRODUCTION The problem of analysing financial time series data is an important task for both research and investment. In the last decades, many researchers took the modeling approach to describe financial data. Modeling give us a way of discovering knowledge from data, and making prediction. From this point, modeling financial time series data is very similar to modeling signals in engineering application. For example, in presence of noise, filtering methods like Kalman filter and particle filter can be applied to financial data too. With the recent development of Bayesian nonparametric modeling in signal processing community, we can model financial data with more flexible tools, like Gaussian Process and Copula Process. In this work, we apply Gaussian process regression to the transition function of volatility modeling. For inference methods, we apply Bayesian inference framework to compute the jointly distribution. Compared with the traditional parametric models, our new method is more flexible and free of over fitting problem. Volatility can be expressed as the standard deviation of an asset return and it is widely used to describe the variability of financial time series data. In some degree, volatility is a measurement of the financial time series magnitude and fluctuation s speed. Volatility modeling has been one of the most active financial time series research areas in the last decade, and it is of great importance for both finance market practitioners and academic researchers. Volatility is a forwardlooking concept, we often model the financial time series return variance conditioned on all the relevant information I t 1 which is defined as follow: σ t = var(r t I t 1 )=E((r t μ t ) I t 1 ), (1) where r t is the asset return and μ t is expected value of r t. II. REVIEW OF EXISTING VOLATILITY MODELS There are two main classes of modeling time changing variance: the generalized autoregressive conditional heteroskedasticity (GARCH) and the stochastic volatility (SV) model. For the GARCH typed models, ARCH [1] was first introduced in 198 by Engle. GARCH [] was generalized form of ARCH. The other type is Stochastic Volatility models, see [3]. There are many variants for both these two types of models. Numerous researchers developed extensions of these models applying for empirical analyses in financial economics and macroeconomics. A lot of variants and extensions have been studied. A. GARCH models Autoregressive Conditional Heteroscedastictiy (ARCH) model was first introduced by Engle(198). Bollerslev extended the model, and give a form of the Generalised Autoregressive Conditional Heteroscedastictiy models. Standard GARCH(1,1) model assumes the return of the asset follows a Gaussian distribution. In this work, we simplify its mean μ to be zero and the variance of the distribution is time-varying: x t N(,σt ) σt = α + α 1 x t 1 + βσt 1 (a) (b) Some GARCH extensions are reviewed here too. The Exponential GARCH (EGARCH) [4] can be summarized as: log(σt )=α + α 1 g(x t 1)+βlog(αt 1) g(x t )=θx t + λ x t (3a) (3b) Another popular GARCH extension is the GJR-GARCH model [5]: σ t = α + βα t 1 + γx t 1I t 1 (4a) 978-1-4799-7591-4/15/$31. 15 IEEE 358

15 IEEE Global Conference on Signal and Information Processing (GlobalSIP) {, if x t 1 I t 1 = (4b) 1, if x t 1 < Parameters of GARCH class models can be learned using maximum likelihood methods. There are a lot of papers on the topic and many software environment provide the program implementing the algorithms. Among these, see Kevin Shepperd s Matlab code of Oxford MFE Toolbox. B. Stochastic Volatility models The first discrete time-varying stochastic volatility model was introduced by Taylor [3], [6]. The logarithm of variance was modeled by a latent AR(1) process. Taylor s stochastic model can be presented as: r t = μ t + a t = μ t + σ t ɛ t (5a) log(σt )=α + α 1 log(σt 1)+σ n η t (5b) where α 1 is a parameter which controls the persistence of logarithm variance and the value of α 1 is between (-1,1). There are two independent and identically distributed random variables ɛ t and η t. The original idea of SV model assume these two noise parts to be i.i.d. normal distributed. The inference of a SV model parameters is not as straightforward as the corresponding simple GARCH typed model. In [7], Shephard reviewed SV models and inference methods like methods of moments (MM) and quasi-maximum likelihood (QML). Simulation-based methods to learn SV models become more and more popular because of their accuracy and flexibility of handling complicated models. III. GAUSSIAN PROCESS STATE-SPACE MODELS A. Gaussian Process A Gaussian Process can be viewed as an extension of multivariate Gaussian distribution to infinite dimensions. Any finite subset of sampled from the process follows a multiple Gaussian distribution. Also, Gaussian Process can be considered as a normal distribution over function, and it is determined by mean function and covariance function. f(x) =GP(m(x),k(x, x )) (6) All values of f(x) at any location x are jointly Gaussian distributed. B. State-Space Models State-space model (SSMs) are one of the most widely used models for effectively modeling time series and describing dynamical system.in Finance area, State-space models can generalize other popular time series models such as ARMA, (G) ARCH and SV models as we mentioned before. A State- Space Model can be a general framework for descripting dynamical phenomena. The model consists of two parts: an hidden state x t and observation variable y t. The general form of State-Space model can be summarized as: x t = f(x t 1 )+ɛ, x t R M (7a) y t = g(x t )+ν, y t R D (7b) where, ɛ and are normal distributed noise. The unknown function f describe the system dynamics and function g links the observation and the system hidden states. Both the f and g functions can be linear or non-lienar. We can combine Gaussian process and state-space model together. The way of combining the two is to use the statespace model s structure and apply Gaussian process to describe the hidden state transition function. The essence of the GP- SSMs is to change the rigid form of states transition function of traditional state-space models with a Gaussian process prior. Financial data exhibits many dynamics because the market is changing all the time and a lot of small change of the involved factors can result in significant fluctuation. As more and more data is available to process, the rigid form of state transition function in traditional state-space models becomes the bottleneck of improving the models forecast performance. We assume the hidden state transition function f is sampled from a Gaussian process, we extend the standard state-space model to a Gaussian process state-space model (GP-SSM). Compared with standard SSM, GP-SSM is a more flexible and powerful tool to model time series data. The linear SSMs with Gaussian noise can be inferenced using Kalman Filter [8]. But linear Gaussian SSMs can only model a limited set of phenomena. To learn a GP-SSM is more difficult than a standard SSM. In our work, we take the Bayesian approach to solve the GP-SSMs inference problem. Bayesian filtering is a type of technique used to estimate the hidden states in dynamic systems. The goal of Bayesian filtering is to recursively compute the posterior distribution of the current hidden state given the whole history of observations. IV. GAUSSIAN PROCESS REGRESSION STOCHASTIC VOLATILITY MODEL Our Gaussian process regression stochastic volatility (GPRSV) model can be viewed as an instant of GP-SSM. We treat volatility as unobserved latent variable, like the standard stochastic volatility models. Instead of assuming that the latent volatility follows a p-step autoregressive process. We place a Gaussian process prior to the unknown transition function f on the volatility process. A GPRSV model can be presented with the following equations: a t = r t μ = σ t ɛ t v t = log(σt )=f(v t 1 )+τη t f GP(m(x),k(x, x )) (8a) (8b) (8c) where r t is the asset return at time t and μ is the mean of asset return series, and a t is the innovation of the return series. v t is the logarithm of variance at time t, ɛ t and η t are i.i.d. Gaussian (or student s t) distributed noises. τ is unknown scaling parameters to be estimated. The function f is the hidden state transition function. Here we assume this function f follows a Gaussian process, which is defined by the mean function m(x) and covariance function k(x, x ). We show the graphical model representation of GPRSV model in Fig. 1. 359

15 IEEE Global Conference on Signal and Information Processing (GlobalSIP) f 1 f f * mdoel with RAPCF algorithm. For particle filter review, see [13] and [14]. v v 1 v a a 1 a... Fig. 1. Graphical model representation of a Gaussian process regression stochastic volatility model, where a t is the observation variable at time t, and v t are the hidden variable (logarithm of volatility) at time t. f t is the Gaussian process sampled function value at time t, and the thick horizontal line represent fully connected nodes. Hyper-parameters of the Gaussian process are omitted in the figure. In GPRSV model we use logarithm of variance instead of standard deviation directly in our model. The advantage of using logarithm form is explained in [3] and [4]. We build a GPRSV model with the following four steps: Step 1: specify mean equation. First we need to test the serial dependence in the return series. If the series are linear dependent, we should use an econometric model to remove the linear dependence in return series. Step : test ARCH effect. The residuals of the asset return a t expressed in (8a) are often used to test the series conditional heteroscedasticity. This conditional heteroscedasticity is also known as the ARCH effects [9]. Step 3: specify volatility equation. We need to specify both the mean and covariance functions for Gaussian process. Besides these functions, the initial value of hyperparameters associated with them need to be specified as well. Step 4: estimate model parameters and check model fitness. We use training data to estimate the unknown parameters. Once we get our estimated parameters we can use testing data to test learned model, and it is necessary to check the fitness of model. V. INFERENCE FOR THE GPRSV MODEL It is much more challenging to learn a GPRSV model than its parametric competitors. We target at jointly learning the hidden states trajectory and the unknown Gaussian process regression function values and hyper-parameters. Our approach is marginalizing out the Gaussian process regression function values, and then jointly learning the hidden volatility states and hyper-parameters using Monte Carlo methods. Most of previous work on inference GP-SSMs focused on filtering and smoothing hidden variables without jointly learning the Gaussian process hyper-parameters. In [1], a novel particle Markov chain Monte Carlo (particle MCMC) algorithm, particle Gibbs with ancestor sampling (PGAS) was proposed. In [11], Frigola et al. apply the algorithm to learn a GP-SSM s hidden states and Gaussian process dynamics jointly. In [1], a regularized auxiliary particle filter which the authors named as Regularized Auxiliary Particle Chain Filter (RAPCF) was introduced. In Algorithm 1 we show how to learn a GPRSV v * a * Algorithm 1 RAPCF for GPRSV Model 1: Input: return data r 1:T, number of particles N, shrinkage parameter <λ<1, prior p(θ). : Remove linear dependence from r 1:T to get the residuals a 1:T. 3: Sample N parameter particles from the prior, and set initial importance weights, W i =1/N 4: for t =1to T do 5: Shrink parameter particles towards empirical means θ t 1 =Σ N i=1wt 1θ i t 1 i (9a) θ t i = λθt 1 i +(1 λ) θ t 1 (9b) 6: Compute expected states: 7: Compute important weights μ i t = E(v t θ i t,v i 1:t 1) (1) g i t W i t 1p(a t μ i t, θ i t) (11) 8: 9: Resample N auxiliary indices {j} according to {gt}. i Propagate the chains of v t forward, {v j 1:t 1 } j J. 1: Add jitter: θt 1 i N(θ j t, (1 λ )V t 1 ), and V t 1 is empirical covariance of θ t 1. 11: Propose new states v j t p(v t θ j t,v j 1:t 1,a 1:t 1) 1: Adjust weights with newly proposed states: W i t p(a t v j t, θ j t )/p(a t μ j t, θ j t ) (1) 13: end for 14: Output: particles of v j 1:T, particles of θj t and particle weights W j t. VI. EXPERIMENTS We apply both the simulated and empirical financial data to demonstrate performance of our GPRSV model and inference method. For simulated data, we generated one synthetic dataset of length T = according to equation 8. We sample our hidden state transition function f from a Gaussian process prior. We specified that the mean function m(x t ) and the covariance function k(y, z) as follow: m(x t )=ax t 1 k(y, z) =γexp(.5 y z /l ) (13a) (13b) We show the learned hidden states density for the simulated data in Fig. and the predictive Log-likelihood value compared with the true value in Fig. 3. For real financial data, we chose IBM stock daily closing price data. The data period is from January 1, 1988 to September 14, 3. We compare our GPRSV model with standard parametric volatility models: GARCH, GJR-GARCH and standard SV. For GPRSV model, the Gaussian process dynamics are specified as follows: the mean function is m(x t )=ax t 1 36

15 IEEE Global Conference on Signal and Information Processing (GlobalSIP) 6 5 4 3 1 Evolution of the state density of the most widely-used loss functions in financial research literature. In our experiment, we adopt a class of statistical loss functions instead of a particular one. We take the following loss functions: MAD : L(ˆσ t+m,σ t+m )=n 1 ˆσ t+m σ t+m (14) 1.5 1.5 -.5-1 -1.5 - Fig.. Estimated hidden states densities of simulated Data. There are iteration steps for the simulated data, and we plot every 5 densities in this figure. The distribution densities are generated using particles and weights as described in Algorithm 1. Value -.8-1 -1. -1.4-1.6-1.8 Predictive Log-likelihood Values 4 6 8 1 1 14 True Value GPRSV 16 18 MLAE : L(ˆσ t+m,σ t+m )=n 1 QLIKE : L(ˆσ t+m,σ t+m) =n 1 HMSE : L(ˆσ t+m,σ t+m) =n 1 n log( ˆσ t+m σ t+m ) (15) (ˆσ t+m/σ t+m+logσ t+m) (ˆσ t+m/σ t+m 1) (16) (17) In our experiment, we apply the high frequency data to calculate the realized volatility [16] and make it as the proxy for true volatility. Compared with the squared return, realized volatility is considered to be more precise proxy for volatility forecast evaluation. The loss function result is presented in Table II. TABLE II RESULTS OF IBM VOLATILITY FORECAST USING LOSS FUNCTIONS - -. 5 1 15 Time(t) Fig. 3. RAPCF algorithm learned predictive Log-likelihood value are compared with true value calculate from equation 8. We discard the first 5 burn in iterations. and the covariance function is the squared exponential covariance function k(y, z) =γexp(.5 y z /l ). The hyperparameters include a, γ, l and likelihood function parameter log(sn). The learned parameters are presented in Table I. There is many metrics to evaluate different forecast models. One of the most popular approach is using a particular statistical loss function, the model which achieved a minimized loss function value is the best forecasting model [15]. There are extensive choices of lost functions. QLIKE function is one TABLE I ESTIMATED GPRSV MODEL HYPER-PARAMETERS RESULTS FOR IBM DAILY RETURN DATA a γ l log(sn) 1.8777 3.364 1.344-1.7664 Model MAD MLAE HMSE QLIKE GARCH 3.787.7876 9.3797.13466 GJR-GARCH 3.79.7717 6.9648.917 SV 3.8.7944 9.1739.131 GPRSV 3.367.677 1.8617 1.8549 Note: The lowest loss function value is marked using the bold text. GPRSV model obtained the lowest loss function values for all loss functions. The volatility proxy is the 65-minutes sampled realized volatility. VII. CONCLUSIONS In this paper, we proposed a Gaussian process regression based volatility model for the problem of predicting the time varying volatility of financial time series data. After we introduced the GPRSV model, we gave a solution to jointly learning the hidden volatility states and the Gaussian process dynamics. Based on our experiment results, we can successfully learn the Gaussian process regression stochastic volatility model s hidden states and hyper-parameters. Also the Gaussian process regression based stochastic volatility models can achieve better performance compared with the standard economic parametric models. 361

15 IEEE Global Conference on Signal and Information Processing (GlobalSIP) REFERENCES [1] R. F. Engle, Autoregressive conditional heteroscedasticity with estimates of the variance of united kingdom inflation, Econometrica: Journal of the Econometric Society, pp. 987 17, 198. [] T. Bollerslev, Generalized autoregressive conditional heteroskedasticity, Journal of Econometrics, vol. 31, pp. 37 37, 1986. [3] S. J. Taylor, Modelling financial time series, 1986. [4] D. Nelson, Conditional Heteroskedasticity in Asset Returns: A New Approach, Econometrica, vol. 59, no., pp. 347 7, March 1991. [5] L. Glosten, R. Jagannathan, and D. Runkle, On the relation between the expected value and the volatility of the nominal excess return on stocks, The Journal of Finance, vol. 48, no. 5, pp. 1779 181, 1993. [6] S. J. Taylor, Financial returns modelled by the product of two stochastic processes, a study of daily sugar prices. Oxford University Press, 5. [7] S. Neil and T. Andersen, Stochastic Volatility: Origins and Overview, University of Oxford, Department of Economics, Economics Series Working Papers 389, 8. [8] R. E. Kalman, A new approach to linear filtering and prediction problems, Transactions of the ASME Journal of Basic Engineering, vol. 8, no. Series D, pp. 35 45, 196. [9] R. Tsay, Analysis of financial time series. Wiley, 1. [1] F. Lindsten, M. Jordan, and T. Schön, Particle gibbs with ancestor sampling, Journal of Machine Learning Research, vol. 15, pp. 145 184, 14. [11] R. Frigola, F. Lindsten, T. B. Schön, and C. E. Rasmussen, Bayesian inference and learning in gaussian process state-space models with particle MCMC, in Advances in Neural Information Processing Systems 6, 13, pp. 3156 3164. [1] Y. Wu, J. M. Hernández-Lobato, and Z. Ghahramani, Gaussian process volatility model, in Advances in Neural Information Processing Systems, 14, pp. 144 15. [13] M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking, Signal Processing, IEEE Transactions on, vol. 5, no., pp. 174 188,. [14] C. Hue, J.-P. Le Cadre, and P. Perez, Sequential monte carlo methods for multiple target tracking and data fusion, Signal Processing, IEEE Transactions on, vol. 5, no., pp. 39 35, Feb. [15] C. Brownlees, R. Engle, and B. Kelly, A practical guide to volatility forecasting through calm and storm, Journal of Risk, vol. 14(), pp. 1, 11. [16] A. Torben, T. Bollerslev, F. Diebold, and P. Labys, Modeling and forecasting realized volatility, Econometrica, vol. 71, no., pp. 579 65, 3. 36