Project exam for STK Computational statistics

Transcription

1 Project exam for STK Computational statistics Fall 2017 Part 1 (of 2) This is the first part of the exam project set for STK4051/9051, fall semester It is made available on the course website on Monday 2 October, and candidates must submit their written reports by Monday 15 October 13:00 (or earlier), to the reception office at the Department of Mathematics, in two copies. Reports may be written in Norwegian or English, and should preferably be textprocessed (LaTeX, Word). Give your student number on the first page. Write concisely. Relevant figures need to be included in the report. Copies of relevant parts of machine programs used (in R, or matlab, or similar) are also to be included, perhaps as an appendix to the report. Candidates are required to work on their own (i.e. without cooperation with any others). They are graciously allowed not to despair should they not manage to answer all questions well. This exam set contains four exercises and comprises four pages (with an extra fifth page, see below). Data and some R-code is available from the course web-page. You are free to use other software, but would then need to translate or write your own code for that part included in the R-script. Importantly, each student needs to submit one special extra page with her or his report. This page is the self-declaration form, properly signed; it is available on the last page of this project set. The second part of the exam project set will be made available in the beginning of November. The supplementary four-hour no-book written examination takes place Thursday November 30 (practical details concerning this are provided elsewhere). 1

2 Exercise 1 (Discrete optimization). We will in this exercise consider data on the volume of the S&P500 index from Yahoo Finance (https: // finance. yahoo. com/ ) for the period from 1/1/1950 to 9/9/2012. The data is available at the course web-page: https: // www. uio. no/ studier/ emner/ matnat/ math/ STK4051/ h17/ data/ sp500. txt and can be read into R with the command webdir = https : //www. uio. no/ s t u d i e r /emner/matnat/math/stk4051/h17/ data / sp500 = read. t a b l e ( paste ( webdir, sp500. txt, sep= ) ) Due to the strong trend and also temporal dependence, analysis of such data are in most cases performed on the log-return scale: x t = log(v t /v t 1 ) where v t is the volume at time t. (a). Plot both the volume itself and the log-return data as a function of time. Evaluate how well the data fit to a Gaussian distribution. Yahoo Finance Although there seem to be no trend and less temporal correlation in this case, there still seems to be some differences in the variability at different periods. We will model these log-return data by a mixture of Gaussian distributions through a Hidden Markov model: X t C t = k N(0, σ 2 k) Pr(C t = l C i 1 = k) =p kl, Pr(C 1 = k) =1/K k, l = 1,..., K Our final aim will be to estimate p(l k), k, l = 1,..., K and σ 2 k, k = 1,..., K as well as the hidden variables {C t, t = 1,..., n}. Assume however first the parameters are fixed to K =3, σ 2 =(0.1, 0.55, 1.0), 0.9 for l = k; p kl = 0.05 for l k. Our interest in the beginning will then be to optimize p(c x; θ) where C = (C 1,..., C n ), x = (x 1,..., x n ) and θ is the vector of parameters (so far assumed known). (b). Show that n p(c x; θ) Pr(C 1 )f C1 (x 1 ) p ci 1,c i f Ci (x i ) i=2 where f k (x i ) is the density for x i C i = k. Discuss why it may be better to consider p(c x; θ) on a log-scale. 2

3 (c). Implement a greedy (local search) algorithm for optimizing p(c x; θ). Specify how you choose initial values. Calculate p(c x; θ) based on your result. Make a plot of x and C as functions of time (perhaps zoomed in on a smaller time-frame) and discuss the results. (d). Implement simulated annealing for optmization of p(c x; θ). Specify how you choose the initial temperature and the temperature schedule. Calculate p(c x; θ) based on your result. Make a plot of x and C as functions of time (perhaps zoomed in on a smaller time-frame) and discuss the results. Exercise 2 (The EM-algorithm). We will continue to work on the log-return values of the S&P500 data. Consider now however the problem of simultaneous estimation of C and θ. We will in this exercise apply the EM-algorithm. On the course web-page there is an R-script, HMM.E.R which calculates q i i 1 (k) = Pr(C i = k x 1,..., x i 1 ) q i i (k) = Pr(C i = k x 1,..., x i ) q i n (k) = Pr(C i = k x 1,..., x n ) Prediction Updating Smoothing as well as q i 1,i n (k, l = Pr(C i = l, C i 1 = k x 1,..., x n ) which is based on the forward-backward algorithm discussed in the lecture. (a). Make a function which, given the q s above, calculates estimates for the unknown parameters (the M-step in the EM-algorithm). (b). Combine your function with the one calculating the q s to implement a function which performs the EM-algorithm. Specify your convergence criterion. (c). Include a routine which calculates the log-likelihood value for a given value of θ. Hint: Use that l(θ) = log f(x; θ) n = log f(x 1 ; θ) + log f(x i x 1,..., x i 1 ; θ) i=2 and show that the terms f(x i x 1,..., x i 1 ; θ) are possible to be calculated by the output from the HMM.E.R script. (d). How does the number of computational steps in this algorithm depend on n and K? (e). Run the EM-algorithm on the log-return data. Try out different starting values. Confirm that the log-likelihood value is non-decreasing with the number of iterations in the EM-algorithm. Discuss the results. 3

4 (f). Also try out with K = 2. Use some appropriate model selection criterion in order to choose between K = 2 and K = 3. For your best model, make a similar plot to the ones you made in Exercise 1, (c) and (d). Exercise 3 (Direct parameter optimization). An alternative to the EM-algorithm is to implement a function which directly calculates the log-likelihood for a given set of parameters and then throw this function into a numerical optimizer. One problem that needs to be considered in this case is the constraints on the parameters involved. (a). Specify which constraints that are involved in the parameter vector θ = {p kl, k, l = 1,..., K, σ 2 k, k = 1,..., K}. How many free parameters do we then have? (b). Suggest some reparametrization of θ which removes the constraints on the parameters. Make sure that this reparametrization is invertible. (c). Modify the routine you implemented in Exercise 2(c) to calculate the log-likelihood using the reparametrized parameters as input. Optimize l(θ) through some numerical optimizer, both for K = 2 and K = 3. Hint: It might be useful to start with K = 2 since in that case there are much less parameters. You might expect that this optimization both is time-consuming and non-stable. Exercise 4 (Summary). Write a (maximum) one-page summary of the previous exercises where you in particular consider the following points: (a). Ways of obtaining uncertainty measures on the parameter estimates. (b). The use of the EM algorithm compared with the direct optimization route for this problem. (c). The results you obtained. dependence in the C i s? In particular, is there a need to include the temporal 4

5 Declaration Print, fill out, sign, and deliver the Department of Mathematics in connection with submission of the project exam: I hereby declare that part 1 of my home exam report, delivered for the course (crossed) STK 4051 STK 9051 at the Department of Mathematics, University of Oslo, (a). have not been used for another exam at another department or university or college, home or abroad; (b). does not refer to the work of others without this being stated; (c). does not refer to own previous work without this being stated; (d). has provided all references in the literature list; (e). is not a collaboration with one or more others. I am aware that violations of these provisions are considered cheating and that violates university regulations. Oslo, the Student s signature: Student Name [in clear block letter]: 5