Copula information criterion for model selection with two-stage maximum likelihood estimation

Transcription

1 Copula information criterion for model selection with two-stage maximum likelihood estimation Vinnie Ko, Nils Lid Hjort Department of Mathematics, University of Oslo PB 1053, Blindern, NO-0316 Oslo, Norway February 2018 Abstract In parametric copula setups, where both the margins and copula have parametric forms, two-stage maximum likelihood estimation, often referred to as inference functions for margins, is used as an attractive alternative to the full maximum likelihood estimation strategy. Exploiting basic results derived earlier by the present authors, we develop a copula information criterion (CIC) for model selection. The CIC is defined as CIC = 2l n( η) 2 p η, where l n( η) is the maximized log-likelihood under the two-stage maximum likelihood estimation scheme, with η the full parameter vector for the candidate model in question, and p η is a certain penalization factor. In a nutshell, CIC aims for the model that minimizes the Kullback Leibler divergence from the real data generating mechanism. CIC does not assume that the chosen parametric model captures this true model, unlike what is assumed for AIC. In this sense CIC is analogous to the Takeuchi Information Criterion (TIC), which is defined for the full maximum likelihood. If we make an additional assumption that a candidate model is correctly specified, then CIC for that model simplifies to AIC. Further, since both CIC and TIC are estimating the same part of the Kullback Leibler divergence, they are compatible, in the sense that they can be used to compare the performance of full maximum-likelihood and two-stage maximum likelihood for a given model. Additionally, we show that CIC can easily be extended to the conditional copula setup where covariates are parametrically linked to the copula model. As a numerical illustration, we perform a simulation and find that CIC outperforms AIC in terms of prediction performance from the selected models. However, as sample size grows, the difference between CIC and AIC becomes minimal because the log-likelihood part outgrows the bias correction part. Further, we learn from the simulation that p η, the bias correction term of CIC, has a strong positive relationship with the prediction performance of the model. So, a model with bad prediction performance is being penalized more by CIC. Keywords: copula, Akaike information criterion, copula information criterion, model robust, two-stage maximum likelihood, inference functions for margins, Corresponding author. addresses: vinniebk@math.uio.no (V. Ko), nils@math.uio.no (N.L. Hjort) 1

2 1 Introduction and copula models One of the main practical issues in copula modeling is model selection. In the full parametric setup, where both the copula and margins are assumed to have a parametric form, one often has multiple candidates for both the copula and margins. As the dimension of the model increases, a list of possible combinations of margins and copula grows rapidly. Hence, there is a need for a model selection criterion that can evaluate each model systematically according to certain philosophy or criteria and assign a score to each model. In the end, one would choose the model with the best score. Throughout this paper, we consider the full parametric setup. In this setup, one can simultaneously estimate all parameters of the model (i.e. both copula parameters and margin parameters) by using maximum likelihood (ML) estimation. In this ML estimation framework, one can for instance use AIC ML (Akaike, 1974) or TIC (also known as model-robust AIC ML ) (Takeuchi, 1976) as model selection criterion and select the model with the best score. (Note that we denote the AIC under ML estimation as AIC ML to distinguish it from the two-stage ML based AIC 2ML, which we will derive in Section 2.3.) However, when the dimension of the copula model gets high, the number of parameters increases quickly and the ML estimation is not always feasible in terms of speed and numerical stability. Two-stage maximum likelihood (two-stage ML) estimation, also often referred to as inference functions for margins (IFM), is a popular alternative estimation strategy that is designed to overcome these drawbacks of the ML estimation. In stage 1 of the two-stage ML estimation, the parameter vectors of each marginal distribution are estimated separately by ML. In stage 2, the estimates from stage 1 are plugged into the log-likelihood of the model. Then, the parameters of the copula, which are now the only unknown parameters, are estimated by using ML estimation again. One of the advantages of this multi-stage approach is that it is computation-wise much faster than estimating all parameters simultaneously, because it does not have to search for the global maximum in high-dimensional space. A drawback of the two-stage ML estimation method, however, is that we cannot use the classical results based on ML estimation, which include model selection criteria such as TIC and BIC. In practice, different sorts of goodness-of-fit testing are often used as substitutes, to choose the best model (Genest & Favre, 2007). Another often used model selection strategy for the two-stage ML is that one first evaluates candidates of each marginal distribution with AIC ML and consequently chooses the best distribution for each margin. Once the margins are chosen, one fits different copulae and evaluates the copula part with AIC ML. However, this piecewise model evaluation cannot evaluate the model as a whole. In this paper, we develop the copula information criterion (CIC) for two-stage ML estimation, which has the form CIC = 2l n ( η) 2 p η. Here l n ( η) is the maximized log-likelihood with the two-stage ML estimation method, in terms of the full parameter vector η of the model in question, and p η is a suitable penalization factor, worked out in Section 2.2. The main advantage of CIC is that it can evaluate a parametric copula with parametric margins as a whole. CIC is also a model-robust model selection criterion which means that it does not assume that the candidate model contains the true model. As the overlap of the name already suggests, our CIC is analogous to CIC from Grønneberg & Hjort (2014), which is designed for copulae estimated with pseudo maximum likelihood (PML). In PML framework, margins are estimated empirically, while two-stage ML assumes parametric forms of margins. 2

3 Our technical setting, identical to Ko & Hjort (2018), is as follows. Let (Y 1,, Y d ) T be a d-variate continuous stochastic variable originating from a joint density g(y 1,, y d ) and let y i = (y i,1,, y i,d ) T, for i = 1,..., n, be independent observations of this variable. The true joint distribution g is typically unknown. Let f(y 1,, y d, η) be our parametric approximation of g, with the parameter vector η, belonging to some connected subset of the appropriate Euclidean space. In addition, G and F (, η) indicate cumulative distribution functions corresponding to g and f(, η), respectively. Here G j (y j ) and F j (y j, α j ) indicate j-th marginal distribution functions corresponding to G and F (, η) respectively, with α j as the parameter vector belonging to margin component j. According to Sklar s theorem (Sklar, 1959), there always exists a copula C(u 1,..., u d, ) that satisfies F (y 1,, y d, η) = C(F 1 (y 1, α 1 ),, F d (y d, α d ), ) where the full parameter vector η is now blocked as η = (α T, T ) T = (α T 1,, α d T, T ) T. By assuming the regulatory conditions from Ko & Hjort (2018), C(, ) can be differentiated, d f(y 1,, y d, η) = c (F 1 (y 1, α 1 ),, F d (y d, α d ), ) f j (y j, α j ), where c(u 1,..., u d ) = d C(u 1,..., u d, )/ u 1 u d and f j (y j, α j ) = F j (y j, α j )/ y j. For further details of copula modeling, see Joe (1997) and Nelsen (2006). Analogously, the true density g can also be decomposed into marginal densities and the copula density d g(y 1,, y d ) = c 0 (G 1 (y 1 ),, G d (y d )) g j (y j ), with c 0 ( ) the true copula. The further structure of this paper is as follows. In Section 2.1, we briefly explain Kullback Leibler divergence and its relationship to TIC and AIC ML. In Section 2.2, we derive and define our copula information criterion. In Section 2.3, we prove that the AIC 2ML formula holds under the two-stage ML estimation. In Section 2.4, we summarize the relationship between TIC, CIC, AIC ML and AIC 2ML. In Section 2.5, we illustrate what CIC looks like in the two-dimensional setting and show how CIC easily can be extended to the conditional copula setting. In Section 3, we study the numerical behavior of those model selection criteria. In our final Section 4, we offer a few concluding remarks and suggestions for future research. j=1 j=1 3

4 2 The copula information criterion for two-stage maximum likelihood estimation 2.1 Kullback Leibler divergence The Kullback Leibler (KL) divergence from g to f measures how the density f diverges from g (Kullback & Leibler, 1951) and is defined as KL(g, f) = g(y) log g(y) f(y) dy = g(y) log g(y)dy g(y) log f(y)dy. It is well known that the ML estimator aims for parameter values that minimize the Kullback Leibler divergence (Akaike, 1998). Consider a case where one has competing models for certain data and the parameters are estimated by ML. An arbitrary candidate model with ML parameter estimates can be denoted as f(y, η). (Throughout this paper, we use to indicate that a quantity is estimated with ML and to indicate that a quantity is estimated with two-stage ML.) Since g(y) is the same across all candidate models, minimizing KL divergence is equal to maximizing Q( η) = g(y) log f(y, η)dy. This quantity is, however, not directly observable since g(y) is unknown. As an alternative, one may use the empirical equivalent of Q( η): Q( η) = 1 n l( η) = 1 n log f 1 (y i,1, α 1 ) + + log f d (y i,d, α d ) + log c ( F 1 (y i,1, α 1 ),, F d (y i,d, α d ), )]. Yet, this estimator of Q( η) is a biased estimator. By identifying and subtracting the bias, we obtain the unbiased estimator of Q( η): Q ( η) = 1 n l( η) 1 n tr( Î 1 η K η ). where Îη is the observed information and K η is the estimated covariance matrix of η. The TIC (Takeuchi, 1976) aims for the model that maximizes Q ( η) and is defined as TIC = 2l( η) 2 p η,tic, (1) where p η,tic = tr( ) Îη 1 K η. This shows that TIC is basically a scaled version of Q ( η). When one boldly makes the assumption that the candidate model is correct, i.e. contains the true data generating mechanism, TIC simplifies to AIC ML, possibly the most well known model selection criterion in statistics, which is defined as AIC ML = 2l( η) 2p η, (2) where p η is the length of the parameter vector η. Note that the formula of TIC and AIC ML are only valid if the parameters are estimated by the ML estimator. For more details about TIC and AIC ML, see chapter 2 of Claeskens & Hjort (2008). 4

5 2.2 Derivation of the copula information criterion When the copula model is estimated with the two-stage ML, the bias correction term of TIC, i.e. p η,tic, is not valid. We derive the copula information criterion (CIC) which is analogous to TIC and is made for copula models estimated with the two-stage ML. is When the copula model is estimated with the two-stage ML, the non-constant part of the KL divergence Q( η) = { g(y) log f 1 (y 1, α 1 ) + + log f d (y d, α d ) + log c ( F 1 (y 1, α 1 ),, F d (y d, α d ), )} dy. The empirical equivalent is Q( η) = 1 n Now we check the bias of Q( η): log f 1 (y i,1, α 1 ) + + log f d (y i,d, α d ) + log c ( F 1 (y i,1, α 1 ),, F d (y i,d, α d ), )]. E G Q( η) ] Q( η) = EG1 1 n E Gd n 1 + E G n ] log f 1 (y i,1, α 1 ) ] log f d (y i,d, α d ) log c g(y) log c g(y 1 ) log f 1 (y 1, α 1 )dy 1 g(y d ) log f d (y d, α d )dy d ( F 1 (y i,1, α 1 ),, F d (y i,d, α d ), ) ] ( F 1 (y 1, α 1 ),, F d (y d, α d ), ) dy. Since the parameters of marginals from stage 1 ( α j s) are obtained with ML estimation, we can directly use the results from the derivation of TIC and obtain ] 1 E Gj log f j (y i,j, α j ) g(y j ) log f(y j, α j )dy j = 1 ( ) n n tr Iα 1 j K αj + o(n 1 ) = 1 n p α j + o(n 1 ), with Iα 1 j and K αj as defined in Lemma 1 of Ko & Hjort (2018). In a nutshell, I αj is the Fisher information of j-th margin and K αj is the covariance matrix of the score vector that belongs to j-th margin. Further, let Q c ( η) = g(y) log c ( F 1 (y 1, α 1 ),, F d (y d, α d ), ) dy and Q c ( η) = E 1 n log c ( F 1 (y i,1, α 1 ),, F d (y i,d, α d ), )]. 5

6 So, we can write E G Q( η) ] Q( η) = 1 n d j=1 ( ) ] tr Iα 1 j K αj + E G Qc ( η) Q c ( η) + o(n 1 ). ] Now, E G Qc ( η) Q c ( η) is the only element that should be evaluated. Let Q c (η 0 ) = g(y) log c (F 1 (y 1, α 0,1 ),, F d (y d, α 0,d ), 0 ) dy, Z i = log c (F 1 (y i,1, α 0,1 ),, F d (y i,d, α 0,d ), 0 ) Q c (η 0 ), A η = n ( η η 0 ) = ( ) n Iη 1 U n,α (α 0 ) U n, (α 0, 0 ) Iα, T I 1 α U n,α (α 0 ) which stems from Proposition 1 in Ko & Hjort (2018), and furthermore ( ) o p (1) +, o p (1) U n,η (η) = 1 U η (y i, η) = 1 log c (F 1 (y i,1, α 1 ),, F d (y i,d, α d ), ), n n η H n,η (η) = 1 H η (y i, η) = 1 2 log c (F 1 (y i,1, α 1 ),, F d (y i,d, α d ), ) n n η η T, Iη = E G H η (y, η 0 )] = g(y)h η (y, η 0 ) dy, which, incidentally, should not be confused with I η in Proposition 1 of Ko & Hjort (2018). Then we have EZ n ] = 1 n n EZ i] = 0, along with Q c ( η) = 1 n = 1 n = 1 n log c(y i, η) log c(y i, η 0 ) Q c (η 0 ) + Q c (η 0 ) + ( η η 0 ) T U η (y i, η 0 ) + 1 ] 2 ( η η 0) T H η (y i, η 0 )( η η 0 ) + o p (n 1 ) Z i ] + Q c (η 0 ) + ( η η 0 ) T U n,η (η 0 ) ( η η 0) T H n,η (η 0 )( η η 0 ) + o p (n 1 ) = Q c (η 0 ) + Z n + 1 n A T η U n,η (η 0 ) + 1 2n AT η H n,η (η 0 )A η + o p (n 1 ), 6

7 Q c ( η) = = = and ( g(y) log c ) dy F 1 (y 1, α 1 ),, F d (y d, α d ), g(y) log c(y, η 0 ) + ( η η 0 ) T U η (y, η 0 ) + 1 ] 2 ( η η 0) T H η (y, η 0 )( η η 0 ) dy + o p (n 1 ) g(y) log c(y, η 0 ) dy + ( η η 0 ) T g(y)u η (y, η 0 ) dy ( η η 0) T g(y)h η (y, η 0 ) dy ( η η 0 ) + o p (n 1 ) = Q c (η 0 ) + ( η η 0 ) T n( η η0 ) T g(y)h η (y, η 0 ) dy n( η η 0 ) + o p (n 1 ) 2n = Q c (η 0 ) 1 2n AT η I ηa η + o p (n 1 ) n{ Q c ( η) Q c ( η)} = nz n + na T η U n,η (η 0 ) AT η H n,η (η 0 )A η AT η I ηa η + o p (1). Further, note that U n,η (η 0 ) = 1 n has the following convergence, by the central limit theorem: log c (F 1 (y i,1, α 0,1 ),, F d (y i,d, α 0,d ), 0 ) η 0 nun,η (η 0 ) = n {U n,η (η 0 ) E U η (Y, η 0 )]} d Λ η N ( 0, Kη ) where K η = Var (U η (y, η 0 )) = E U η (y, η 0 ) U η (y, η 0 ) T]. (Here Λ η and K η should not be confused with Λ η and K η in Proposition 1 of Ko & Hjort (2018).) Now we evaluate E G n{ Q c ( η) Q c ( η)}]: ( )] E G n Qc ( η) Q c ( η) = E G nz n + na T η U n,η (η 0 ) AT η H n,η (η 0 )A η + 1 ] 2 AT η IηA η + o(1) p (I ) 1 T ] E G η L Λ η Λ η ( = E G tr Iη 1 ( ) )] L Λ η Λ T η = tr ( Iη 1 L E G Λη (Λ η) T]) = tr ( Iη 1 L Kη) = p, where K η = E G Λη (Λ η) T] = E G ( Λ α (Λ α) T Λ (Λ α) T )] Λ α Λ T = K α K α, ( ) T. Λ Λ T K K α, 7

8 It is practical to note that tr ( Iη 1 L Kη ( ) = tr Iα 1 0 = tr = tr 0 I 1 I 1 (( I 1 ) ( I T α, I 1 α ) I 0 K α K α, ( ) T I Kα, K I T α, I 1 α I 1 α Kα 0 0 I 1 Iα, T I 1 α α Kα Iα 1 ( ) T Kα + I 1 Kα, I 1 Iα, T I 1 α K α, + I 1 K )). K α, K α, + I 1 Consistent estimators for Iα 1 j, K αj, Iη 1, L and Kη can be obtained by using plug-in sample averages. The consequent estimators are denoted as Ĩ 1 α j, Kαj, Ĩ 1 η, L, K η. For regularity conditions that ensure convergence in probability, see Jullum & Hjort (2017). Thus, the unbiased estimator of Q( η) is Q ( η) = 1 n l n( η) 1 n d j=1 (Ĩ 1 ) (Ĩ 1 tr α Kαj j + tr η L K ) η. By defining p α j = tr(ĩ 1 α Kαj j ), p = tr(ĩ 1 η L K η), p η = d j=1 p α j + p and scaling Q ( η), we can finally define the copula information criterion as K CIC = 2l n ( η) 2 p η. (3) 2.3 AIC for two-stage maximum likelihood estimator The CIC, derived in Section 2.2, is a model robust model selection criterion. This means that the CIC does not assume that the parametric model includes the true model that generated data. In this section we show that, if we do make such a true model assumption, the CIC simplifies to the AIC 2ML. To our knowledge, this is the first time that the validity of the AIC 2ML formula is proven for the two-stage ML estimator. Lemma 1. Under the assumption that the margins and copula are correctly specified, it holds that Kα = I α K α. Proof. Assume the candidate model worked with contains the true data generating mechanism, i.e. that 8

9 f = g at the relevant parameter point. Then 0 = E G U α (y, α 0 )] T α 0 = f d j=1 log f j α 0 α0 T dy d f j=1 = log f j α 0 α0 T + f 2 d j=1 log f j α 0 α0 T dy = f log f d j=1 log f j α 0 α0 T + f 2 d j=1 log f j α 0 α0 T ( d j=1 = f log f ) j + log c d j=1 log f j = α 0 f d j=1 log f j α 0 α 0 d j=1 log f j α T 0 α T 0 dy log c dy + α 0 + f 2 d j=1 log f j α 0 α0 T d j=1 log f j α T 0 dy dy + = E G Uα (y, α 0 ) U α (y, α 0 ) T] + E G U α (y, α 0 ) U α (y, α 0 ) T] E G H α (y, α 0 )] = K α + K α I α. f 2 d j=1 log f j α 0 α0 T dy If we make the true model assumption (i.e. f = g), we have from the classical results of maximum likelihood theory that I αj = K αj. This implies that we have for the bias correction term in the marginal parameters p α j = tr(iα 1 j K αj ) = dim(α j ), for each j. For the bias correction term in the copula parameters part, the true model assumption results in Lemma 1. Combining Lemma 1 with Lemma 3 and Lemma 5 from Ko & Hjort (2018) gives tr ( Iη 1 L Kη ) ((I 1 = tr α Kα 0 0 I 1 Iα, T I 1 α (( )) I Iα 1 K α 0 = tr 0 I (( )) 0 0 = tr = dim(). 0 I Thus, the unbiased estimator Q ( η) can be simplified as K α, + I 1 K Q ( η) = 1 n l n( η) 1 d dim(α j ) + dim(). n j=1 By defining p η = d j=1 dim(α j) + dim() = dim(η) and scaling Q ( η), we obtain AIC 2ML for the two-stage ML estimated copula models )) AIC 2ML = 2l n ( η) 2p η. (4) 9

10 Compared to AIC ML from Section 2.1, the only difference is that the log-likelihood is now estimated under the two-stage ML instead of ML. i.e. we use η instead of η. 2.4 Relationship between the model selection criteria So far, we have discussed four model selection criteria for copula models. Table 1 shows an overview of the relationship between them. When the model (i.e. both copula and margins) is correctly specified, CIC and AIC 2ML become equal, and the same happens between TIC and AIC ML. Thus, one can compare CIC and AIC 2ML (or TIC and AIC ML in case of ML estimation) to check whether the model is correctly specified. Further, since both CIC and TIC are estimating the same part of the KL divergence under the same model robust environment, they are compatible. This implies that one can compare CIC to TIC to measure how much one looses in terms of KL divergence by using two-stage ML estimation instead of ML estimation. The same can be done by comparing AIC 2ML to AIC ML when one believes in the model. However, one should not compare CIC with AIC ML (or TIC with AIC 2ML ) since they are based on two different model beliefs (i.e. presence of the true model assumption). Figuratively speaking, one is in this situation comparing apples with pears. Model robust Yes No Estimation ML TIC AIC ML 2ML CIC AIC 2ML Table 1: An overview of the relationship between model selection criteria discussed in this paper. 2.5 Illustration for two-dimensional case and extension to the conditional copula regression. To make things more concrete, we now give an example of the two-dimensional case with (Y 1, Y 2 ) from the unknown g(y). As candidate margins we choose a two-parameter distribution (e.g. normal, gamma, Weibull, etc.) for both F 1 and F 2. For the copula part, we choose a one-parameter copula (e.g. Gumbel, Frank, Clayton, etc.). The candidate model then has the form f(y 1, y 2, η) = c (F 1 (y 1, α 1 ), F 2 (y 2, α 2 ), ) f 1 (y 1, α 1 ) f 2 (y 2, α 2 ) where η = (α 1,1, α 1,2, α 2,1, α 2,2, ) T. The ingredients for the CIC can then be written down by using the block matrix form, in line with Ko & Hjort (2018): K α1 K α1,α 2 K α1, I α1 0 0 K η = K α2,α 1 K α2 K α2,, I η = 0 I α2 0, Kα T 1, Kα T 2, K 0 0 I I Kα 1 Kα 1,α 2 K α1, L = 0 I 2 2 0, Kη = Kα 2,α 1 K α 2 K α2, ( ) T ( T. Iα T 1, I 1 α 1 Iα T 2, I 1 α 2 I 1 1 Kα 2,) K K α 1, 10

11 We consequently have p η = p α 1 + p α 2 + p = tr ( Iα 1 ) ( ) ( 1 K α1 + tr I 1 α 2 K α2 + tr I 1 η L Kη ) = tr ( Iα 1 ) ( ) ( 1 K α1 + tr I 1 α 2 K α2 + tr I 1 α 1 Kα ) ( 1 + tr I 1 α 2 Kα ) 2 + tr ( I 1 Iα T 1, I 1 α 1 K α1, I 1 I T α 2, I 1 α 2 K α2, + I 1 K ). More generally, we can consider the conditional copula regression where all marginal distributions and copula are conditioned on the k-variate covariate X parametrically. Patton (2002) extends the existing theories of copula to the conditional copula setting, including the conditional version of Sklar s theorem, which gives conditional copula density f(y 1,, y d, η x) = c (F 1 (y 1, α 1 x), F 2 (y 2, α 2 x), x) f 1 (y 1, α 1 x) f 2 (y 2, α 2 x). For simplicity, we consider the case where the copula parameter is modeled by the linear calibration function with = Xβ where β = (β 0,, β k ) T is a k + 1 dimensional parameters. We can then consider β as the copula parameter instead of, which results in η = (α 1,1, α 1,2, α 2,1, α 2,2, β 0,, β k ) T. One can easily define matrices K η, I η, L and Kη accordingly. For details about conditional copula regression, see Patton (2006), Acar et al. (2013) and Palaro & Hotta (2006). 3 Simulation study To study the behavior of CIC, we have performed a simulation study. 3.1 Simulation 1 In simulation 1, we generated datasets of 3 sizes (n = 100, 1000, 10000) with the data generating model described in Table 2. We then, for each dataset, fitted 18 different copula models, based on the possible combinations of the candidate copulas and margins described in Table 3. We repeated this process 1000 times and the averaged results from the fitted models can be found in Table 7, Table 8 and Table 9 in the Appendix. Table 2: Description of the data generating model used in simulation 1. Data generating model Copula Margin 1 Margin 2 Weibull Gamma Gumbel α 1 = (1.5, 4) T α 2 = (2, 1) T = 3 (shape, scale) (shape, rate) 11

12 Table 3: List of the candidate copulae and margins used in simulation 1 Candidates Copula Margin 1 Margin 2 Gumbel, Gaussian Weibull, Gamma, Log-normal Weibull, Gamma, Log-normal From all three tables, we can see that CIC and AIC 2ML result in similar model ranks. This is expected since both are aiming for the model that minimizes KL divergence. When the model is correctly specified (model 1), the estimated value of CIC and AIC 2ML are essentially the same. This confirms that CIC and AIC 2ML are equal under the true model assumption (analytically proven in Section 2.3). Further, we can observe that better models according to CIC and AIC 2ML have smaller value of MSE( P) in general. To clarify, MSE( P) indicates mean squared error of two-stage ML estimated P(b 0.8 < y). Here P(b 0.8 < y) is the joint probability that each marginal variable has larger value than its 0.8-quantile value, defined by each marginal model. Similarly, MSE( P) is mean squared error of ML estimated P(b 0.8 < y). When the model is correctly specified (model 1), p η is virtually equal to p η, the length of the parameter vector η. When the model has misspecification, the CIC value is penalized more as p η increases. However, this is not the case for AIC 2ML since p η = 5 across all models. Thus, CIC has higher chance of choosing a less wrong model. As n increases, the influence of p η on CIC decreases since the absolute value of loglikelihood grows much faster than the penalty term. This is observable in Table 4. When n = 100, the best models chosen by the model robust model selection criteria (TIC and CIC) result in smaller MSE values. However, when n = 10000, the penalty term of these model selection criteria is very small compared to the log-likelihood value. Consequently, CIC and TIC choose the same models as their model non-robust variants (AIC 2ML and AIC ML ) do. This results in the same MSE performance. Table 4: Result from simulation 1. For each dataset, the best model was chosen among 18 candidate models by using CIC, TIC, AIC 2ML or AIC ML. Then, P(b 0.8 < y) was computed from the best models. The table contains mean squared error of the estimated P(b 0.8 < y) multiplied by n TIC AIC ML CIC AIC 2ML Simulation 2 In simulation 2, we generated datasets of size n = 1000 with the data generating model described in Table 5. We then fitted 486 different copula models, which are based on the possible combinations of the candidate copulae and margins described in Table 6. We repeated this process 100 times and averaged the results. Like in simulation 1, P(b 0.8 < y) was computed from every fitted models. Figure 1 displays the relationship between mean squared error of estimated P(b 0.8 < y) and CIC and AIC 2ML. We can see that both model selection criteria evaluate the models that have lower mean squared error as better models. The difference 12

13 between CIC and AIC 2ML in this perspective is minimal. This is because the log-likelihood, the element that is shared by both model selection criteria, has much bigger absolute value than the bias correction term. Table 5: Description of the data generating model used in simulation 2. Data generating model Copula Margin 1 Margin 2 Margin 3 Margin 4 Margin 5 Weibull Weibull Gamma Gamma Gumbel α 1 = (1.5, 4) T α 2 = (2, 3) T α 3 = (2, 1) T α 4 = (3, 1) T = 3 (shape, scale) (shape, scale) (shape, rate) (shape, rate) Gamma α 5 = (4, 2) T (shape, rate) Table 6: List of the candidate copulae and margins used in simulation 2 Candidates Copula Margin 1 Margin 2 Margin 3 Margin 4 Margin 5 Gumbel, Gaussian Weibull, Gamma, Log-normal Weibull, Gamma, Normal Weibull, Gamma, Log-normal Weibull, Gamma, Log-normal Weibull, Gamma, Log-normal Figure 2 plots the same as Figure 1, but the x-axis is now the bias correction term ( p η for CIC and p η for AIC 2ML ). The difference between CIC and AIC 2ML is now more clear. While p η (dimension of the parameter vector η) for AIC 2ML is fixed at 11 across all models, p η tend to penalize misspecified models more and forms a strong relationship with the mean squared error of estimated P(b 0.8 < y). 13

14 Gumbel copula Gaussian copula 10 5 MSE(P) CIC AIC 2ML CIC (trend) AIC 2ML (trend) 10 5 MSE(P) CIC AIC 2ML CIC (trend) AIC 2ML (trend) Model selection criterion value Model selection criterion value Figure 1: Result from simulation 2. On the x-axis is the value of CIC or AIC 2ML. The 486 different copula models, defined by using the candidate copulae and margins described in Table 6, are fitted to the dataset generated from the data generating model described in Table 5. The y-axis is the mean squared error of P(b 0.8 < y), which indicates the two-stage ML estimated joint probability that each marginal variable has larger value than its 0.8-quantile value, defined by each marginal model. Since different choices of copulae leads to a big difference in model selection score, the result is separately displayed for each copula. The left plot contains 283 models with the Gumbel copula and the right plot contains 283 models with the Gaussian copula. The data generating model had Gumbel copula. As mentioned in Section 2.4, one can compare CIC to TIC, or AIC 2ML to AIC ML, to measure how much one loses in terms of KL divergence by performing two-stage ML estimation instead of ML estimation. Figure 3 shows that TIC CIC or AIC 2ML AIC ML has a relationship with the loss of MSE of P(b 0.8 < y) caused by two-stage ML estimation. However, this relationship seems weaker when the copula is misspecified (right panel). We tried to identify any sub-pattern in the plot that can cause this, for example by plotting only a subset of models that have specific margins. Yet, we weren t able to detect any sub-pattern. 14

15 Gumbel copula Gaussian copula 10 5 MSE(P) CIC AIC 2ML CIC (trend) AIC 2ML (trend) 10 5 MSE(P) CIC AIC 2ML CIC (trend) AIC 2ML (trend) Bias correction term (p* η or p η ) Bias correction term (p* η or p η ) Figure 2: Result from simulation 2. The 486 different copula models, defined by using the candidate copulae and margins described in Table 6, are fitted to the dataset generated from data generating model described in Table 5. The y-axis is the mean squared error of P(b 0.8 < y). The x-axis is the value of the bias correction terms in model selection criteria ( p η for CIC and p η for AIC 2ML ). Like for Figure 1, the result is separately displayed for each copula. The data generating model had Gumbel copula. 15

16 Gumbel copula Gaussian copula 10 5 {MSE(P) MSE(P)} TIC CIC AIC ML AIC 2ML TIC CIC (trend) AIC ML AIC 2ML (trend) 10 5 {MSE(P) MSE(P)} TIC CIC AIC ML AIC 2ML TIC CIC (trend) AIC ML AIC 2ML (trend) TIC CIC or AIC ML AIC 2ML TIC CIC or AIC ML AIC 2ML Figure 3: Result from simulation 2. The 486 different copula models, defined by using the candidate copulae and margins described in Table 6, are fitted to the dataset generated from data generating model described in Table 5. The y-axis is the difference between MSE( P) (MSE of P(b 0.8 < y) estimated by ML) and MSE( P) (MSE of P(b 0.8 < y) estimated by two-stage ML). Like in Figure 1 and Figure 2, the result is displayed separately for each copula. The data generating model had Gumbel copula. 4 Conclusions and further research In this paper, we have developed the copula information criterion (CIC), which is a TIC-like model robust model selection criterion for two-stage ML estimated copulae. When we make an assumption that the parametric candidate model contains the true model, CIC becomes equal to AIC 2ML. This validates the use of AIC 2ML for the two-stage ML estimated copula models. To our knowledge, this is the first time that AIC 2ML formula is analytically justified. Further, since both TIC and CIC are estimating the same part of the KL divergence, without the presence of the true model assumption, they are compatible to each other and can be used to check possible disadvantages caused by the two-stage ML estimation. The same can be done by comparing AIC ML and AIC 2ML, when one believes in the model. Regarding the assumption that a candidate model is correct, one can compare p η (bias correction term of CIC) and p η (bias correction term of AIC 2ML ) to check whether the model severely diverges from the data generating model, i.e. as a separate goodness-of-fit test. It may be noted that the job of the CIC is to rank models according to a sensible criterion, and to identify the best ones, but doing well in this ranking is not the same as claiming that the model passes goodness-of-fit tests. In yet other words, the winning model, using the CIC, may still not be a perfect model, perhaps since the list of candidate models has not been the best. We performed a simulation study. For relatively small sample sizes, CIC outperforms AIC 2ML in terms 16

17 of prediction performance from the selected models. When the sample size is large, the log-likelihood term grows much faster than the bias correction term and the difference between CIC and AIC 2ML is minimal. Naturally, for large n, the best models are those with high values of the maximized (two-stage) log-likelihoods, which means richly parametrised models. From our simulation, cf. Figure 2, we can see that p η alone has a strong correlation with the prediction performance (measured in MSE). So, one can consider to use p η (without the log-likelihood part) to judge the model. In addition, TIC CIC and AIC ML AIC 2ML turn out to have high correlation with the loss of prediction performance (measured as difference in MSE) caused by the switch from ML estimation to two-stage ML estimation. The results from the simulation study hold mostly both when the copula is correctly specified and misspecified. Yet, in Figure 3, although the overall tendency is similar, we observe that the result from the misspecified Gaussian copula seems to consist of two different sub-patterns. However, we were not able to find a possible cause of this. Because of the large number of possible models in high dimensional setting, the number of situations that we could examine, was limited. (In case of a 5-dimensional copula model with 2 candidate copulae and 3 candidate for each margin, there are 486 models that we have to test, and each model has to be fitted by 2 different estimation schemes.) Another problem was that we could not try all copulae and margins on the simulated data since fitting a heavily misspecified copula and margins would cause numerical problems. A further large-scale simulation study that examines the behavior of different types of copulae in variety of situations would be fruitful. Furthermore, CIC is computationally expensive mainly because K α, K α, and K require score functions for every data point separately. For example, for Ĩ 1 α and Ĩ, one can avoid this by swapping the order of summation and differentiation, but for K α, Kα, and K, this is not possible. A numerical technique that can make CIC less computationally extensive would be appreciated. Although CIC performs decently well in selecting a good model that fits the data best in terms of KL divergence, there are situations where one is interested in a model that is suitable for specific tasks. The task of interest could be for example estimating tail probabilities, the mean, or the median. A model selection criterion for copula models under the two-stage ML scheme that can take this into account would be useful. Acknowledgments The authors would like to thank Ingrid Hobæk Haff and Steffen Grønneberg for their valuable comments and fruitful discussions. The authors also acknowledge partial funding from the Norwegian Research Council supported research group FocuStat: Focus Driven Statistical Inference With Complex Data, and from the Department of Mathematics at the University of Oslo. References Acar, E. F., Craiu, R. V., Yao, F. et al. (2013). Statistical testing of covariate effects in conditional copula models. Electronic Journal of Statistics 7,

18 Akaike, H. (1974). A new look at the statistical model identification. IEEE transactions on automatic control 19, Akaike, H. (1998). Information theory and an extension of the maximum likelihood principle. In Selected papers of hirotugu akaike. Springer, pp Claeskens, G. & Hjort, N. L. (2008). Model Selection and Model Averaging, vol Cambridge University Press Cambridge. Genest, C. & Favre, A.-C. (2007). Everything you always wanted to know about copula modeling but were afraid to ask. Journal of hydrologic engineering 12, Grønneberg, S. & Hjort, N. L. (2014). The copula information criteria. Scandinavian Journal of Statistics 41, Joe, H. (1997). Multivariate Models and Multivariate Dependence Concepts. CRC Press. Jullum, M. & Hjort, N. L. (2017). Parametric or nonparametric: the fic approach. Statistica Sinica. Ko, V. & Hjort, N. L. (2018). Model robust inference for copulae via two-stage maximum likelihood estimation. Submitted to Journal of Multivariate Analysis. Kullback, S. & Leibler, R. A. (1951). On information and sufficiency. The Annals of Mathematical Statistics 22, Nelsen, R. B. (2006). An Introduction to Copulas. Springer Science & Business Media. Palaro, H. & Hotta, L. (2006). Using conditional copula to estimate value at risk. Journal of Data Science 4, Patton, A. J. (2002). Applications of copula theory in financial econometrics. Ph.D. thesis, University of California, San Diego. Patton, A. J. (2006). Modelling asymmetric exchange rate dependence. International economic review 47, Sklar, A. (1959). Fonctions de répartition à n dimensions et leurs marges. Publ. Inst. Statist. Univ. Paris 8, Takeuchi, K. (1976). The distribution of information statistics and the criterion of goodness of fit of models. Mathematical Science 153,

19 Appendix 19 n = 100 Model no. TIC AIC ML CIC AIC 2ML p η,tic p η p η 10 5 MSE( P) 10 5 MSE( P) Copula Margin 1 Margin Gumbel Weibull Gamma Gumbel Weibull Weibull Gumbel Gamma Gamma Gaussian Weibull Weibull Gaussian Weibull Gamma Gumbel Gamma Weibull Gumbel Gamma Log-normal Gaussian Gamma Gamma Gaussian Gamma Weibull Gumbel Weibull Log-normal Gumbel Log-normal Log-normal Gaussian Weibull Log-normal Gaussian Gamma Log-normal Gumbel Log-normal Gamma Gumbel Log-normal Weibull Gaussian Log-normal Gamma Gaussian Log-normal Weibull Gaussian Log-normal Log-normal Table 7: Result of simulation 1 with n = 100. The simulation was repeated 1000 times and the results were averaged.

20 20 n = 1000 Model no. TIC AIC ML CIC AIC 2ML p η,tic p η p η 10 5 MSE( P) 10 5 MSE( P) Copula Margin 1 Margin Gumbel Weibull Gamma Gumbel Weibull Weibull Gumbel Gamma Gamma Gaussian Weibull Weibull Gaussian Weibull Gamma Gumbel Gamma Weibull Gumbel Gamma Log-normal Gaussian Gamma Gamma Gaussian Gamma Weibull Gumbel Weibull Log-normal Gumbel Log-normal Log-normal Gaussian Weibull Log-normal Gaussian Gamma Log-normal Gumbel Log-normal Gamma Gumbel Log-normal Weibull Gaussian Log-normal Gamma Gaussian Log-normal Weibull Gaussian Log-normal Log-normal Table 8: Result of simulation 1 with n = The simulation was repeated 1000 times and the results were averaged.

21 21 n = Model no. TIC AIC ML CIC AIC 2ML p η,tic p η p η 10 5 MSE( P) 10 5 MSE( P) Copula Margin 1 Margin Gumbel Weibull Gamma Gumbel Weibull Weibull Gumbel Gamma Gamma Gaussian Weibull Weibull Gaussian Weibull Gamma Gumbel Gamma Weibull Gumbel Gamma Log-normal Gaussian Gamma Gamma Gaussian Gamma Weibull Gumbel Weibull Log-normal Gumbel Log-normal Log-normal Gaussian Weibull Log-normal Gaussian Gamma Log-normal Gumbel Log-normal Gamma Gumbel Log-normal Weibull Gaussian Log-normal Gamma Gaussian Log-normal Weibull Gaussian Log-normal Log-normal Table 9: Result of simulation 1 with n = The simulation was repeated 1000 times and the results were averaged.