Testing Out-of-Sample Portfolio Performance

Transcription

1 Testing Out-of-Sample Portfolio Performance Ekaterina Kazak 1 Winfried Pohlmeier 2 1 University of Konstanz, GSDS 2 University of Konstanz, CoFE, RCEA Econometric Research in Finance Workshop 2017 SGH Warsaw School of Economics September 15, 2017

2 Introduction Example: Mean Variance Portfolio Consider an investor who chooses a portfolio among N financial assets r t R N, E [r t] = µ, V [r t] = Σ Efficient (norm constrained) portfolio: max ω CE(ω) = max ω {µ ω γ 2 ω Σω} such that ι ω = 1 ω(m) = number parameters O(N 2 ). Σ 1 ι ι Σ 1 ι + 1 γ (Σ 1 Σ 1 ιι Σ 1 ι Σ 1 ι ) µ Kazak & Pohlmeier (UKon) September 15, / 29

3 Introduction Empirical Portfolio Models: Stylized Facts Stylized facts of empirical portfolio weights: high standard error highly instable across time bad predictive quality some with pathological distribution with no finite first and second moments The naive equally weighted portfolio is often a very strong competitor in within-sample and out-of-sample competitions Kazak & Pohlmeier (UKon) September 15, / 29

4 Introduction Illustration: Estimation Noise 0.15 ˆω1 0.1 ˆω2 0.1 ω1 ω ˆω3 ω3 0.1 ˆω4 ω Figure: Frequency distribution of estimated portfolio weights vs the true theoretical value based on the Monte-Carlo study, r iid N (µ, Σ), number of assets N = 5, estimation window length T = 120 (i.e. 10 years of monthly data). (µ, Σ) set as in the Kenneth R. French data. Kazak & Pohlmeier (UKon) September 15, / 29

5 Introduction Empirical Portfolio Models: Performance Testing Evaluation of Portfolio allocation strategies: Sharpe Ratio or Certainty Equivalent Testing the difference in performance measures by a z-test Related literature: Under iid-normality for Sharpe Ratio: Jobson and Korkie (1981), Memmel (2003) Bootstrap test: Ledoit and Wolf (2008), Ledoit and Wolf (2011) Delta method for Certainty Equivalent: DeMiguel et al. (2009) Test the difference in performance measures of two arbitrary asset returns Portfolio performance testing quality is under-researched Kazak & Pohlmeier (UKon) September 15, / 29

6 Introduction Goals of the Paper Investigation of the stochastic nature of out-of-sample portfolio returns underlying the performance tests Analysis of the size and power properties of portfolio performance tests Give guidance on how to deal with the low power providing an lternative way of using the information of performance test within an algorithmic pre-testig strategy Kazak & Pohlmeier (UKon) September 15, / 29

7 Out-of-Sample Returns and Performance Measures Introduction Out-of-Sample Returns and Performance Measures Monte Carlo Design Test properties Power Optimal Pretest Portfolios Summary Kazak & Pohlmeier (UKon) September 15, / 29

8 Out-of-Sample Returns and Performance Measures Certainty Equivalent Consider a portfolio of N financial assets r t R N with E [r t] = µ and V [r t] = Σ Let ω(s) denote an N 1 vector of portfolio weights of a strategy s Portfolio return r p t (s) = ω(s) r t Portfolio mean µ p(s) = E [r p t (s)] = ω(s) µ Portfolio variance V [r p t (s)] = ω(s) Σω(s) Certainty Equivalent CE(ω(s)) = ω(s) µ γ 2 ω(s) Σ ω(s) CE difference 0(s, s) = CE(ω(s)) CE(ω( s)) Kazak & Pohlmeier (UKon) September 15, / 29

9 Out-of-Sample Returns and Performance Measures Certainty Equivalent Out-of-sample portfolio return of strategy s: ˆr p t+1 (s) = ˆω t+1 t(s) r t+1 = ˆω t(s) r t+1 µ op(s) = E [ˆr p t+1 (s)] = E [ˆω t(s)] µ σop(s) 2 = V [ˆr p t+1 (s)] = E [ˆω t(s) Σ ˆω t(s)] + µ V [ˆω t(s)] µ Out-of-sample CE: CE op(ˆω t(s)) = µ op(s) γ 2 σ2 op(s) = CE(ω(s)) γ 2 tr( Σ V [ˆω(s)] ) γ 2 µ V [ˆω t(s)] µ Out-of sample difference: op(s, s) 0(s, s) γ 2 [ ] [ ] tr(σ V [ˆω t(s)]) tr(σ V [ˆω t( s)]) γ2 µ V [ˆω t(s)] V [ˆω t( s)] µ Kazak & Pohlmeier (UKon) September 15, / 29

10 Out-of-Sample Returns and Performance Measures CE difference GMVP vs 1/N (s, s) op(s, s) γ = 0.1 γ = 1 γ = γ = 0.1 γ = 1 γ = N γ 2 tr[σv[ŵ]] N γ 2 µt V[ŵ]µ -1e γ = 0.1 γ = 1 γ = N -2e-05-3e-05 γ = 0.1 γ = 1 γ = N Average CE differences over random N out of 100 asset combinations for GMVP and equally weighted portfolio by dimension of the asset universe N for different values of the risk aversion parameter γ. The estimation window length T is set to 120 (10 years of monthly observations). Upper-left plot: difference in theoretical CE, 0 (s, s). Lower-left plot: estimation noise penalty. Lower-right plot: out-of-sample risk penalty. Upper-right plot: overall out-of-sample CE difference. Kazak & Pohlmeier (UKon) September 15, / 29

11 Out-of-Sample Returns and Performance Measures Result 1: the null hypothesis Applied research: comparison is based on ĈEop(ˆωt(s)) H 0 : ] op(s, s) = E [ĈEop(ˆω t(s)) ĈEop(ˆωt( s)) = 0 ] E [ĈEop(ˆω t(s)) = CE(ω(s)) γ tr [Σ V [ˆω(s)]] γ 2 2 µ V [ˆω t(s)] µ Takes into account estimation risk γ tr [Σ V [ˆω(s)]] 2 Takes into account forecasting risk γ 2 µ V [ˆω t(s)] µ Evaluate test properties by a Monte Carlo Study Kazak & Pohlmeier (UKon) September 15, / 29

12 Monte Carlo Design Introduction Out-of-Sample Returns and Performance Measures Monte Carlo Design Test properties Power Optimal Pretest Portfolios Summary Kazak & Pohlmeier (UKon) September 15, / 29

13 Monte Carlo Design Distribution of the out-of-sample returns In the following the out-of-sample portfolio returns r p t+1 (s) are based on the estimated weights and r p t+1 ( s) are based on the non-stochastic weights Assume r t iid N (µ, Σ) Okhrin and Schmid (2006): estimated Global Minimum Portfolio (GMVP) weights follow a multivariate elliptical t-distribution Mixture design: f (ˆr p t+1(s), r p t+1( s) ) = f (ˆω t(s) r t+1, ω t( s) r t+1 ˆω t(s) ) g (ˆω t(s) ) ˆrp t+1 (s) r p t+1 ( s) ˆω t (s) N ˆωt(s) µ, ˆωt(s) Σˆω t(s) ω t( s) µ ω t( s)σˆω t(s) ˆω t(s) Σ ω t( s) ω t( s) Σ ω t( s) Kazak & Pohlmeier (UKon) September 15, / 29

14 Monte Carlo Design Distribution of the out-of-sample returns Marginal density of ˆr p (s) t5 N mix. design data Marginal density of ˆr p ( s) t5 N mix. design data Marginal distribution of the out-of-sample portfolio returns for the GMVP (ˆr p (s)) and equally weighted portfolio (ˆr p ( s)): based on the real data (in red), simulated from bivariate t 5, simulated from bivariate normal and simulated from the proposed mixture design (in blue). The mean and standard deviation of the simulated returns are adjusted to be the same as of the empirical portfolio returns. Kazak & Pohlmeier (UKon) September 15, / 29

15 Test properties Introduction Out-of-Sample Returns and Performance Measures Monte Carlo Design Test properties Power Optimal Pretest Portfolios Summary Kazak & Pohlmeier (UKon) September 15, / 29

16 Test properties Test Properties The following comparison is based on: Type of the test Delta Method Bootstrap (percentile and t-statistic) Out-of-sample evaluation horizon (H) Risk aversion (γ) Estimation noise N/T, with N = 30 and T denoting the size of in-sample window Benchmark strategy (correlatedness of the out-of-sample returns) Kazak & Pohlmeier (UKon) September 15, / 29

17 Test properties Size (ˆα) Table: Empirical rejection probabilities under H 0 for GMVP vs 1/N. α = 5%. N/T = 0.01 Two-sided One-sided Delta Bootstrap Bootstrap Delta Bootstrap Bootstrap method Percentile t-statistic method Percentile t-statistic γ = H = 100 γ = γ = γ = H = 500 γ = γ = γ = H =1000 γ = γ = Figures in the table correspond to the share of Monte Carlo draws where the null hypothesis was rejected (out of draws). H denotes the out-of-sample evaluation window length and γ denotes risk aversion coefficient. Kazak & Pohlmeier (UKon) September 15, / 29

18 Test properties Power (1 ˆβ) Table: Power at 1% expected CE difference for GMVP vs 1/N. α = 5%. N/T = 0.01 Two-sided One-sided Delta Bootstrap Bootstrap Delta Bootstrap Bootstrap method Percentile t-statistic method Percentile t-statistic γ = H = 100 γ = γ = γ = H = 500 γ = γ = γ = H =1000 γ = γ = Figures in the table correspond to the share of Monte Carlo draws where the null hypothesis was rejected (out of draws). H denotes the out-of-sample evaluation window length and γ denotes risk aversion coefficient. Kazak & Pohlmeier (UKon) September 15, / 29

19 Test properties ROC curves: Delta Method 1 Estimation Noise 1 Side of the test 1 ˆβ ˆβ 0.5 N/T = 0.01 J =0.297 N/T = 0.1 J = ˆα Out-of-sample horizon 1 One-sided J =0.297 Two-sided J = ˆα Benchmark 1 1 ˆβ ˆβ H = 1000 J =0.297 H = 500 J = ˆα 0 GMVP vs 1/N J =0.297 Ridge vs 1/N J = ˆα ROC curves for a two-sided Delta Method. Risk aversion is set to γ = 1 for the asset space N = 30. Left panel: GMVP vs 1/N for different estimation noise N/T ratios. Right panel: GMVP combined with the ridge covariance matrix estimator vs 1/N with N/T = Kazak & Pohlmeier (UKon) September 15, / 29

20 Test properties Result 2: properties of tests The tests are heavily influenced by: out-of-sample horizon length estimation noise correlation degree among the out-of-sample returns of the two strategies The power of the tests is very low Kazak & Pohlmeier (UKon) September 15, / 29

21 Test properties Result 2: properties of tests The tests are heavily influenced by: out-of-sample horizon length estimation noise correlation degree among the out-of-sample returns of the two strategies The power of the tests is very low However One-sided tests have better testing properties Tests can be used in pretesting Kazak & Pohlmeier (UKon) September 15, / 29

22 Power Optimal Pretest Portfolios Introduction Out-of-Sample Returns and Performance Measures Monte Carlo Design Test properties Power Optimal Pretest Portfolios Summary Kazak & Pohlmeier (UKon) September 15, / 29

23 Power Optimal Pretest Portfolios Pretest estimation Deciding between two portfolio strategies in the presence of low power: The pretest estimator depends either on strategy s in case H 0 is rejected or on s otherwise: H 0 : op(s, s) 0 and H 1 : op(s, s) > 0 ω t+h (s, s) = 1l ( ˆ op(s, s) > (α) )( ω t+h (s) ω t+h ( s) ) + ω t+h ( s), h = 1,..., H [ ] [ E ĈEop(s, s) op(s, s) > 0 = π(α) E ˆ op(s, s) ˆ ] op(s, s) > (α) ] + E [ĈEop( s) [ ] [ E ĈEop(s, s) op(s, s) 0 = α E ˆ op(s, s) ˆ ] op(s, s) > (α) ] + E [ĈEop( s) How to choose an optimal α? Kazak & Pohlmeier (UKon) September 15, / 29

24 Power Optimal Pretest Portfolios Pretesting strategy: infeasible CE ˆ α CE(s) ˆ CE( s) ˆ CE(s, ˆ s) α = 5% Pretesting strategy: GMVP vs 1/N, γ = 1. Kenneth R.French data on 5 industry portfolios with estimation window length of T = 60 (5 years of monthly observations), corresponding to N/T = 0.08 ratio. Dashed line corresponds to the conventional α = 5%. Kazak & Pohlmeier (UKon) September 15, / 29

25 Power Optimal Pretest Portfolios Pretesting strategy Feasible solution: the in-sample CE optimizing significance level α t is chosen for the test, determining the strategy for the next period t + 1: αt+1 = arg max α CE in(α, s, s t T,..., t). Shrinking α t+1 towards a target α 0, e.g. to the conventional 5% level: α s t+1 = (1 λ)α t+1 + λα 0, Adaptive smoothing the series according to α m t+1 = (1 λ)α t+1 + λα m t, Kazak & Pohlmeier (UKon) September 15, / 29

26 Power Optimal Pretest Portfolios Pretesting strategy: out-of-sample CE T=60 T=120 γ=0.1 γ=1 γ=3 γ=0.1 γ=1 γ=3 Strategy N=5 GMVP /N In-sample Shrinking Smoothing N=30 GMVP /N In-sample Shrinking Smoothing The numbers in the table correspond to the annualized average out-of-sample CE over 1000 randomly formed portfolios of the specified size. T denotes the estimation window length, γ denotes risk aversion coefficient and N is the number of assets. The numbers in bold correspond to the largest CE obtained for a given γ, N, T combination. The evaluation window length H = 600. The tuning parameter λ for both shrinking and smoothing the α series is set to be 0.5. Kazak & Pohlmeier (UKon) September 15, / 29

27 Power Optimal Pretest Portfolios Refinements of the pretesting strategy adaptive smoothing: optimizing over the smoothing papameter bagging the indicator pretesting of ˆω t(g) Kazak & Pohlmeier (UKon) September 15, / 29

28 Summary Concluding remarks Why is the 1/N strategy performing so well? no estimation risk low power of performance test Tests (incl. various implementations) have very similar properties One can improve on testing properties by choosing another benchmark and longer evaluation horizon One-sided tests do better in terms of power Choosing a lower α reduces the probability of random selection Kazak & Pohlmeier (UKon) September 15, / 29

29 Summary Thank you! Kazak & Pohlmeier (UKon) September 15, / 29

30 Summary References DeMiguel, V., L. Garlappi, and R. Uppal (2009): Optimal Versus Naive Diversification: How Inefficient is the 1/N Portfolio Strategy? Review of Financial Studies, 22, Jobson, J. D. and B. M. Korkie (1981): Performance Hypothesis Testing with the Sharpe and Treynor Measures, The Journal of Finance, 36, Ledoit, O. and M. Wolf (2008): Robust performance hypothesis testing with the Sharpe ratio, Journal of Empirical Finance, 15, (2011): Robust Performances Hypothesis Testing With the Variance, Wilmott, 2011, Markowitz, H. (1952): Portfolio Selection, The Journal of Finance, 7, Memmel, C. (2003): Performance Hypothesis Testing with the Sharpe Ratio, Finance Letters, 1, Okhrin, Y. and W. Schmid (2006): Distributional properties of portfolio weights, Journal of Econometrics, 134, Kazak & Pohlmeier (UKon) September 15, / 29