A Generalized Approach to Portfolio Optimization: Improving Performance By Constraining Portfolio Norms

Transcription

1 A Generalized Approach to Portfolio Optimization: Improving Performance By Constraining Portfolio Norms Victor DeMiguel Lorenzo Garlappi Francisco J. Nogales Raman Uppal July 16, 2007 Abstract In this paper, we provide a general framework for identifying portfolios that perform well out-of-sample even in the presence of estimation error. This general framework relies on solving the traditional minimum-variance problem (based on the sample covariance matrix) but subject to the additional constraint that the norm of the portfolio-weight vector be smaller than a given threshold. We show that our unifying framework nests as special cases the shrinkage approaches of Jagannathan and Ma (2003) and Ledoit and Wolf (2004b), and the 1/N portfolio studied in DeMiguel, Garlappi, and Uppal (2007). We also use our general framework to propose several new portfolio strategies. For these new portfolios, we provide a moment-shrinkage interpretation and a Bayesian interpretation where the investor has a prior belief on portfolio weights rather than on moments of asset returns. Finally, we compare empirically (in terms of portfolio variance, Sharpe ratio, and turnover), the out-of-sample performance of the new portfolios we propose to nine strategies in the existing literature across five datasets. We find that the norm-constrained portfolios we propose have a lower variance and a higher Sharpe ratio than the portfolio strategies in Jagannathan and Ma (2003) and Ledoit and Wolf (2004b), the 1/N portfolio, and also other strategies in the literature such as factor portfolios and the parametric portfolios in Brandt, Santa-Clara, and Valkanov (2005). We wish to thank Vito Gala, Francisco Gomes, Garud Iyengar, Igor Makarov, Catalina Stefanescu, and Bruce Weber for comments and suggestions. DeMiguel and Uppal are from London Business School; Garlappi is from The University of Texas at Austin; Nogales is from the Universidad Carlos III de Madrid. 1

2 Contents 1 Introduction 3 2 Existing Approaches: Shrinking the Sample Covariance Matrix Base Case: The Shortsale-Unconstrained Minimum-Variance Portfolio The Shortsale-Constrained Minimum-Variance Portfolio Shrinking the Sample-Covariance Matrix Toward the Identity Matrix A Generalized Approach: Constraining the Portfolio Norms The General Norm-Constrained Minimum-Variance Portfolio First Particular Case: The 1-Norm-Constrained Portfolios Second Particular Case: The 2-Norm-Constrained Portfolios Third Particular Case: The Partial Minimum-Variance Portfolios Analytical Expression for the First Partial Minimum-Variance Portfolio Relation to the 2-Norm-Constrained Portfolios A Bayesian Interpretation of the Norm-Constrained Portfolios A Moment-Shrinkage Interpretation of the Norm-Constrained Portfolios Out-of-Sample Evaluation of the Proposed Portfolios 26 5 Conclusion 29 References 31 A Appendix: Details of the Partial Minimum-Variance Portfolios 34 A.1 Expressing the Minimum-Variance Problem Without the Adding-Up Constraint A.2 Applying the Conjugate-Gradient Method B Appendix: Proofs for All the Propositions 38 C Appendix: Details of Out-of-Sample Evaluation of Portfolios 45 C.1 Calibration of the Norm-Constrained Portfolios C.2 Description of the Portfolios Considered C.3 Description of the Empirical Datasets Considered C.4 Description of the Methodology Used to Evaluate Performance C.5 Discussion of the Out-of-Sample Performance Tables 56 Figures 62

3 1 Introduction Markowitz (1952) showed that an investor who cares only about the mean and variance of static portfolio returns should hold a portfolio on the efficient frontier. To implement these portfolios in practice, one needs to estimate the means and covariances of asset returns. Traditionally, the sample means and covariances have been used for this purpose. But due to estimation error, the portfolios that rely on the sample estimates typically perform poorly out of sample. 1 In this paper, we provide a general framework for determining portfolios with superior out-of-sample performance even in the presence of estimation error. This general framework relies on solving the traditional minimum-variance problem (based on the sample covariance matrix) but subject to the additional constraint that the norm of the portfolio-weight vector be smaller than a given threshold. It is well known that it is more difficult to estimate means than covariances of asset returns (see Merton (1980)), and also that errors in estimates of means have a larger impact on portfolio weights than errors in estimates of covariances. For this reason, recent research has focused on minimumvariance portfolios, which rely solely on estimates of covariances, and thus, are less vulnerable to estimation error than mean-variance portfolios. Indeed, the superiority of minimum-variance portfolios is demonstrated by extensive empirical evidence that shows that this portfolio usually performs better out of sample than any other mean-variance portfolio even when the Sharpe ratio or other performance measures that depend on both the portfolio mean and variance are used for evaluating performance. For example, Jagannathan and Ma (2003, p ) report: 2 1 For evidence of the poor performance of the Markowitz portfolio based on sample estimates of means and covariances, see Frost and Savarino (1986, 1988), Michaud (1989), Best and Grauer (1991), Chopra and Ziemba (1993), Broadie (1993), and Litterman (2003). DeMiguel, Garlappi, and Uppal (2007) provide a comprehensive review of the performance of various methods to deal with the estimation error in means and covariances. These include the following. (i) The Bayesian approach with uninformed diffuse priors Barry (1974), Klein and Bawa (1976), and Bawa, Brown, and Klein (1979), with priors based on data Jobson, Korkie, and Ratti (1979) and Jorion (1985), with priors based on an asset-pricing model Pástor (2000), Pástor and Stambaugh (2000), Wang (2005) and Chevrier (2006), and with priors based on an asset-pricing model and the subjective views of an investor Black and Litterman (1990, 1992). (ii) Non-Bayesian approaches to estimation error, which include robust portfolio allocation rules Halldórsson and Tütüncü (2000), Goldfarb and Iyengar (2003), Tütüncü and Koenig (2003), and Garlappi, Uppal, and Wang (2006), and portfolio rules designed to optimally diversify across market and estimation risk Kan and Zhou (2005). (iii) Portfolios that exploit the moment restrictions imposed by the factor structure of returns MacKinlay and Pastor (2000). (iv) Parametric portfolios that exploit asset-specific information in the cross section of asset returns Brandt, Santa-Clara, and Valkanov (2005). 2 For additional evidence, see Jorion (1985, 1986, 1991) and DeMiguel, Garlappi, and Uppal (2007). 3

4 The estimation error in the sample mean is so large nothing much is lost in ignoring the mean altogether when no further information about the population mean is available. For example, the global minimum variance portfolio has as large an out-of-sample Sharpe ratio as other efficient portfolio when past historical average returns are used as proxies for expected returns. In view of this we focus our attention on global minimum variance portfolios in this study. Just like Jagannathan and Ma (2003), we too focus on minimum-variance portfolios, even though the general framework we develop applies also to mean-variance portfolios. But even the performance of the minimum-variance portfolio depends crucially on the quality of the estimated covariances. Although the estimation error associated with the sample covariances is smaller than that for sample mean returns, it can still be substantial. In the literature, several approaches have been proposed to deal with the problem of estimating the large number of elements in the covariance matrix. One approach is to use higher-frequency data, say daily instead of monthly returns (see Jagannathan and Ma (2003)). A second approach is to impose some structure on the estimator of the covariance matrix; for instance, Chan, Karceski, and Lakonishok (1999) propose a variety of factor models, which reduce the number of parameters to be estimated, and therefore, mitigate the impact of estimation error. A third approach is suggested by Green and Hollifield (1992). They propose a two-step method if returns are generated by a single factor model. First, diversify over the set of high beta stocks and the set of low beta stocks separately; then, short the high beta portfolio and go long the low beta portfolio in oder to reduce the systematic risk. A fourth approach has been proposed by Ledoit and Wolf (2004b), who use as an estimator a weighted average of the sample covariance matrix and the identity matrix. This approach can be interpreted as a method that shrinks the sample covariance matrix toward the identity matrix. A fifth approach, which is often used in practice, is to impose shortsale constraints on the portfolio weights (see Frost and Savarino (1988) and Chopra (1993)). Jagannathan and Ma (2003) show that imposing a shortsale constraint when minimizing the portfolio variance is equivalent to shrinking the extreme elements of the covariance matrix. This simple remedy for dealing with estimation error performs very well. In fact, Jagannathan and Ma (2003, p. 1654) 4

5 find that the sample covariance matrix [with shortsale constraints] performs almost as well as those [covariance matrices] constructed using factor models, shrinkage estimators or daily returns. Finally, DeMiguel, Garlappi, and Uppal (2007) demonstrate that even constraining shortsales may not mitigate completely the error in estimating the covariance matrix, and thus, an investor may be best off (in terms of Sharpe ratio, certainty-equivalent returns, and turnover) ignoring data on asset returns altogether and using the naive 1/N rule to allocate an equal proportion of wealth across each of the N assets. In this paper, we develop a new approach for determining the optimal portfolio weights in the presence of estimation error. Following the idea in Brandt (1999) and Britten-Jones (1999), we treat the weights rather than the moments of assets returns as the objects of interest to be estimated. So, rather than shrinking the moments of asset returns, we introduce the constraint that the norm of the portfolio-weight vector be smaller than a given threshold. The general framework we develop is then based on solving the traditional minimum-variance problem subject to this constraint on the norm of the portfolio-weight vector; that is, rather than shrinking the moments of asset returns, we require that the weights themselves be bounded. We base our analysis on the minimum-variance portfolio rather than the mean-variance portfolio because, as mentioned above, it is difficult to estimate expected returns precisely, and so portfolios that ignore sample mean returns often outperform portfolios relying on estimated means. 3 Our paper contributes to the literature on optimal portfolio choice in the presence of estimation error in several ways. One, we show that our framework nests as special cases the shrinkage approaches of Jagannathan and Ma (2003) and Ledoit and Wolf (2004b). In particular, we prove that if one solves the minimum-variance problem subject to the constraint that the sum of the absolute values of the weights (1-norm) be smaller than 1, then one retrieves the shortsale-constrained minimum-variance portfolio considered by Jagannathan and Ma (2003). If, on the other hand, one imposes the constraint that the sum of the squares of the portfolio weights (2-norm) be smaller than a given threshold, we prove that then one recovers the class of portfolios considered by Ledoit and 3 The norm constraints can be imposed also on the traditional mean-variance-portfolio problem, and our analysis extends in a straightforward manner to this case. As mentioned above, we focus on the minimum-variance problem because it performs better empirically due to the large error associated with estimating mean returns. For completeness, however, in our empirical work we consider as benchmarks both the classical Markowitz mean-variance portfolio and also the Bayesian mean-variance portfolio, as implemented in Jorion (1985). 5

6 Wolf (2004b). Finally, we show that if one imposes the constraint that the squared 2-norm of the portfolio-weight vector be smaller than 1/N, then one gets the 1/N portfolio studied in DeMiguel, Garlappi, and Uppal (2007). Two, we use this general unifying framework to develop new portfolio strategies. For example, we show how the shortsale-constrained portfolio considered in Jagannathan and Ma (2003) can be generalized. In particular, we show that by imposing the constraint that the 1-norm of the portfolio-weight vector be smaller than a threshold that is strictly larger than 1, then we obtain a new class of shrinkage portfolios where we limit the total amount of shortselling in the portfolio, rather than limiting the shorting asset-by-asset, as in the traditional shortsale-constrained portfolio. To the best of our knowledge, this kind of portfolio has not been analyzed before in the academic literature, although it corresponds closely to the actual portfolio holdings allowed in personal margin accounts. Moreover, these portfolios have recently become quite popular among practitioners see the articles in The Economist (Buttonwood (2007)) and The New York Times (Hershey Jr. (2007)) that describe portfolios where investors are long 130% and short 30% of their wealth. More importantly, we use our general framework to also develop several new portfolio strategies. Specifically, we propose a different class of portfolios that we term partial minimum-variance portfolios. These portfolios are obtained by applying the classical conjugate-gradient method (see Nocedal and Wright (1999)) to solve the minimum-variance problem. We show that these portfolios may be interpreted as a discrete first-order approximation to the shrinkage portfolios proposed by Ledoit and Wolf (2004b). Three, we show how the norm-constrained portfolios we propose and also those proposed by Jagannathan and Ma (2003) and Ledoit and Wolf (2004b) can be interpreted as those of a Bayesian investor who has a certain prior belief on portfolio weights rather than moments of asset returns. Jorion (1986) also shows that the shrinkage estimators of mean returns he considers can be obtained by assuming the investor has a certain prior belief on the means of asset returns. Thus, the main difference between his approach and ours is that we assume a prior belief on the portfolio weights, as opposed to a prior belief on the means of the asset returns. 6

7 Four, our approach to minimum-variance portfolio selection is related also to a number of approaches proposed in the statistics and chemometrics literature to reduce estimation error in regression analysis. It is known in the literature that optimal portfolios weights in an unconstrained mean- or minimum-variance problem can be thought of as coefficients of an OLS regression (see, for example, Britten-Jones (1999)). It then follows that, in general, constrained weights are the outcome of similarly specified restricted regressions. In particular, the case where the 1-norm of the portfolio vector is constrained to be less than a certain threshold, is analogous to the statistical technique for regression analysis known as least absolute shrinkage and selection operator (lasso) (Tibshirani (1996)), the case where the 2-norm of the portfolio vector is constrained to be less than a certain threshold corresponds to the statistical technique known as ridge regression (Hoerl and Kennard (1970)), and the partial minimum-variance portfolio that we study, which is a discrete first-order approximation to the 2-norm-constrained minimum-variance portfolio, corresponds to the technique developed in chemometrics that is known as partial least squares (Wold (1975); Frank and Friedman (1993)). These regression techniques and the distribution theory associated with them have been used extensively in the statistics literature. By allowing a more general constrained structure in the construction of the portfolio weights and linking this to regression techniques, our paper provides a unified framework for understanding the existing methods proposed in the literature, and our analysis suggests new strategies that perform well out of sample. Five, the generalized framework allows one to calibrate the model using historical data in order to improve its out-of-sample performance. To demonstrate this, we show how the nonparametric technique known as cross validation can be used to estimate the optimal amount of shrinkage (the optimal value of the threshold on the portfolio norm) that minimizes the estimated out-of-sample variance; see Efron and Gong (1983) and Campbell, Lo, and MacKinley (1997, Section ) for discussions of cross validation. We also show how the time-series properties of portfolio returns (as opposed to individual security returns) that have been documented by Campbell, Lo, and MacKinley (1997) can be used to set the level of the constraint on the portfolio norm in order to improve the portfolio Sharpe ratio. 7

8 Finally, we compare empirically the out-of-sample performance of the three norm-constrained portfolios we propose to nine strategies in the existing literature for five different datasets. The portfolios we evaluate are listed in Table 1 and the datasets we consider are listed in Table 2. We compare performance along three dimensions: (i) out-of-sample portfolio variance, (ii) out-ofsample portfolio Sharpe ratio, 4 and (iii) portfolio turnover or trading volume. We find that the new portfolios we propose have a lower variance and a higher Sharpe ratio than those studied in Jagannathan and Ma (2003), Ledoit and Wolf (2004b), the 1/N portfolio evaluated in DeMiguel, Garlappi, and Uppal (2007), and also other strategies proposed in the literature, including factor portfolios and the parametric portfolios in Brandt, Santa-Clara, and Valkanov (2005), which use firm-specific characteristics of the cross-section of stock returns. 5 Our work is related closely to Lauprete (2001), who also considers the 1- and 2-norm-constrained portfolios. 6 Our additional contribution is first to demonstrate that the 1- and 2-norm-constrained portfolios include as particular cases the shortsale-constrained minimum-variance portfolios considered by Jagannathan and Ma (2003), the shrinkage portfolios proposed by Ledoit and Wolf (2004b), and the 1/N portfolio. Second, we propose the partial minimum-variance portfolios and show that they are a discrete first-order approximation to the 2-norm-constrained portfolios. Third, we show how our general framework can be used to calibrate the portfolio strategies. Fourth, we provide comprehensive empirical results comparing the 1- and 2-norm-constrained portfolios, as well as the partial minimum-variance portfolios, to other portfolios from the literature. The remainder of this paper is organized as follows. Section 2 reviews the approaches considered in Jagannathan and Ma (2003) and Ledoit and Wolf (2004b), which shrink some or all of the elements of the sample covariance matrix. In Section 3, we propose our general approach, which shrinks the portfolio weights directly, and we show that it nests as special cases the approaches in Jagannathan and Ma (2003), Ledoit and Wolf (2004b), and the 1/N portfolio. We also give Bayesian and moment-shrinking interpretations of the proposed portfolios. In Section 4, we provide a brief 4 We consider the Sharpe ratio as a performance criterion because although none of the portfolios considered uses an estimate of the mean return, as argued above these portfolios tend to outperform those that take also mean returns into account. 5 Because the parametric portfolios in Brandt, Santa-Clara, and Valkanov (2005) rely on firm-specific characteristics they are not really comparable to the other portfolios we evaluate; however, we decided to include them in our empirical analysis because these portfolios achieve very high Sharpe ratios, and hence, are a very important benchmark. 6 See also, Lauprete, Samarov, and Welsch (2002) and Welsch and Zhou (2007). 8

9 discussion of the performance of the different portfolios on empirical data, with a more detailed discussion given in Appendix C. Section 5 concludes. Details of how to compute the partial minimum-variance portfolios are provided in Appendix A. Our main results are highlighted in propositions and proofs for all the propositions are collected in Appendix B. 2 Existing Approaches: Shrinking the Sample Covariance Matrix This section is divided into three parts. First, in Section 2.1, we describe the problem of identifying the minimum-variance portfolio in the absence of shortsale constraints. Then, we review two existing approaches for reducing the impact of the error in estimating the covariance matrix of asset returns. In Section 2.2, we describe the approach analyzed in Jagannathan and Ma (2003), in which shortsale constraints are imposed on the minimum-variance portfolio problem. In Section 2.3, we describe the method developed by Ledoit and Wolf (2004b). Both approaches can be interpreted as methods that shrink some or all of the elements of the sample covariance matrix to reduce the impact of estimation error. In Section 3 that follows, we show that these portfolios can also be interpreted as special cases of the general framework we develop, which shrinks directly the portfolio weights rather than the elements of the covariance matrix. 2.1 Base Case: The Shortsale-Unconstrained Minimum-Variance Portfolio In the absence of shortsale constraints, the minimum-variance portfolio is the solution to the following optimization problem: min w w ˆΣw, (1) s.t. w e = 1, (2) in which w R N is the vector of portfolio weights, ˆΣ R N N is the estimated covariance matrix, w ˆΣw is the variance of the portfolio return, e R N is the vector of ones, and the constraint w e = 1 ensures that the portfolio weights sum up to one. We denote the solution to this shortsaleunconstrained minimum-variance problem by w MINU. 9

10 2.2 The Shortsale-Constrained Minimum-Variance Portfolio Jagannathan and Ma (2003) study the effect of imposing shortsale constraints on the minimumvariance portfolio; that is, they consider the solution to the shortsale-constrained minimum-variance problem, min w w ˆΣw, (3) s.t. w e = 1, (4) w 0. (5) We denote the solution to the shortsale-constrained minimum-variance problem by w MINC. Jagannathan and Ma show that the solution to the shortsale-constrained problem coincides with the solution to the unconstrained problem in (1) (2) if the sample covariance matrix in (1) is replaced by the matrix ˆΣ JM = ˆΣ λe eλ, (6) in which λ R N is the vector of Lagrange multipliers for the shortsale constraint w 0 at the solution to the constrained problem (3) (5). Because λ 0, the matrix ˆΣ JM may be interpreted as the sample covariance matrix after shrinkage, because if the shortsale constraint corresponding to the i th asset is binding (w i = 0), then the sample covariance of this asset with any other asset is reduced by λ i, the magnitude of the Lagrange multiplier associated with its shortsale constraint. Figure 1 depicts the shortsale-constrained minimum-variance portfolio for the case with three risky assets. The three axes in the reference frame give the portfolio weights, w 1, w 2, and w 3, for the three risky assets. Two triangles are depicted in the figure. The larger triangle depicts the intersection of the plane formed by all portfolios whose weights sum up to one with the reference frame. The smaller triangle (colored) represents the set of portfolios whose weights are nonnegative and sum up to one; that is, the set of shortsale-constrained portfolios. The ellipses centered around the minimum-variance portfolio, w MINU, depict the iso-variance curves; that is, the curves formed by portfolios with equal variance. The shortsale-constrained minimum-variance portfolio is at the point where the colored triangle is tangent to the iso-variance curves. The figure also shows the 10

11 location of the 1/N portfolio, which can be interpreted as the portfolio that ignores both the mean returns and the covariances of returns. 2.3 Shrinking the Sample-Covariance Matrix Toward the Identity Matrix To reduce the effect of estimation error, Ledoit and Wolf (2004b) propose replacing the sample covariance matrix with a convex combination of the sample covariance matrix and the identity matrix. Concretely, they propose solving problem (1) (2), in which the matrix ˆΣ is replaced by the following matrix ˆΣ LW = ˆΣ + νi, (7) in which ν R is a positive constant, and I R N N is the identity matrix. Ledoit and Wolf also show how one can estimate the value of ν that minimizes the expected Frobenius norm of the difference between the matrix ˆΣ LW and the true covariance matrix. They show that this method can be interpreted as shrinking the sample covariance matrix toward the identity matrix. 7 Figure 2 depicts the Ledoit-Wolf portfolios for the case with three risky assets. The set of shortsale-constrained portfolios and the iso-variance curves are depicted as in Figure 1. The Ledoit and Wolf portfolios form a curve that joins the 1/N portfolio, which is the Ledoit and Wolf portfolio obtained if ν = in equation (7), with the minimum-variance portfolio equation w MINU, which is the Ledoit and Wolf portfolio obtained if ν = 0 in equation (7). 3 A Generalized Approach: Constraining the Portfolio Norms In this section, we propose a general class of portfolios that results from solving the traditional minimum-variance problem but subject to the additional constraint that the norm of the portfolioweight vector is smaller than a certain threshold δ. These portfolios can be viewed as resulting from shrinking the portfolio weights of the shortsale-unconstrained minimum-variance portfolio instead of shrinking the moments of asset returns. 7 Ledoit and Wolf (2004b) actually propose solving the minimum-variance problem in (1) (2) with the sample covariance matrix ˆΣ LW = 1 ˆΣ + ν I. Note that the portfolio weights that solve this problem are the same as 1+ν 1+ν the ones obtained from using the matrix ˆΣ LW = ˆΣ + νi. We focus on this second matrix because it is easier for our analytical purposes. 11

12 We start by describing the general class of portfolios in Section 3.1. Then, we consider three particular cases, which are motivated by the three methods used in the statistics literature to reduce estimation error in regression analysis: least absolute shrinkage and selection operator (lasso), ridge regression, and partial least squares. First, in Section 3.2, we consider the case where the 1-norm of the portfolio vector is constrained to be less than a certain threshold. Second, in Section 3.3, we consider the case where the 2-norm of the portfolio vector is constrained to be less than a certain threshold. Third, in Section 3.4, we study a portfolio that is a discrete first-order approximation to the 2-norm-constrained minimum-variance portfolio. We call this the partial minimum-variance portfolio. Finally, we provide two different interpretations of the normconstrained portfolios: in Section 3.5 we provide a Bayesian interpretation and in Section 3.6 we give a moment-shrinkage interpretation. 3.1 The General Norm-Constrained Minimum-Variance Portfolio We define the norm-constrained minimum-variance problem as the one that solves the traditional minimum-variance problem subject to the additional constraint that the norm of the portfolioweight vector is smaller than a certain threshold δ: min w w ˆΣw, (8) s.t. w e = 1, (9) w δ, (10) in which w is the norm of the portfolio-weight vector. We consider the 1-norm and the 2-norm, which are defined as: ( N ) 1/p w p = w i p, (11) i=1 for p = 1 and 2, respectively. We denote the solution to the general norm-constrained problem by w NC. 8 8 Observe that we take as the starting point of our analysis the minimum-variance portfolio problem rather than the mean-variance problem, even though we will evaluate performance in terms of the Sharpe ratio, which includes both the mean and variance of portfolio returns. As discussed in the introduction, the reason for starting with the minimum-variance portfolio is that it is difficult to estimate mean returns with much precision, and Jagannathan and Ma (2003) and DeMiguel, Garlappi, and Uppal (2007) find empirically that the out-of-sample performance is better if 12

13 Note that the traditional shortsale-unconstrained minimum-variance portfolio, w MINU, is the solution to the norm-constrained problem with δ =. Consequently, if δ < w MINU, then the norm of the portfolio that solves problem (8) (10) must be strictly smaller than that of the unconstrained minimum-variance portfolio, w MINU. To highlight this result we state it in the following proposition. Proposition 1 For each δ < w MINU, the solution to the norm-constrained problem, w NC, satisfies the following inequality: w NC < w MINU. (12) Proposition 1 states that the norm-constrained minimum variance portfolio, w NC, is a shrinkage estimator of the shortsale-unconstrained minimum-variance portfolio, w MINU. Shrinkage estimators have been a popular method for reducing estimation error ever since their introduction by James and Stein (1961). The idea behind shrinkage estimators is that shrinking an unbiased estimator toward a deterministic target has a negative and a positive effect. 9 The negative effect is that the shrinkage introduces bias into the resulting estimator. The positive effect is that shrinking the estimator toward a deterministic target reduces the variance of the estimator. The challenge is to choose the amount of shrinkage that optimizes the tradeoff between bias and variance. We explain in Section C.1 how this can be done for the norm-constrained policies. 3.2 First Particular Case: The 1-Norm-Constrained Portfolios In this section, we consider the class of norm-constrained portfolios obtained by constraining the 1-norm of the portfolio-weight vector. The 1-norm is defined as the sum of the absolute values of one ignores estimates of mean returns and considers only the covariances between returns. If one wanted to consider the mean-variance portfolio problem with norm constraints, one would only need to replace the objective function in the problem defined in Equations (8) (10). Specifically, one needs to replace (8) with: maxw w T ˆµ t γ 2 wt ˆΣw, where γ is the investor s risk aversion parameter and µ is the R N vector of expected returns. With this change, our analysis would then apply to the mean-variance portfolio problem. We have implemented the resulting norm-constrained mean-variance portfolios and our empirical results (not reported) show that these portfolios do indeed achieve lower out-of-sample Sharpe ratios than the norm-constrained minimum-variance portfolios. 9 An estimator is unbiased if its expected value coincides with the parameter being estimated. 13

14 the portfolio weights w 1 = N i=1 w i. The resulting 1-norm-constrained portfolio problem is: min w w ˆΣw, (13) s.t. w e = 1, (14) N w i δ, (15) i=1 and we denote its solution by w NC1. The following proposition shows that for the case δ = 1, the solution to the 1-norm-constrained minimum-variance problem is the same as that for the shortsale-constrained minimum-variance portfolio analyzed by Jagannathan and Ma (2003). Proposition 2 The solution to the 1-norm-constrained problem (13) (15) with δ = 1 coincides with the solution to the shortsale-constrained problem (3) (5). A couple of comments are in order. First, note that Proposition 2 states that the shortsaleconstrained minimum-variance portfolio, which is studied in Jagannathan and Ma (2003), is a special case of the 1-norm-constrained minimum-variance portfolio for the threshold value of δ = 1. Then, by Proposition 1, we have that the shortsale-constrained portfolio can be interpreted as a portfolio resulting from shrinking the portfolio weights of the unconstrained minimum-variance portfolio. This interpretation is different from that given by Jagannathan and Ma (2003), who show that the shortsale-constrained portfolio can be interpreted as resulting from shrinking some of the coefficients of the sample covariance matrix. The interpretation we provide is useful because for an investor it is typically easier to think in terms of portfolio weights than the elements of the sample covariance matrix. Second, for threshold values of δ in (15) that are strictly larger than 1, our approach generates a class of portfolios that generalize the shortsale-constrained minimum-variance portfolio. To see this, note that the 1-norm can be rewritten as w 1 = N w i = w i w i, (16) i=1 i P(w) i N (w) 14

15 in which w i is the portfolio weight on the i th asset, P(w) is the set of asset indexes for which the corresponding portfolio weight is greater or equal than zero, P(w) = {i : w i 0}, and N (w) is the set of asset indexes for which the corresponding portfolio weight is negative, N (w) = {i : w i < 0}. Moreover, if the portfolio weights sum up to one (w e = 1), we have that w i = 1 w i. (17) i P(w) i N (w) From (16) and (17) we obtain w 1 = 1 2 w i. (18) i N (w) Equation (18) gives insight into the structure of this new class of portfolios. In particular, observe from (18) that imposing the constraint w 1 < δ for some δ > 1 is equivalent to assigning a shortsale budget; that is, a budget for the total amount of shortselling allowed in the portfolio. Indeed, using (18) we can rewrite the constraint in (15) on the 1-norm of the portfolio-weight vector as follows: i N (w) w i < δ 1 2, (19) in which the term i N (w) w i is the total proportion of wealth that is sold short and (δ 1)/2 is the shortsale budget. This shortsale budget can then be freely distributed among all of the assets. For instance, one could decide to short sell only one asset, and assign to this particular asset the weight of (δ 1)/2, or one could distribute the shortsale budget equally among a few of the assets. The w NC1 portfolio is different also from the generalized shortsale-constrained portfolio considered in DeMiguel, Garlappi, and Uppal (2007), which is the minimum-variance portfolio subject to the constraint w ξe for some ξ > 0, which requires that the amount of shortselling allowed for each assets is exactly the same. In contrast, the 1-norm constraint allows the investor more flexibility about how to distribute the allowed shortsale budget across all of the assets. Figure 3 depicts the 1-norm-constrained portfolios for the case with three risky assets. The three axes in the reference frame give the portfolio weights for the three risky assets. Two triangles are depicted in the figure. The larger triangle depicts the intersection of the plane formed by all portfolios whose weights sum up to one with the reference frame. The smaller triangle (colored) 15

16 represents the set of portfolios whose weights are nonnegative and sum up to one; that is, the set of shortsale-constrained portfolios. The ellipses centered around the minimum-variance portfolio, w MINU, depict the iso-variance curves; that is, the curves formed by portfolios with equal variance. The hexagons represent the iso-1-norm curves; that is, the sets of portfolios with equal 1-norm, which are obtained from the intersection of the 2-dimensional plane formed by all portfolios whose weights sum to one with the cube describing the region of weights satisfying the 1-norm constraint. For each value of the threshold δ = {δ 1, δ 2,..., δ 5 }, the 1-norm-constrained minimum-variance portfolio is the point where the corresponding iso-1-norm hexagon is tangent to the iso-variance curve. The 1-norm-constrained portfolios corresponding to values of the threshold parameter δ ranging from 1 to w MINU 1 describe a curve that joins the shortsale-constrained minimum-variance portfolio, w MINC, with the shortsale-unconstrained minimum-variance portfolio, w MINU. 3.3 Second Particular Case: The 2-Norm-Constrained Portfolios We now consider the class of portfolios obtained by constraining the 2-norm of the vector of portfolio weights. The 2-norm is the traditional Euclidean norm in R N, obtained from (11) by setting p = 2, or, in matrix form, w 2 = (w w) 1/2. Note, however, that the solution to the minimum-variance problem with a constraint on the 2-norm w 2 < δ is the same as the solution to the problem with a constraint on the squared 2-norm w 2 2 < δ2. Because the squared 2-norm is easier to analyze than the 2-norm, we define the 2-norm-constrained minimum-variance portfolio problem as follows: min w w ˆΣw, (20) s.t. w e = 1, (21) N w 2 i δ, (22) i=1 and we denote the solution to this problem by w NC2. 16

17 To gain intuition about the 2-norm constrained minimum-variance portfolios, note that the constraint in (22) can be reformulated equivalently as follows: 10 N i=1 ( w i 1 ) 2 ( δ 1 ). (23) N N The reformulated constraint (23) demonstrates that imposing the 2-norm constraint on the portfolio weights in (22) is equivalent to imposing a constraint that the 2-norm of the difference between this portfolio and the 1/N portfolio is bounded by δ 1/N. Note also from (20), (21) and (23) that the 1/N portfolio is a special case of the 2-norm-constrained portfolio if we set δ = 1/N. The following proposition shows that the 2-norm constrained portfolio belongs to the class of unconstrained portfolios that would be obtained by using shrinkage techniques on the covariance matrix as in Ledoit and Wolf (2004b). Proposition 3 Provided ˆΣ is nonsingular, for each δ 1/N there exists a ν 0 such that the solution to problem (20) (22) is the solution to the minimum-variance problem in (1) (2) with the sample covariance matrix, ˆΣ, replaced by ˆΣ LW = ˆΣ + νi. Proposition 3 shows that the Ledoit and Wolf (2004b) portfolio can be interpreted as one that shrinks the portfolio weights. Again, this is a different interpretation than the one given in Ledoit and Wolf (2004b), in which the portfolio is viewed as one obtained from shrinking the sample covariance matrix toward the identity matrix. Ledoit and Wolf (2004a) consider other targets toward which to shrink the sample covariance matrix; for instance, the single-factor matrix and the constant correlation matrix. The 2-norm-constrained portfolios described above can be extended to generalize also these other approaches discussed in Ledoit and Wolf. To do this, one simply needs to replace the constraint w w δ 2 by the constraint w F w δ 2, where F is the target matrix. For expositional simplicity, we focus on the case where the target matrix is the identity. Figure 4 depicts the 2-norm-constrained portfolios for the case with three risky assets. The isovariance curves are depicted as in Figure 3. The circumferences centered on the equally-weighted 10 To understand why these two formulations are equivalent, observe that P N i=1 (wi 1/N)2 = P N i=1 w2 i + P N i=1 1/N 2 P N i=1 2wi/N = P N i=1 w2 i 1/N, where the last result follows from the fact that P N i=1 1/N 2 = 1/N and 2wi/N = 2/N. P N i=1 17

18 portfolio, 1/N, are the iso-2-norm curves; that is, the set of portfolios with the same 2-norm. We depict four iso-2-norm curves corresponding to four different threshold levels: δ 1, δ 2, δ 3, and δ 4. For a given threshold δ, the 2-norm-constrained portfolio is the point where the corresponding iso- 2-norm curve is tangent to an iso-variance curve. 11 For values of δ ranging from 1/N to w MINU 2 2, the 2-norm-constrained portfolios describe a smooth curve that joins the 1/N portfolio (for δ = 1/N) with the minimum-variance portfolio (for δ w MINU 2 2 ); this will be formally proven in Proposition 7. We now compare the 1-norm and the 2-norm portfolios in Figures 3 and 4, respectively. From Figure 3, observe that because the 1-norm level sets are shaped as hexagons, the 1-norm-constrained minimum-variance portfolios tend to be located at the vertices of these hexagons in Figure 3 it can be observed that this is the case for δ 1, δ 2, and δ 3. This implies that the 1-norm-constrained portfolios are likely to assign a zero weight to at least some of the assets. This phenomenon is well known for the case of the shortsale-constrained minimum-variance portfolio, which tends to assign a zero weight to a large subset of the total number of assets available for investment; for a discussion of this see, for instance, Jagannathan and Ma (2003). But Figure 3 shows that, although this phenomenon also takes place for the rest of the 1-norm-constrained portfolios with δ > 1, these portfolios will tend in general to invest in a higher number of assets than the shortsale-constrained minimum-variance portfolio. To see this, note that the vertices of the triangle (level set for δ = 1) correspond to portfolios that assign a zero weight to two out of the three assets, whereas the vertices of each hexagon (level sets for δ > 1) correspond to points where the weight on only one of the three assets is zero. Hence, we may expect the 1-norm-constrained portfolios with δ > 1 to invest in a larger number of assets than the shortsale-constrained minimum-variance portfolio. In contrast, the 2-norm-constrained portfolios will typically assign a non-zero (positive or negative) weight to all assets. Thus, in situations where the investor prefers a diversified portfolio that has some weight in all assets, the investor should use the 2-norm-constrained portfolios, while if the investor is interested in portfolios with investment in a smaller number of assets, then she should focus on the 1-norm-constrained portfolios. 11 To see why the solution must be at a tangency point of the two level sets, note that at a point where the two level sets cross each other, we can always find a point with smaller sample variance by moving along the level set for the portfolio norm. 18

19 This is consistent with the interpretation of the 1- and 2-norm portfolios given in the previous sections. Specifically, in Section 3.3 we showed that the 2-norm-constrained portfolio is the portfolio that minimizes the sample variance subject to the constraint that the square of the 2-norm of the difference with the 1/N portfolio is bounded by δ 1/N. Consequently, we would expect that the 2-norm constrained portfolios will, in general, remain relatively close to the 1/N portfolio, and thus, will assign a positive weight to all assets. Also, in Section 3.2 we showed that the 1-norm constrained portfolios are a generalization of the shortsale-constrained portfolios in which the total amount of shortselling on all assets must remain below a shortsale budget of (δ 1)/2. Therefore, we may expect the 1-norm constrained portfolios to have the well-known property of shortsaleconstrained minimum-variance portfolios, which tend to assign a weight different from zero to only a few of the assets. Summarizing, investors who believe that the optimal portfolio is close to the well-diversified 1/N portfolio, would want to use a 2-norm constraint when solving the minimumvariance problem to reduce estimation error. Investors who, on the other hand, believe the total amount of shortselling of the optimal portfolio should not exceed a given budget, would want to use a 1-norm constraint. 3.4 Third Particular Case: The Partial Minimum-Variance Portfolios In this section, we propose a class of portfolios that are obtained by applying the classical conjugategradient method (Nocedal and Wright (1999, Chapter 5)) to solve the minimum-variance problem. The conjugate gradient is an iterative method that, starting from an initial guess reaches the optimal solution, which is the shortsale-unconstrained minimum-variance portfolio, in N 1 successive steps, where N is the number of assets (in Appendix A we provide a detailed description of this algorithm). This method generates a sequence of portfolios, where the first portfolio is some initial guess (taken to be the 1/N portfolio in our implementation) and the terminal portfolio is the shortsale-unconstrained minimum-variance portfolio. We term each of the intermediate portfolios a partial minimum-variance portfolio. Even though the partial minimum-variance portfolios are not obtained by imposing explicitly a constraint on the norm of the minimum-variance portfolio, we show that the norm of the par- 19

20 tial minimum-variance portfolios is indeed smaller than the norm of the shortsale-unconstrained minimum-variance portfolios. Moreover, we also show that the partial minimum-variance portfolios can be viewed as a discrete first-order approximation to the 2-norm-constrained portfolios. For this reason, we consider the partial minimum-variance portfolios as a particular case of the norm-constrained portfolios. The partial minimum-variance portfolios are the counterpart in portfolio selection to the statistical technique of partial least squares for regression analysis, which has been shown to perform very well under certain conditions (see Wold (1975); Frank and Friedman (1993), and Friedman and Popescu (2004)). Our empirical analysis in Section 4 shows that the partial minimum-variance portfolios perform very well also on financial data and that they often outperform not just the portfolios in the existing literature but also the 1- and 2-norm-constrained portfolios described above. The partial minimum-variance portfolios form a set of N 1 portfolios that join the 1/N portfolio and the shortsale-unconstrained minimum-variance portfolio. Specifically, the first of these N 1 portfolios, which we term the first partial minimum-variance portfolio, is a weighted average of the 1/N portfolio and the so-called first conjugate portfolio. The term partial refers to the fact that the first partial minimum-variance portfolio minimizes the sample variance within the subset of portfolios formed by combinations of the 1/N portfolio and the first conjugate portfolio. This first conjugate portfolio is the zero-cost portfolio (i.e., a portfolio whose weights sum up to zero) that induces the maximum marginal decrease in the sample variance when combined with the 1/N portfolio. The second partial minimum-variance portfolio is then that combination of the first partial minimum-variance portfolio and the second conjugate portfolio that minimizes the sample variance, where the second conjugate portfolio is the zero-cost portfolio that, when combined with the first partial minimum-variance portfolio, induces the maximum marginal decrease in the sample portfolio variance, subject to the condition that it is conjugate with respect to the first conjugate portfolio; that is, subject to the condition that the first two conjugate portfolios are uncorrelated with respect to the sample covariance matrix. 20

21 By iterating this process N 1 times we generate a discrete set of N 1 portfolios (including the shortsale-unconstrained minimum-variance portfolio) that join the 1/N and the shortsaleunconstrained minimum-variance portfolios. Details of how to compute the partial minimumvariance portfolios are provided in Appendix A Analytical Expression for the First Partial Minimum-Variance Portfolio In this section, we provide an analytical expression for the first partial minimum-variance portfolio and use this to give some intuition for its properties. To do so, we first note that the gradient of the sample portfolio variance, w ˆΣw, with respect to the portfolio weight, w, is ) w (w ˆΣw = 2ˆΣw. (24) Moreover, this gradient evaluated at the starting 1/N portfolio is ( ) e/n w w ˆΣw = 2ˆΣ e N. (25) Hence, ˆΣe/N is the portfolio that when combined with the 1/N portfolio yields the largest marginal decrease in the portfolio sample variance. 12 But note that this portfolio is not a zero-cost portfolio; that is, its weights do not add up to zero in general. The properties of the gradient then imply that the first conjugate portfolio is simply the zero-cost portfolio that is closest (in 2-norm) to the portfolio ˆΣe/N. The properties of Euclidean spaces and some algebra then imply that the first conjugate portfolio, w CG1, is w CG1 = ) (I ee ˆΣ e N N, (26) because the matrix (I ee N ) is the projection matrix that projects any portfolio into the set of portfolios with zero-cost. The first partial minimum-variance portfolio is the combination of the 1/N portfolio and the first conjugate portfolio that minimizes the sample portfolio return variance; that is, the first partial 12 Note that any scalar multiple of the vector 2ˆΣe/N yields the largest marginal decrease in sample variance when combined with 1/N. For notational convenience, we use ˆΣe/N instead of 2ˆΣe/N. 21

22 minimum-variance portfolio is w P AR1 = e/n + α 0 w CG1, (27) in which α 0 is chosen to minimize the sample variance of the first partial minimum-variance portfolio. We now explain the intuition underlying this portfolio. First, in Proposition 4, we show that the i th component of the vector ˆΣ e N is the sample covariance of the return on the 1/N portfolio with the return on the i th risky asset. Proposition 4 The i th component of the vector ˆΣ e N is the sample covariance between the 1/N portfolio and the return on the i th risky asset. Then, in Proposition 5, we show that the i th component of the first conjugate portfolio is the negative of the deviation of this covariance from the average of the covariances between the 1/N portfolio and each of the individual assets. Proposition 5 Let σ ei be the covariance between the 1/N portfolio return and the i th risky asset return. Then, the i th component of the first conjugate portfolio, (w CG1 ) i, is (w CG1 ) i = ( σ ei 1 N N ) σ ej. (28) j=1 This result makes sense: the first conjugate portfolio is a zero-cost portfolio that assigns a positive weight to those assets whose covariance with the return of the 1/N portfolio is below average. As a result, given that α 0 > 0, the first partial minimum-variance portfolio assigns a weight larger than 1/N to those assets whose covariances with the 1/N portfolio are below average. This improves the diversification of the portfolio, and thus, decreases the portfolio variance. Also, note that the minimum-variance portfolio is the portfolio whose covariance with all other assets is equal to a constant (see Huang and Litzenberger (1988, Chapter 3, Proposition 12)). Thus, according to the interpretation above, the first partial minimum-variance portfolio should be closer to the shortsale-unconstrained minimum-variance portfolio than the 1/N portfolio. 22