Nonparametric nearest neighbor based empirical portfolio selection strategies

Transcription

1 Finance and Stochastics manuscript No. (will be inserted by the editor) Nonparametric nearest neighbor based empirical portfolio selection strategies László Györfi, Frederic Udina 2, Harro Walk 3 Department of Computer Science and Information Theory. Budapest University of Technology and Economics. 52 Stoczek u. 2, Budapest, Hungary. gyorfi@szit.bme.hu 2 Department of Economics and Business, Universitat Pompeu Fabra. Ramon Trias Fargas 25-27, Barcelona, Spain. udina@upf.es Fax: Institute of Stochastics and Applications. Universität Stuttgart. Pfaffenwaldring 57, D Stuttgart, Germany. walk@mathematik.uni-stuttgart.de Received: date / Revised version: date Abstract In recent years optimal portfolio selection strategies for sequential investment have been shown to exist. Although their asymptotical optimality is well established, finite sample properties do need the adjustment of parameters that depend on dimensionality and scale. In this paper we introduce some nearest neighbor based portfolio selectors that solve these problems, and we show that they are also log-optimal for the very general class of stationary and ergodic random processes. The newly proposed algorithm shows very good finite-horizon performance when applied to different markets with different dimensionality or scales without any change: we see it as a very robust strategy. Key words sequential investment, universally consistent portfolios, nearest neighbor estimation JEL Classification: C4. Mathematics Subject Classification (2000): 60G0, 62G05, 62L2, 62M20, 62P05, 9B28. The first author acknowledges the support of the Computer and Automation Research Institute and the Research Group of Informatics and Electronics of the Hungarian Academy of Sciences. The work of the second author was supported by the Spanish Ministry of Science and Technology and FEDER, grant BMF We thank Michael Greenacre for his careful and annotated reading.

2 2 László Györfi et al. Introduction The purpose of this paper is to further investigate sequential investment strategies for financial markets. Sequential investment strategies use information collected from the past of the market and determine, at the beginning of a trading period, a portfolio, that is, a way to distribute the current capital among the available assets. The goal of the investor is to maximize his wealth in the long run, and this is to be done without knowing the underlying distribution generating the stock prices. The only mathematical assumption we use in our analysis is that the daily price relatives form a stationary and ergodic process. Under this assumption the asymptotic rate of growth has a well-defined maximum, which can be achieved in full knowledge of the distribution of the entire process, see Algoet and Cover [3]. Universally consistent procedures achieving the same asymptotic growth rate without any previous knowledge have been known to exist, see Algoet [], Györfi and Schäfer [], Györfi, Lugosi, and Udina [0]. In this paper new universal strategies are proposed which not only guarantee an optimal asymptotic growth rate of capital for all stationary and ergodic markets, but also have a good finitehorizon performance in practice, and, as main novelty, are very robust in the sense that no parameter tuning is needed to guarantee this good finite-horizon performance. This is demonstrated on an experimental study in which the performance of the proposed methods is measured on different data sets, including past New York Stock Exchange (NYSE) data spanning a 22 year period with 36 stocks included. To make the analysis feasible, some simplifying assumptions are used that need to be taken into account. We discuss it in detail in Section 4, where the algorithms are checked with data from real financial markets. Since the predictions made by our methods are based only on the information available from past behaviour of the market, no wealth increase will be possible under the assumption of efficiency of the market. This means that only in the presence of inefficiency of markets can our algorithms give the very good performance that we show in the examples. In other words, our numerical results provide strong empirical evidence for the inefficiency of some stock markets in some time periods. This may partially be explained by the fact that the dependence structures of the markets revealed by the proposed investment strategies are quite complex and, even though all information we use is publicly available, the way this information can be exploited remains hidden from most traders. The rest of the paper is organized as follows. In Section 2 the mathematical model is described, and related results are surveyed briefly. In Section 3 a nearest neighbor (NN) based nonparametric sequential investment strategy is introduced and its main consistency properties are stated. Numerical results based on various data sets are described in Section 4. The proof of the main theoretical result (Theorem ) is given in the Appendix.

3 Nonparametric nearest neighbor based empirical portfolio selection strategies 3 2 Mathematical model The stock market model investigated in this paper is the one considered, among others, by Breiman [7], Algoet and Cover [3]. Consider a market of d assets. A market vector x = (x (),... x (d) ) R d + is a vector of d nonnegative numbers representing price relatives for a given trading period. That is, the j-th component x (j) 0 of x expresses the ratio of the closing and opening prices of asset j. In other words, x (j) is the factor by which capital invested in the j-th asset grows during the trading period. The investor is allowed to diversify his capital at the beginning of each trading period according to a portfolio vector b = (b (),... b (d) ). The j-th component b (j) of b denotes the proportion of the investor s capital invested in asset j. Throughout the paper we assume that the portfolio vector b has nonnegative components with d j= b(j) =. This unit-sum constraint means that the investment strategy is self-financing and consumption of capital is excluded. The non-negativity of the components of b means that short selling and buying stocks on margin are not permitted. Let S 0 denote the investor s initial capital. Then at the end of the trading period the investor s wealth becomes S = S 0 d b (j) x (j) = S 0 b, x, j= where, denotes inner product. The evolution of the market in time is represented by a sequence of market vectors x, x 2,... R d +, where the j-th component x (j) i of x i denotes the amount obtained after investing a unit capital in the j-th asset on the i-th trading period. For j i we abbreviate by x i j the array of market vectors (x j,..., x i ) and denote by d the simplex of all vectors b R d + with nonnegative components summing up to one. An investment strategy is a sequence B of functions i= b i : ( R d +) i d, i =, 2,... so that b i (x i ) denotes the portfolio vector chosen by the investor on the i-th trading period, upon observing the past behavior of the market. We write b(x i ) = b i (x i ) to ease the notation. b is a constant portfolio vector, usually (/d,.../d). Starting with an initial wealth S 0, after n trading periods, the investment strategy B achieves the wealth { n n S n = S 0 b(x i ), x i = S0 exp log b(x i } ), x i. This may be written as S 0 exp {nw n (B)}, where W n (B) denotes the average growth rate W n (B) = n log b(x i ), x i. n i= i=

4 4 László Györfi et al. Obviously, maximization of S n = S n (B) and maximization of W n (B) are equivalent. In this paper we assume that the market vectors are realizations of a random process, and describe a statistical model. Our view is completely nonparametric in that the only assumption we use is that the market is stationary and ergodic, allowing arbitrarily complex distributions. More precisely, assume that x, x 2,... are realizations of the random vectors X, X 2,... drawn from the vector-valued stationary and ergodic process {X n }. (Note that by Kolmogorov s theorem any stationary and ergodic process {X n } can be extended to a bi-infinite stationary process on some probability space (Ω, F, P), such that ergodicity holds for both n and n.) The sequential investment problem, under these conditions, have been considered by, e.g., Breiman [7], Algoet and Cover [3], Algoet [, 2], Györfi and Schäfer [], Györfi, Lugosi, and Udina [0]. The fundamental limits, determined in [3], [,2], reveal that the so-called log-optimum portfolio B = {b ( )} is the best possible choice. More precisely, on trading period n let b ( ) be such that E { log b (X n ), X n X n } = max E { log b(x n } ), X n X n. b( ) If S n = S n(b ) denotes the capital achieved by a log-optimum portfolio strategy B, after n trading periods, then for any other investment strategy B with capital S n = S n (B) and for any stationary and ergodic process {X n }, lim sup n n log S n Sn 0 almost surely and where lim n n log S n = W almost surely, { W = E max E { log b(x ), X 0 X } } b( ) is the maximal possible growth rate of any investment strategy. Thus, (almost surely) no investment strategy can have a faster rate of growth than a log-optimal portfolio. Of course, to determine a log-optimal portfolio, full knowledge of the (infinite-dimensional) distribution of the process is required. Strategies achieving the same rate of growth without knowing the distribution are called universally consistent in this paper. More precisely, an investment strategy B is called universally consistent with respect to a class of stationary and ergodic processes {X n }, which can be considered as an estimate of B, if for each process in the class, lim n n log S n(b) = W almost surely. The surprising fact that there exists a strategy universal with respect to the class of all stationary and ergodic processes was first proved by Algoet [].

5 Nonparametric nearest neighbor based empirical portfolio selection strategies 5 2. Histogram based strategy Next we describe Györfi and Schäfer s version of Algoet s scheme as the investment strategies defined in this paper are generalizations of this method. We call this scheme a histogram based investment strategy and denote it by B H. B H is constructed as follows. We first define an infinite array of elementary strategies (the so-called experts) H (k,l) = {h (k,l) ( )}, indexed by the positive integers k, l =, 2,.... Each expert H (k,l) is determined by a period length k and by a partition P l = {A l,j }, j =, 2,..., m l of R d + into m l disjoint sets. To determine its portfolio on the nth trading period, expert H (k,l) looks at the market vectors x n,..., x n of the last k periods, discretizes this kd-dimensional vector by means of the partition P l, and determines the portfolio vector which is optimal for those past trading periods whose preceding k trading periods have identical discretized market vectors to the present one. Formally, let G l be the discretization function corresponding to the partition P l, that is, G l (x) = j, if x A l,j. With some abuse of notation, for any n and x n Rdn, we write G l (x n ) for the sequence G l (x ),..., G l (x n ). Then define the expert H (k,l) = {h (k,l) ( )} by writing, for each n > k +, h (k,l) (x n ) = arg max b, x i, () b d i J k,l,n where J k,l,n = { k < i < n : G l (x i i ) = G l(x n n )}, if J k,l,n, and uniform b 0 = (/d,..., /d) otherwise. That is, h n (k,l) discretizes the sequence x n according to the partition P l, and browses through all past appearances of the last seen discretized string G l (x n n ) of length k. Then it designs a fixed portfolio vector optimizing the return for the trading periods following each occurrence of this string. The histogram based strategy B H forms a mixture of all experts H (k,l) using a positive probability distribution {q k,l } on the set of all pairs (k, l) of positive integers (i. e. such that for all k, l, q k,l > 0). The strategy B H simply weights the experts H (k,l) according to their past performances and {q k,l } such that after the nth trading period, the investor s capital becomes S n (B H ) = k,l q k,l S n (H (k,l) ), where S n (H (k,l) ) is the capital accumulated after n periods when using the portfolio strategy H (k,l) with initial capital S 0 =. This may easily be achieved by distributing the initial capital S 0 = among all experts such that expert H (k,l) trades with initial capital q k,l S 0. It is shown in [] that the strategy B H is universally consistent with respect to the class of all ergodic processes such that E{ log X (j) } <, for all j =, 2,..., d under the following two conditions on the partitions used in the discretization:

6 6 László Györfi et al. (a) the sequence of partitions is nested, that is, any cell of P l+ is a subset of a cell of P l, l =, 2,...; (b) if diam(a) = sup x,y A x y denotes the diameter of a set, then for any sphere S R d centered at the origin, lim max diam(a l,j) = 0. j:a l,j S l 2.2 Kernel based strategy Györfi, Lugosi, and Udina [0] introduced kernel based strategies, here we describe only the simplest moving-window version, corresponding to a uniform kernel function. Just like before, we start by defining an infinite array of experts H (k,l) = {h (k,l) ( )}, where k, l are positive integers. For fixed positive integers k, l, choose the radius r k,l > 0 such that for any fixed k, lim r k,l = 0. l Then, for n > k +, define the expert h (k,l) by h (k,l) (x n ) = arg max b d i J k,l,n b, x i, where J k,l,n = { k < i < n : x i i xn n r k,l}, if J k,l,n, and b 0 = (/d,..., /d) otherwise. These experts are mixed the same way as in the case of the histogram based strategy. That is, let {q k,l } be a positive probability distribution as before. If S n (H (k,l) ) is the capital accumulated by the elementary strategy H (k,l) after n periods when starting with an initial capital S 0 =, then after period n the investor s capital becomes S n (B K ) = q k,l S n (H (k,l) ). k,l Györfi, Lugosi, and Udina [0] proved that the portfolio scheme B K is universally consistent with respect to the class of all ergodic processes such that E{ log X (j) } <, for j =, 2,... d. 3 Nearest neighbor based strategy Define an infinite array of experts H (k,l) = {h (k,l) ( )}, where 0 < k, l are integers. Just like before, k is the window length of the near past, and for each l choose p l (0, ) such that lim p l = 0. (2) l

7 Nonparametric nearest neighbor based empirical portfolio selection strategies 7 Put ˆl = p l n. At a given time instant n, the expert searches for the ˆl nearest neighbor (NN) matches in the past. For fixed positive integers k, l (n > k + ˆl + ) and for each vector s = s of dimension kd introduce the set of the ˆl nearest neighbor matches: Ĵ (k,l) n,s = {i; k + i n such that x i i is among the ˆl NNs of s in x k,..., xn n }. Define the portfolio vector by b (k,l) (x n, s) = arg max b d i Ĵ(k,l) n,s b, x i. We define the expert h (k,l) by h (k,l) (x n ) = b (k,l) (x n, x n ), n =, 2,... (3) That is, h n (k,l) is a fixed portfolio vector according to the returns following these nearest neighbors. These experts are mixed the same way as before. That is, let {q k,l } be a probability distribution over the set of all pairs (k, l) of positive integers such that for all k, l, q k,l > 0. If S n (H (k,l) ) is the capital accumulated by the elementary strategy H (k,l) after n periods when starting with an initial capital S 0 =, then after period n the investor s capital becomes n S n (B NN ) = k,l q k,l S n (H (k,l) ). We say that a tie occurs with probability zero if for any vector s = s k the random variable has continuous distribution function. X k s Theorem Assume (2) and that a tie occurs with probability zero. The portfolio scheme B NN is universally consistent with respect to the class of all stationary and ergodic processes such that E{ log X (j) } <, for j =, 2,... d.

8 8 László Györfi et al. 4 Empirical results In this section we present some numerical results obtained by applying the above algorithms to two sets of financial data. The first data set, described and analyzed in Section 4., includes prices for 36 NYSE stocks along 22 years. In Section 4.2 we analyze currency exchange data for eight currencies during a period of more than 3 years. Testing the algorithms with data from real financial markets is meaningful, but it needs some previous considerations about the assumptions implied in our model that are not found in real markets. First, we assume that assets are available in the desired quantities at a given price at any trading period. The investment period is modeled as a single instant: there is a single price for the entire period, the closing price at the previous period. After we do the desired transaction, we are informed about the closing price and the period ends. We also assume that assets are arbitrarily indivisible. As in the mathematical analysis, we ignore transaction costs to be paid when switching portfolios. Moreover, all the wealth achieved in the last period is fully invested in the next one, without any extra investment allowed (no short sales on margin). Also, the set of assets involved is fixed: no asset may disappear, no new assets are allowed to be introduced in the market. Another implicit assumption is that prices are not affected by our actions on the market (since we use historical data, we are forced to assume that). In the NYSE data example below, this is not a realistic assumption since we are trading enormous amount of asset values, not negligible even in comparison with the full market (we multiply our initial wealth by 0 2 after about 20 years of trading). All the proposed universally consistent algorithms use an infinite array of experts. In practice we take a finite array of size K L (usually 5 0). We also include, as an additional expert, with index k = l = 0, the strategy that uses the full history to calculate the portfolio by h (0,0) (x n ) = arg max b d 0<i<n b, x i, n >. In all cases we take the uniform distribution {q k,l } = /(KL+) over the experts in use. Implementation of the histogram B H and kernel B K portfolios is described in detail in Györfi, Lugosi, and Udina [0]. In the results presented below, B K (c) denote the kernel portfolio where the expert (k, l) uses r k,l = c/l as radius for the kernel. For computational complexity reasons, see below, we introduce a variant of the nearest neighbor rule based on the one described previously. Just like before, k is the window length of the near past, however, s l is a segment length. At a given time instant n, the expert searches for a single nearest neighbor match within each segment of length s l. To define H (k,l), for fixed positive integers k, l introduce first the nearest neighbors within each segment. For i (n )/s l, let

9 Nonparametric nearest neighbor based empirical portfolio selection strategies 9 N i be the instant of nearest neighbor match of x n n within the ith segment: N i = arg min x j j xn n. (i )s l +k j<is l We define the expert h (k,l), for n > s l +, by h (k,l) (x n ) = arg max b d { i (n )/s l } b, x Ni. Then we combine experts in the usual way and this way we obtain the firstneighbor that we shall denote by B fn. It is not difficult to see that Theorem is also valid for the first neighbor variant, provided that (2) is replaced by lim s l =. (4) l When using real data, ties occur quite often, sometimes due to rounding. But in many cases it is also true that real data can not be assumed coming from a continuous process: for example, the NYSE data discussed below present many cases of relative price equal to one, much more than one may expect coming from rounding. This is not surprising as one may expect that a given asset does not vary its price on several trading days. Presence of ties does not result in any problem or degradation of performance of the nearest-neighbor portfolio, as can be seen in the results presented below. We deal with ties in the following way: when taking ν elements from a set that have distances d i (assuming distances in increasing order), if d ν = d ν+, we take also the element with index ν + together with all subsequent elements that show the same distance. In the first neighbor case, in the presence of ties within a segment, we take the more recent period of those tied. In the nearest neighbor portfolio implementation, we take p l = l L, so our 50 experts take from 2% to 52% of the history as matching periods. For the first neighbor portfolio we take s l = l L. With this choice the number of matches used by the experts is very similar in both variants. It is possible that other choices of these quantities may improve the performance of the algorithms, but this would be true for some situations or data sets and not for other. We do not want to find parameter choices tailored for a particular data set. We want to stress that a single and reasonable choice of these parameters works very well in very different markets, not depending on the dimensionality or the scale of the relative prices being considered. In the examples below, the very same choice works for d = 2 or d = 36, for stock exchange data or for currency exchange data, scales in those last cases being very different one from another.

10 0 László Györfi et al. 4. Stock market data The first data set we use is a standard set of New York Stock Exchange data used by Cover [8], Singer [3], Hembold, Schapire, Singer, and Warmuth [2], Blum and Kalai [4], Borodin, El-Yaniv, and Gogan [5], and others. It includes daily prices of 36 assets along a 22-year period (565 trading days) ending in 985. Table Wealth achieved by different strategies by investing in the pairs of NYSE stocks used in Cover [8]. In the second column we show some reference results from the literature. In the right part of the table results are shown for the histogram, kernel and nearest neighbors strategies. Stocks Best Exp. [k, l] Iroquois Best asset 8.92 B H 2.3e+0.395e+ [,] Kin Ark BCRP B K 4.038e e+ [2,2] Oracle 6.85e+53 B NN.56e+2.439e+3 [2,8] Cover UP B fn 5.094e+ 6.08e+2 [2,3] Singer SAP 43.7 Com. Met. Best asset B H [2,] Mei. Corp BCRP 03.0 B K [2,5] Oracle 2.2e+35 B NN 3.505e e+4 [3,6] Cover UP B fn.08e e+4 [4,2] Singer SAP 07.7 Com. Met. Best asset B H.33e e+0 [,] Kin Ark BCRP 44.0 B K.e+.4e+2 [3,3] Oracle.84e+49 B NN 4.78e e+3 [3,7] Cover UP B fn 9.03e e+2 [3,2] Singer SAP IBM Best asset 3.36 B H [,5] Coca-Cola BCRP 5.02 B K [,6] Oracle.08e+5 B NN [,7] Cover UP 4.24 B fn [4,9] Singer SAP 5.05 We first take the pairs used in the aforementioned papers, see Table. In the second and third columns of the table we show the wealth achieved by investing one US$, respectively, in the best of both assets, following the best constantly rebalanced portfolio (BCRP), by an Oracle that knows in advance the prices for the next period, using the universal portfolio as reported in Cover [8], and the SAP portfolio as reported in Singer [3]. Note that BCRP and Oracle do not correspond to any valid investment strategy; they can only be determined in full hindsight. In the right part of the table we report our results for histogram (B H ), kernel (B K, with constant c = 0.05), nearest neighbor (B NN ) and first neighbor (B fn ) portfolios. For each pair, we start with one unit of wealth (say one US dollar), we use K =,..., 5 and L =,..., 0 for a total of 5 experts, including the expert that uses the full available past to compute the optimum portfolio. In the last

11 Nonparametric nearest neighbor based empirical portfolio selection strategies column we report the wealth achieved by the best expert among the 5 competing, and its values for k, l. Investing in a fixed pair of assets involves the difficult choice of the pair, so we prefer to report results for a more blind strategy: simply invest in all assets available. Table 2 summarizes the wealth achieved by several portfolio strategies when one dollar is split between the 36 assets in the first period. Our implementation of the histogram portfolio does not allow for such a large dimensionality. For the kernel portfolio we take c =.00 as a good value for this dimensionality. Nearest and first neighbor portfolios are as described before. In all cases, we use K = 5, L = 0. For the sake of reference, we also indicate the wealth achieved by BCRP. Table 2 Wealth achieved by various strategies. In all cases one unit is invested in the first period uniformly in all 36 stocks included in our NYSE data set. B K is the kernel strategy with constant c =.00, B NN is the nearest neighbor portfolio, B fn is the first neighbor variant of the last one, and BCRP is the best constantly rebalanced portfolio. After period BCRP B K (.00) B NN B fn e e+5.326e e e e e+6.489e e e+7.322e e e+7.238e e e e e e e e+ A better graphical comparison of results achieved by our algorithms can be seen in Figure where the full time series is represented. It is interesting to observe that while the kernel portfolio needs about 2000 periods to start getting some wealth, the nearest neighbor is able to exploit the information very soon: even it detects around n = 400 that there is one particular asset that is growing fast in these periods. A more detailed comparison of the different performance of kernel versus nearest neighbor is made in Table 3. There we list the wealth achieved by each expert at the end of the full time period. In the lower half of the listing it can be seen that most of the experts in the nearest neighbor setting have quite good performance, all of them being within a factor of 0 5 of the overall wealth. On the contrary, in the upper half there are experts with very good performance but there are also many with very low final wealth, of the order of 0 8 times less than the overall final wealth. Obviously, some better choice of the parameters that define the radius of

12 2 László Györfi et al. Investing on 36 NYSE assets e+2 Nearest Neigh. Unif. kernel (.0) BCRP e+0 e+08 e Fig. Wealth achieved along N = 565 daily periods by investing one US$ in 36 NYSE stocks (data set described in the text) using several asymptotically optimal strategies. Horizontal axis is time period number, vertical is wealth achieved, on logarithmic scale. the kernel will provide better results, but these would be tailored for this particular case, dimension, and data set, while the parameters chosen for the nearest neighbor are valid without any change for all situations, for different values of k and dimensions. This is why we see (in this and the rest of the examples we studied, not all reported here) the nearest neighbor as a very robust algorithm that performs fairly well without any adjustment for different data sets, dimensionalities and scales. 4.2 Currency exchange data We applied our algorithms to a data set of currency exchange rates to check that the nearest neighbor is very robust against changes of scale or dimensionality. Data were obtained from Datastream Advance, a commercial database, and include exchange rates to US$ for seven currencies (see Table 4) from December 6, 99 to January 27, 2005, a total of 3429 daily periods in just over 3 years. The currencies are Japanese Yen, Swiss Franc, Norwegian Crown, European Currency Unit followed by the Euro when it was introduced, British Pound, Canadian Dollar and Hungarian Forint. In Table 4 we report, for each currency, the final wealth of one US$ invested in that currency from the first period, and also some statistics of the series of daily returns (ratio of price to previous price, minus one) for all 3429 periods. Statistics include the mean, standard deviation, minimum, 25th and 75th percentiles, and maximum. We do not include the median because it is identical to zero in all cases. Table 5 and Figure 2 show the results of investing one US$ split into the seven currencies and cash and then selecting the portfolio for each period according the histogram, kernel, nearest neighbors and first neighbor. We use K = 5 in all cases,

13 Nonparametric nearest neighbor based empirical portfolio selection strategies 3 Table 3 Wealth achieved by each expert at the last period when investing one unit in the 36 NYSE assets. Upper part is for the kernel (c =.0) portfolio, while the lower part is for the nearest neighbor portfolio. Experts are indexed by k =..5 in columns and l =..0 in rows. B K (.0) on 36 NYSE assets. S 565 =.e + 9 l k e+ 4.2e+ 8.2e e+ 7.5e+2 7.6e+4 4.9e+6.0e e+3 4.7e+6.2e+7 4.9e+5 5.4e+2 5.7e+8 8.0e+7 7.e+2 2.e+2 3.e e+9 3.9e+6.e+2 2.2e+ 2.7e e+0 3.5e+3.7e+ 2.7e+ 2.7e e+9 3.8e+ 2.4e+ 2.7e+ 2.7e e+8 4.3e+ 2.7e+ 2.7e+ 2.7e e+5 3.4e+ 2.7e+ 2.7e+ 2.7e+ B NN on 36 NYSE assets. S 565 = 3.3e + 5.2e+0.3e+8.2e+7.5e+8 3.0e+7 2.4e+ 6.4e+8 3.5e+7 2.2e+8.9e+8 3.9e+ 2.0e+9.e+8.e+9 9.5e+8 4.6e+ 8.e+8 6.0e+8.5e+0 2.9e e+ 7.0e+8.3e+8 2.0e+0 7.0e e+ 2.9e+9.9e+9 4.3e+9.3e e+2 8.e+8.7e+9 7.7e+9 2.0e e+2 2.4e+9 4.6e+8.7e+0 4.3e e+.3e+9 7.3e+8 4.e+9.5e e+ 9.8e+8.9e+7 5.5e+7 7.e+6 Table 4 Some descriptive statistics for the exchange rate data. Currency Final Mean St. Dev. Min. p 25 p 75 Max. Jap. Yen Swiss Fr Norw. Kr Ecu/Euro Brit. Pnd Can. Dol Hung. Fnt and L = 0 in all cases except for the histogram that uses L = 6. The kernel algorithm is used here with c = 0. to adjust it to the dimensionality and scale of this case. The nearest neighbor algorithms are used as described before, without any parameter tuning. The second column of Table 5 reports results for the best constant rebalanced portfolio. Note that we introduce cash as a possible asset to invest in along with the seven currencies. We consider cash not having any variation, i. e. no interest rate at all.

14 4 László Györfi et al. Results here are fairly good, note that a wealth factor of along 3429 periods is roughly equivalent to a yearly increase rate of 57%, so even in this case our algorithms are able to efficiently exploit the existing inefficiency of this market. What should be remarked again is that the nearest neighbor algorithms perform better than the rest without needing any parameter adjustment. Table 5 Wealth achieved by various strategies. In all cases one unit is invested in the first period uniformly in all currencies included in our data set, cash is also included. BCRP is the best constantly rebalanced portfolio, B H is the histogram strategy, B K (0.) is the kernel strategy, B NN is the nearest neighbor portfolio, and B fn is the first neighbor variant of the last one. After period BCRP B H B K (0.) B NN B fn Investing in seven currencies and cash 00 Nearest Neighbor First Neighbor Kernel c=0.0 Histogram BCRP Yen Cash Swiss Norw Euro Brit Can Hung Fig. 2 Wealth achieved along N = 3429 daily periods by investing one US$ in seven foreign currencies and cash. Labels in the legend are in the same top down order as the lines in the graph. Horizontal axis is time period number, vertical is the wealth achieved, in logarithmic scale.

15 Nonparametric nearest neighbor based empirical portfolio selection strategies Computational cost The methods introduced here are quite costly to compute. There are essentially two steps that are very computing intensive: the selection of the matches in the past and the optimization step. The computing cost of the optimization step is related to the number of matches found. To give an idea of the importance of this, note that the kernel algorithm, when used for our NYSE data with c = 0.5, took half the time (2 hours) than when used with c =.0. All times reported here are approximate and refer to a run over the full period of the data set using a PC with a Pentium Xeon CPU at 2.0Ghz. In the nearest neighbor case, running all n = 565 periods for the d = 36 assets took 36 hours, while using the first neighbor variant this time reduced to 2 hours. The computing load due to the optimization is the same in both cases, but searching for the matches in the first case is much more costly because it requires sorting all previous data sequences according to the distance to the current one. To show a more complete comparison of the discussed algorithms, we may mention that computation of the optimal portfolio for the Iroquois/Kin Ark pair mentioned in Section 4. took 46 minutes for the nearest neighbor portfolio, 37 minutes for the first neighbor, and 0 minutes for the kernel portfolio with c = Conclusion We have shown how to construct and implement an algorithm for sequential investment. The algorithm is based on nearest neighbor estimation and we prove that it has asymptotical optimality for the very general class of stationary and ergodic processes. The more interesting property of the new algorithm is its robustness: it may be applied to different situations without needing any adjustment to the scale or dimensionality. Empirical results on real financial data show very good finite-horizon performance and are even spectacular in some cases. Appendix. Proofs The proof of Theorem uses the following three auxiliary results. The first is known as Breiman s generalized ergodic theorem [6], see also Algoet [2]. Lemma (BREIMAN [6]). Let Z = {Z i } be a stationary and ergodic process. For each positive integer i, let T i denote the operator that shifts any sequence {..., z, z 0, z,...} by i digits to the left. Let f, f 2,... be a sequence of realvalued functions such that lim n f n (Z) = f(z) almost surely (a.s.) for some function f. Assume that E sup n f n (Z) <. Then lim n n n f i (T i Z) = Ef(Z) i= (a.s.)

16 6 László Györfi et al. The next two lemmas are due to Algoet and Cover [3, Theorems 3 and 4]. Lemma 2 (ALGOET AND COVER [3]). Let Q n N { } be a family of regular probability distributions over the set R d + of all market vectors such that E{ log U n (j) } < for any coordinate of a random market vector U n = (U n (),..., U n (d) ) distributed according to Q n. In addition, let B (Q n ) be the set of all log-optimal portfolios with respect to Q n, that is, the set of all portfolios b that attain max b d E{log b, U n }. Consider an arbitrary sequence b n B (Q n ). If Q n Q weakly as n then, for Q -almost all u, lim b n, u b, u n where the right-hand side is constant as b ranges over B (Q ). Lemma 3 (ALGOET AND COVER [3]). Let X be a random market vector defined on a probability space (Ω, F, P) satisfying E{ log X (j) } <. If F k is an increasing sequence of sub-σ-fields of F with F k F F, then { } { } E max E [log b, X F k] E max E [log b, X F ] b b as k where the maximum on the left-hand side is taken over all F k -measurable functions b and the maximum on the right-hand side is taken over all F -measurable functions b. Proof of Theorem. The proof is based on techniques used in related prediction problems, see Györfi and Schäfer [], Györfi, Lugosi, and Udina [0]. We need to prove that lim inf W n(b) = lim inf n n n log S n(b) W (a.s.).

17 Nonparametric nearest neighbor based empirical portfolio selection strategies 7 Without loss of generality we may assume S 0 =, so that Thus W n (B) = n log S n(b) = n log q k,l S n (H (k,l) ) k,l ) (sup n log q k,l S n (H (k,l) ) k,l = ( ) n sup log q k,l + log S n (H (k,l) ) = sup k,l k,l ( W n (H (k,l) ) + log q k,l n ). ( lim inf W n(b) lim inf sup W n (H (k,l) ) + log q ) k,l n n k,l sup lim inf k,l n n ( W n (H (k,l) ) + log q k,l n = sup lim inf W n(h (k,l) ). (5) k,l n The simple argument above shows that the asymptotic rate of growth of the strategy B is at least as large as the supremum of the rates of growth of all elementary strategies H (k,l). Thus, to estimate lim inf n W n (B), it suffices to investigate the performance of expert H (k,l) on the stationary and ergodic market sequence X 0, X, X 2,.... First let the integers k, l and the vector s = s Rdk + be fixed. Fix p l (0, ). Put l = p l j. Let S s,r denote the closed sphere centered at s with radius r. Let the interval be the set of values r k,l (s) such that R k,l (s) = [r k,l(s), r k,l(s)] P{X S s,r k,l (s)} = p l. Since tie occurs with probability zero, such interval exists. Because of (2), For j > k + l +, introduce the set J (k,l) lim l r k,l(s) = 0. = {i; j + k + i 0 such that X i i is among the l NNs of s in X,..., X j+k j+ }. )

18 8 László Györfi et al. For all Borel sets A, let P (k,l) denote the (random) measure defined by P (k,l) {A} = i J (k,l) J (k,l) We will show that for all s, with probability one, P (k,l) I {Xi A} P X0 X s r k,l(s) = P (k,l) s (6) with arbitrary r k,l (s) R k,l (s), as j in terms of the weak convergence. To see this, let f be a bounded continuous function defined on R d +. Then we prove that f(x)p (k,l) (dx) f(x)ps (k,l) (dx) almost surely, as j. Notice that if and only if X i i is among the l NNs of s in X,..., X j+k j+ X i i s (the l-th NN of s in X,..., X j+k j+ ) s. Moreover (the l-th NN of s in X,..., X j+k j+ ) s tends to the set R k,l (s) (j ) a.s. by the ergodic theorem in context of empirical measures, thus almost uniformly by Egorov s theorem. Therefore, for arbitrary ɛ > 0 and δ > 0 an i 0 exists such that with probability δ for i i 0 the following implications hold: implies that which implies that Introduce the sets and J (k,l) J (k,l) X i i s r k,l (s) ɛ X i i is among the l NNs of s in X,..., X j+k j+, X i i s r k,l(s) + ɛ. = {i; j + k + i 0, X i i s r k,l (s) ɛ} = {i; j + k + i 0, X i i s r k,l (s) + ɛ}. Without loss of generality, assume that f 0. The ergodic theorem implies that (k,l) j f(x i J i ) E{f(X 0 )I { X lim = s r (s) ɛ}} k,l j (k,l) j J P{ X s r k,l (s) + ɛ}.

19 Nonparametric nearest neighbor based empirical portfolio selection strategies 9 a.s. and with probability δ E{f(X 0 )I { X s r k,l (s) ɛ}} P{ X s r k,l (s) + ɛ} lim inf j lim sup j lim j = j f(x i J (k,l) i ) j j j J (k,l) i J (k,l) (k,l) j J i J (k,l) j J (k,l) f(x i ) f(x i ) E{f(X 0 )I { X s r (s)+ɛ}} k,l P{ X s r k,l (s) ɛ} a.s. by ergodic theorem. ɛ 0 yields that with probability δ j f(x i J (k,l) i ) E{f(X 0 )I { X lim = s r k,l(s)} } j (k,l) j J P{ X s r k,l(s)} for arbitrary r k,l (s) R k,l (s). Thus a.s. f(x i ) lim j j i J (k,l) (k,l) j J = E{f(X 0 ) X s r k,l(s)}, and (6) is proved. Recall that by definition, b (k,l) (X j, s) is a log-optimal portfolio with respect to the probability measure P (k,l). Let b k,l (s) denote a log-optimal portfolio with respect to the limit distribution Ps (k,l). Then, using Lemma 2, we infer from (6) that, as j tends to infinity, we have the almost sure convergence lim j b (k,l) (X j, s), x 0 = b k,l (s), x 0 for Ps (k,l) -almost all x 0 and hence for P X0 -almost all x 0. Since s was arbitrary, we obtain lim j b (k,l) (X j, X ), x 0 = b k,l (X ), x 0 Next we apply Lemma for the function f i (x ) = log h (k,l) (x i ), x 0 defined on x = (..., x, x 0, x,...). Note that (a.s.) (7) = log b (k,l) (x i, x ), x 0 f i (X log ) = h (k,l) (X d i ), X 0 log X (j) 0, j=

20 20 László Györfi et al. which has finite expectation, and f i (X ) b k,l(x ), X 0 almost surely as i, by (7). As n, Lemma yields W n (H (k,l) ) = n Therefore, by (5) we have = n n f i (T i X ) i= n i= log h (k,l) (X i ), X i E { log b k,l (X ), X } 0 def = ɛ k,l (a.s.) lim inf W n(b) sup ɛ k,l sup lim inf ɛ k,l n k,l k l (a.s.) and it suffices to show that the right-hand side is at least W. The rest of the proof is similar to the end of the proof in [0], so the reader may skip it. To this end, define, for Borel sets A, B R d +, and m A (z) = P{X 0 A X = z} µ k (B) = P{X B}. Then for any s support(µ k ), and for all A, Ps (k,l) (A) = P { X 0 A X s r k,l(s) } = P{X 0 A, X s r k,l(s)} P{ X s r k,l(s)} = m A (z)µ k (dz) µ k (S s,rk,l (s)) S s,rk,l (s) m A (s) = P{X 0 A X = s} as l and for µ k -almost all s by the Lebesgue density theorem (see [9, Lemma 24.5]), and therefore as l for all A. P (k,l) X (A) P{X 0 A X }

21 Nonparametric nearest neighbor based empirical portfolio selection strategies 2 Thus, using Lemma 2 again, we have lim inf l ɛ k,l = lim ɛ k,l l = E { log b k(x ), X } 0 (where b k ( ) is the log-optimum portfolio with respect to the conditional probability P{X 0 A X }) = E { E { log b k(x ), X }} 0 X { = E max E { log b(x ), X 0 X } } b( ) def = ɛ k. To finish the proof we appeal to the sub-martingale convergence theorem. First note that the sequence def Y k = E { log b k (X ), X 0 X } = max E { log b(x ), X } 0 X b( ) of random variables forms a sub-martingale, that is, E { Y k+ X } Yk. To see this, note that } { { } = E E log b k+ (X } X X E { Y k+ X ), X 0 E { E { log b k (X ), X 0 X = E { log b k (X ), X 0 X } = Y k. This sequence is bounded by max E { log b(x ), X } 0 X b( ) } } X which has a finite expectation. The sub-martingale convergence theorem (see, e.g., Stout [4]) implies that this sub-martingale is convergent almost surely, and sup k ɛ k is finite. In particular, by the submartingale property, ɛ k is a bounded increasing sequence, so that sup ɛ k = lim k k ɛ k. Applying Lemma 3 with the σ-algebras σ ( X ) ( ) σ X yields sup ɛ k = lim k = E { E k = W max b( ) E { log b(x ), X 0 { max b( ) E { log b(x ), X 0 and the proof of the theorem is finished. X } } X } }

22 22 László Györfi et al. References. P. Algoet. Universal schemes for prediction, gambling, and portfolio selection. Annals of Probability, 20:90 94, P. Algoet. The strong law of large numbers for sequential decisions under uncertainity. IEEE Transactions on Information Theory, 40: , P. Algoet and T. Cover. Asymptotic optimality asymptotic equipartition properties of log-optimum investments. Annals of Probability, 6: , A. Blum and A. Kalai. Universal portfolios with and without transaction costs. Machine Learning, 35:93 205, A. Borodin, R. El-Yaniv, and V. Gogan. On the competitive theory and practice of portfolio selection (extended abstract). In Proc. of the 4th Latin American Symposium on Theoretical Informatics (LATIN 00), pages 73 96, Punta del Este, Uruguay, L. Breiman. The individual ergodic theorem of information theory. Annals of Mathematical Statistics, 28:809 8, 957. Correction. Annals of Mathematical Statistics, 3:809 80, L. Breiman. Optimal gambling systems for favorable games. In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, pages 65 78, Berkeley, 96. University of California Press. 8. T.M. Cover. Universal portfolios. Mathematical Finance, : 29, L. Györfi, M. Kohler, A. Krzyżak, and H. Walk. A Distribution-Free Theory of Nonparametric Regression. Springer, New York, L. Györfi, G. Lugosi, and F. Udina. Nonparametric kernel based sequential investment strategies. Mathematical Finance, 6: , L. Györfi and D. Schäfer. Nonparametric prediction. In J. A. K. Suykens, G. Horváth, S. Basu, C. Micchelli, and J. Vandevalle, editors, Advances in Learning Theory: Methods, Models and Applications, pages IOS Press, NATO Science Series, D. P. Helmbold, R. E. Schapire, Y. Singer, and M. K. Warmuth. On-line portfolio selection using multiplicative updates. Mathematical Finance, 8: , Y. Singer. Switching portfolios. International Journal of Neural Systems, 8: , W. F. Stout. Almost Sure Convergence. Academic Press, New York, 974.