House-Hunting Without Second Moments

Transcription

1 House-Hunting Without Second Moments Thomas S. Ferguson, University of California, Los Angeles Michael J. Klass, University of California, Berkeley Abstract: In the house-hunting problem, i.i.d. random variables, X,X 2,... are observed sequentially at a cost of c>0 per observation. The problem is to choose a stopping rule, N, to maximize E(X N Nc). If the X s have a finite second moment, the optimal stopping rule is N =min{n :X n >V },wherev satisfies E(X V ) + = c. The statement of the problem and its solution requires only the first moment of the X n to be finite. Is a finite second moment really needed? In 970, Herbert Robbins showed, assuming only a finite first moment, that the rule N is optimal within the class of stopping rules, N, such that E(X N Nc) >, but it is not clear that this restriction of the class of stopping rules is really required. In this paper it is shown that this restriction is needed, but that if the expectation is replaced by a generalized expectation, N is optimal out of all stopping rules assuming only first moments. AMS 2000 Subject Classification: 60G40; 90A80. Key Words: Optimal Stopping; Selling an Asset; Job Search.. Introduction. The house-hunting problem, also called the problem of selling an asset or the job search problem, was introduced and solved almost simultaneously in papers by MacQueen and Miller (960), Derman and Sacks (960), Chow and Robbins (96) and Sakaguchi (96). This problem may be described as follows. Independent, identically distributed random variables, X,X 2,... with common distribution function, F (x), are observed sequentially at a cost of c per observation, where c>0. We always assume that the expectation of the positive part of X is finite : EX + <, wherex has distribution F (x). You must take at least one observation. If you stop after n observations,you receive X n as a payoff, so your net return is X n nc. If you never stop, your payoff is defined to be since X n nc a.s. as n when EX + <. The problem is to choose a stopping rule, N, to maximize E(X N Nc). It is customary to assume F (x) has a finite second moment, or more generally that E{(X + )2 } <. Under this assumption, the stopping rule N =min{n :X n >V } () maximizes E(X n nc) among all stopping rules, where V satisfies (x V ) df (x) =c. (2) V Partially supported by NSF Grant DMS

2 In addition, V =E(X N N c) is also the optimal expected return. The statement of the problem and its solution requires only the first moment of F (x) to be finite. In particular, the stopping rule () still gives expected return V. Yet the proofs of optimality of N seem to require that the second moment of F (x) be finite. Is a finite second moment really needed? By an elegant direct argument based on Wald s equation and using only the assumption that EX + <, Robbins (970) shows that the rule N given by () with V given by (2) is still optimal. However, he uses a slightly different definition of optimality, namely, he defines N to be optimal if it maximizes E(X N Nc) within the class of stopping rules, N, such that E(X N Nc) >. This seems innocuous enough. Who would like to accept a random reward the expectation of whose negative part is? The trouble is that this restriction also excludes payoffs the expectation of whose positive part is +. If E(X N Nc) = and E(X N Nc) + <, then you don t need to exclude N. The rule N is definitely better. So a slightly stronger definition of optimality would be to restrict consideration to stopping rules N such that E(X N Nc) + <. This looks more questionable. Why should one exclude rules with infinite positive expectation? Is a finite second moment necessary for N to be optimal out of all stopping rules? 2. Necessity of E{(X + ) 2 } <. Robbins result certainly provides an extension of the optimal property of the rule N that is valid even if E(X + ) 2 =. However, there are difficulties of interpretation that arise because of the restriction to stopping rules that satisfy E(X N Nc) + <. Restricting attention to such rules seems to say that any rule, N, withe X N Nc <, no matter how bad, is better than a rule whose expected payoff does not exist because E(X N Nc) = and E(X N Nc) + =+. For what W do you prefer a gamble giving you a payoff of $W to a gamble giving you $Z, wherez is chosen from a standard Cauchy distribution? Answering such questions seems to require an extension to standard utility theory. One important question that arises is whether or not there are some distributions of X with E(X + ) 2 = for which all stopping rules have E(X N Nc) + <. Then, at least for some distributions with infinite second moment one could say that N is optimal among all stopping rules. It will be shown that for all distributions F with E(X + ) 2 =, there exist stopping rules, N, such that E(X N Nc) = and E(X N Nc) + =+. Theorem. If X, X,X 2,... are i.i.d. with EX + < and E(X + ) 2 =, then the stopping rule, N =min{n :X n 2nc} (3) satisfies E(X N Nc) + =. Thus no new examples of optimality within the class of all stopping rules may be found using Robbins result. Proofs are deferred to the appendix. 3. A Stronger Result. As it stands, Theorem would not impress Robbins. Robbins requires that stopping rules stop with probability one. This contrasts with others 2

3 that allow stopping rules, N, such that P(N = ) > 0 (see for example the electronic text, Ferguson (2006)). This allows treatment of more general problems, for example bandit problems, but it requires specifying what the payoff will be if N =. Generally, one may force the decision maker to use rules, N, that stop with probability one by choosing the payoff to be if N =. However in this problem, the restriction to stopping rules that stop with probability one is very reasonable. All Theorem really says is that if E(X + ) 2 is infinite, a prophet can get an infinite expected return. He simply stops at N if there exists an n such that X n > 2nc and stops at n = otherwise. It seems that those of us without superpowers must be satisfied with V or risk not stopping and so receiving infinite loss. Therefore to satisfy Robbins, we need to answer the question: For what distributions of X with finite first moment and E(X + ) 2 = is it true that there exists stopping rules N that stop with probability one and for which E(X N Nc) + =? The answer is contained in the following theorem. Theorem 2. If X, X,X 2,... are i.i.d. with EX + < and E(X + ) 2 =, then there exists a stopping rule, N, withp(n < ) =such that E(X N Nc) + =, for all c<. Thus there are no distributions with infinite positive second moment for which we may dispense with Robbins restriction to stopping rules, N, such that E(X N Nc) + <. It is interesting to note that the stopping rule, N, of Theorem 2 does not depend on c. The proof is constructive. In addition, the stopping rule has the simple form, N =min{n : X n >a n } for some sequence of constants, a n. 4. Optimality of N among all stopping rules. To extend Robbins result to make it valid for all stopping rules, we must therefore find some way to compare two payoff distributions whose first moments don t exist. Certainly if given the choice between two Cauchy distributions with the same interquartile range, we would prefer the one with the higher median. More generally, we would prefer F to G if F stochastically dominates G (i.e. if F (x) < G(x) for all x). There are many ways to extend this idea further. One sufficient for the problem at hand is the following. We say that a lottery from a distribution G ispreferredtoa0payoffiffori.i.d. Z,Z 2,... from G we have (/n) n Z a.s. i γ for some 0 <γ. For a distribution G with finite mean, this just means that a lottery from G is preferred to 0 if the mean of G is positive. For a distribution G whose mean does not exist, it can still happen that (/n) n Z a.s. i +, in which case we prefer G to 0. This can happen if the mass on the positive axis dominates the mass on the negative axis, even though the expectation of the positive and negative parts are both infinite. Similarly, we prefer 0 to a lottery from G if for i.i.d. Z,Z 2,... from G we have (/n) n Z a.s. i. More generally, we prefer a lottery for G to a lottery for G 2 if for 3

4 i.i.d. Y,Y 2,...from G and independent i.i.d. Z,Z 2,... from G 2,wehave(/n) n (Y i Z i ) a.s. γ for some 0 <γ. Using this extension of the definition of preference between lotteries, one can show that Robbins result is true without restricting the stopping rules one considers. In the paper of Robbins and Samuel (966), an extension of the definition of mathematical expectation is given which is useful in this context. Definition. For a random variable X, we say that the extended expectation of X exists and is equal to µ, insymbolsêx = µ, if n n X i µ a.s. (5) i= when X,X 2,... are i.i.d. with the same distribution as X. If EX exists, then ÊX =EX, including the case where EX = ±. However, if EX does not exist, it still may happen that (/n) n X i converges almost surely to + or. Thus,Ê is indeed an extension of the notion of expectation. Using this notion, we can state the optimality of the stopping rule () for the house hunting problem. Theorem 3. In the house hunting problem with finite first moment, the stopping rule N of () is optimal in the sense that for all stopping rules N, Ê(X N Nc) V. In other words, if E(X + ) <, then for any stopping rule, N, eitherx N Nc has finite expectation less than or equal to V, or its extended expectation is. 5. A Near Counterexample. If the first moment of X barely exists in the sense that EX + < and EX + log + (X + )=, then there is a stopping rule that looks as if it might be a counterexample to Theorem 3. Theorem 4. If EX + < and EX + log + (X + )=, then there exists a stopping rule of the form N =min{n :X n >a n } for some sequence a n such that P(N < ) =, and E{(X n nc)i(n = n)} = P(N >n )E{(X nc)i(x>a n )} = (6) for all c>0. Note that, again, N does not depend on c. To see how curious this result is, examine the second sum in (6). The stopping rule N stops with probability, and when it stops at stage n, the conditional payoff given N = n 4

5 is simply E{(X nc)i(x>a n )}, a fairly large positive number, even though stopping may occur at negative values of X n nc. This has to be multiplied by the probability of reaching that stage which is P(N > n ) = n i= F (a i). The product of this and E{(X nc)i(x>a n )} is the summand of the second sum, and is a fairly small number. Nevertheless, if you add up all these small numbers, you get +. Doesn t that seem better than receiving V as the payoff? The catch is, of course, that this summation is not equal to E(X N Nc), which doesn t exist. This is an example where the expectation of the sum is not the sum of the expectations. Worse, the sum of the expectations is + while the generalized expectation of the sum is ; in other words, if you take a sample from the distribution of X N Nc, the average of the sample will tend to, a.s. APPENDIX Proof of Theorem. We show that the rule N =min{n : X n 2nc} gives E(X N Nc) + = when EX + < and E(X + ) 2 =. E(X N Nc) + = = E(X n nc)i(n = n) ncp(n = n) ncp(n >n )P(X n > 2nc) ncp(n = )P(X n > 2nc) =, since E(X + ) 2 = implies np(x n > 2nc) =, and P(N = ) =P(X n < 2nc for all n) = P(X n < 2nc) = ( P(X n 2nc)) exp{ P(X n > 2nc)} exp{ EX + /2c} > 0. (6) (7) Proof of Theorem 2. Without loss of generality, assume that X>0andthatX has a continuous distribution function. (Otherwise, only stop when X>0 and replace the distribution of X with the distribution of XU where U has a uniform(0,) distribution independent of X.) Since E(X + ) 2 =, wehave k= k P(X >y) dy =. (8) 5

6 Let E k = k P(X >y) dy and Q k = E + E E k, (9) for k>0. Then, k E k = and Q k.itisalsotruethat k= E k Q k =. (0) See, for example, Rudin (976), Problem (b) page 79. Let n be the smallest k such that Q k >. Let a n be defined by P(X a n )= Q n () and for k>n,leta k satisfy P(X a k )= Q k Q k. (2) Let N =min{k n : X k >a k }. Notice that for n>n. P(N >n)=p( n k=n {X k a k }) = n Q k = 0. Q n Q k Q n k=n + (3) Hence N stops with probability one. (This may also be seen using k>n + P(X >a k)= k>n + ( (Q k /Q k )) = k>n + (E k/q k )= ; sop(x n >a n i.o.) =.) Notice that for any <c<, P(X >a k ) P(X >ck) = In particular, Q k <ckfrom some point on. E k (c )kp(x >ck) = Q k P(X >ck) Q k P(X >ck) (c )k Q k. (4) Fix any c>. There exists n c such that Q k ck for all k n c. Therefore, E(X N Nc) + = k=n c ck P(X >y) dyp(n >k ) = E ck > E j =. Q ck 2c Q k=n c j=cn j c E ck Q k k=n c (5) 6

7 Proof of Theorem 3. We use the cute idea, cited in the paper of Robbins (970) as due to David Burdick, entailed in the inequality X n nc = v +(X n v) nc v +(X n v) + nc n v + (X i v) + nc = v + = v + n ((X i v) + c) n W i (6) where W i =(X i v) + c. Now choose v to be any number greater than V. The W i are i.i.d. with expectation EW i < 0sincev>V. Let N be an arbitrary stopping rule. We show below that Ê N W i < 0; this implies that Ê(X N Nc) <vand, since v is an arbitrary number greater than V,thatÊ(X N Nc) V. We now use the idea of Blackwell (946) in his proof of Wald s Equation. Consider n stopping problems as follows. Let N be the stopping rule N applied to the sequence W,W 2,...,letN 2 be the stopping rule N applied to W N +,W N +2,...,etc.,andletN n be the stopping rule N applied to W N + +N n +, W N + +N n +2,... Let the returns for these problems be denoted by Z,...,Z n where Z j = W N + +N j W N + +N j. Then, the Z j are independent with the same distribution as N W i, and we have Z + + Z n n = W + + W N + +N n N + + N n N + + N n. (7) n From the strong law of large numbers, the first term on the right of (7), (W + + W N + +N n )/(N + + N n ), converges a.s. to EW i < 0. The second term on the right of (7), N + + N n )/n converges a.s. to EN, whetheren is finite or +. Therefore, the left side of (7) converges a.s. to EW i EN<0. This shows that Ê N W i < 0aswas to be shown. Proof of Theorem 4. Again assume without loss of generality that X>0andthatX has a strictly increasing continuous distribution function on (0, ). Let ϕ(b) =E(X X >b)= E(X I(X >b)). P(X >b) Then ϕ(b) is increasing and continuous and the inverse function, b(y) =ϕ (y) is welldefined by ϕ(b(y)) = E(X X >b(y)) = y (8) 7

8 for y E(X). Clearly, b(y) is increasing, lim y b(y) =, andb(y) <y.then P(X >b(n)) = = E{X I(X >b(n))} n n ( )E{X I(b(n) <X b(n +))} j j= E{X log(ϕ(x)/2)i(b(n) < X b(n +))} (9) Therefore, using ϕ(y) >y, P(X >b(n)) E{X log(x/2) I(X >b())} =. (20) There exist constants, γ k, increasing to infinity such that P(X >b(kγ k )) =. (2) k= Let a k = b(kγ k ). Notice that as in the first line of (9), There exists a k such that P(X >a k )= E{X I(X >a k)} kγ k = o(/k). (22) P(X >a k ) < 2k for all k k. (23) Let N =min{k k : X k >a k }. Then from (23) and Borel-Cantelli, P(N < ) =. Now fix any c>andchoosek c so that a k >b(2ck) for all k k c. Notice that for k>k, P(N >k)=p(x j a j for k j k) k = ( P(X >a j )) j=k k 2j 2j j=k > 2k 2k 2k (24) 8

9 Putting these inequalities together, P(N >n )E{(X cn)i(x>a n )} n=k c = P(N >n )E((X cn)i(x>b(nγ n ))) (def. of a n ) n=k c = P(N >n )[nγ n cn]p(x >b(nγ n )) (from (8)) n=k c > (γ n c)p(x >b(nγ n )) (from (24)) 2 n=k c = (since γ n and (2)) References. Blackwell, D. (946). On an equation of Wald, Annals of Mathematical Statistics 7: Chow, Y. S., and Robbins, H. (96). A martingale system theorem and applications, Proceedings of the Fourth Berkeley Symposium on Mathathematical Statistics and Probability, : Derman, C., and Sacks, J. (960). Replacement of periodically inspected equipment (an optimal stopping rule), Naval Research Logistics Quarterly 7: Ferguson, T. S. (2006). Optimal Stopping and Applications, Electronic Text at MacQueen, J., and Miller, R. G., Jr. (960). Optimal persistence policies, Operations Research 8: Robbins, H. (970). Optimal stopping, Americam Mathematical Monthly, 77: Robbins, H., and Samuels, E. (966). An extension of a lemma of Wald, Journal of Applied Probability 3: Rudin, W. (976). Principles of Mathematical Analysis, 3rd Edition, McGraw-Hill. Sakaguchi M. (96). Dynamic programming of some sequential sampling design, Journal of Mathematical Analysis and Applications 2: