Mixture Models Simulation-based Estimation

Transcription

1 Mixture Models Simulation-based Estimation p. 1/72 Mixture Models Simulation-based Estimation Michel Bierlaire Transport and Mobility Laboratory

2 Mixture Models Simulation-based Estimation p. 2/72 Outline Mixtures Capturing correlation Alternative specific variance Taste heterogeneity Latent classes Simulation-based estimation

3 Mixture Models Simulation-based Estimation p. 3/72 Mixtures In statistics, a mixture probability distribution function is a convex combination of other probability distribution functions. If f(ε,θ) is a distribution function, and if w(θ) is a non negative function such that w(θ)dθ = 1 then θ g(ε) = θ w(θ)f(ε, θ)dθ is also a distribution function. We say that g is a w-mixture of f. If f is a logit model, g is a continuous w-mixture of logit If f is a MEV model, g is a continuous w-mixture of MEV

4 Mixture Models Simulation-based Estimation p. 4/72 Mixtures Discrete mixtures are also possible. If w i, i = 1,...,n are non negative weights such that then g(ε) = n w i = 1 i=1 n w i f(ε,θ i ) i=1 is also a distribution function where θ i, i = 1,...,n are parameters. We say that g is a discrete w-mixture of f.

5 Mixture Models Simulation-based Estimation p. 5/72 Example: discrete mixture of normal distributions 2.5 N(5,0.16) N(8,1) 0.6 N(5,0.16) N(8,1)

6 Mixture Models Simulation-based Estimation p. 6/72 Example: discrete mixture of binary logit models P(1 s=1,x) P(1 s=2,x) 0. 4 P(1 s=1,x) P(1 s=2,x)

7 Mixture Models Simulation-based Estimation p. 7/72 Mixtures General motivation: generate flexible distributional forms For discrete choice: correlation across alternatives alternative specific variances taste heterogeneity...

8 Mixture Models Simulation-based Estimation p. 8/72 Back to the telephone example Budget measured: U BM = α BM + βx BM + ε BM Standard measured: U SM = α SM + βx SM + ε SM Local flat: U LF = α LF + βx LF + ε LF Extended area flat: U EF = α EF + βx EF + ε EF Metro area flat: U MF = βx MF + ε MF Distributions for ε: logit, probit, nested logit

9 Mixture Models Simulation-based Estimation p. 9/72 Back to the telephone example Covariance of U Logit σ σ σ σ σ 2 π 2 6µ 2 Probit σ 2 BM σ BM,SM σ BM,LF σ BM,EF σ BM,MF σ BM,SM σ 2 SM σ SM,LF σ SM,EF σ SM,MF σ BM,LF σ SM,LF σ 2 LF σ LF,EF σ LF,MF σ BM,EF σ SM,EF σ LF,EF σ 2 EF σ EF,MF σ BM,MF σ SM,MF σ LF,MF σ EF,MF σ 2 MF Nested logit 1 ρ M ρ M ρ F ρ F 0 0 ρ F 1 ρ F 0 0 ρ F ρ F 1, ρ i = 1 µ2 µ 2 i

10 Mixture Models Simulation-based Estimation p. 10/72 Continuous Mixtures of logit Combining probit and logit Error decomposed into two parts U in = V in +ξ +ν Normal distribution (probit): flexibility i.i.d EV (logit): tractability

11 Mixture Models Simulation-based Estimation p. 11/72 Logit Utility: ν i.i.d. extreme value Probability: U auto = βx auto + ν auto U bus = βx bus + ν bus U subway = βx subway + ν subway Pr(auto X) = e βx auto e βx auto +e βx bus +e βx subway

12 Mixture Models Simulation-based Estimation p. 12/72 Normal mixture of logit Utility: U auto = βx auto + ξ auto + ν auto U bus = βx bus + ξ bus + ν bus U subway = βx subway + ξ subway + ν subway ν i.i.d. extreme value, ξ N(0,Σ) Probability: Pr(auto X, ξ) = e βx auto+ξ auto e βx auto+ξ auto +e βx bus +ξ bus +e βx subway +ξ subway P(auto X) = ξ Pr(auto X, ξ)f(ξ)dξ

13 Mixture Models Simulation-based Estimation p. 13/72 Simulation P(auto X) = ξ Pr(auto X, ξ)f(ξ)dξ Integral has no closed form. Monte Carlo simulation must be used.

14 Mixture Models Simulation-based Estimation p. 14/72 Simulation In order to approximate P(i X) = ξ Pr(i X, ξ)f(ξ)dξ Draw from f(ξ) to obtain r 1,..., r R Compute P(i X) P(i X) = 1 R = 1 R R P(i X,r k ) k=1 R k=1 e V 1n+r k e V 1n+r k +e V 2n +r k +e V 3n

15 Mixture Models Simulation-based Estimation p. 15/72 Capturing correlations: nesting Utility: U auto = βx auto + ν auto U bus = βx bus + σ transit η transit + ν bus U subway = βx subway + σ transit η transit + ν subway ν i.i.d. extreme value, η transit N(0,1), σ 2 transit =cov(bus,subway) Probability: Pr(auto X,η transit ) = P(auto X) = e βx auto e βx auto +e βx bus +σ transit η transit +e βx subway+σ transit η transit η Pr(auto X, η)f(η)dη

16 Mixture Models Simulation-based Estimation p. 16/72 Nesting structure Example: residential telephone ASC_BM ASC_SM ASC_LF ASC_EF BETA_C σ M σ F BM ln(cost(bm)) η M 0 SM ln(cost(sm)) η M 0 LF ln(cost(lf)) 0 η F EF ln(cost(ef)) 0 η F MF ln(cost(mf)) 0 η F

17 Mixture Models Simulation-based Estimation p. 17/72 Nesting structure Identification issues: If there are two nests, only one σ is identified If there are more than two nests, all σ s are identified Walker (2001) Results with 5000 draws..

18 NL NML NML NML NML σ F = 0 σ M = 0 σ F = σ M L Value Scaled Value Scaled Value Scaled Value Scaled Value Scaled ASC BM ASC EF ASC LF ASC SM LOGCOST FLAT MEAS σ F σ M F + σ2 M

19 Mixture Models Simulation-based Estimation p. 18/72 Comments The scale of the parameters is different between NL and the mixture model Normalization can be performed in several ways σ F = 0 σ M = 0 σ F = σ M Final log likelihood should be the same But... estimation relies on simulation Only an approximation of the log likelihood is available Final log likelihood with draws: Unnormalized: σ M = σ F : σ F = 0: σ M = 0:

20 Mixture Models Simulation-based Estimation p. 19/72 Cross nesting 5 5Nest 1 5Nest Bus Train Car Ped. Bike U bus = V bus +ξ 1 +ε bus U train = V train +ξ 1 +ε train U car = V car +ξ 1 +ξ 2 +ε car U ped = V ped +ξ 2 +ε ped U bike = V bike +ξ 2 +ε bike P(car) = P(car ξ 1,ξ 2 )f(ξ 1 )f(ξ 2 )dξ 2 dξ 1 ξ 2 ξ 1

21 Mixture Models Simulation-based Estimation p. 20/72 Identification issue Not all parameters can be identified For logit, one ASC has to be constrained to zero Identification of NML is important and tricky See Walker, Ben-Akiva & Bolduc (2007) for a detailed analysis

22 Mixture Models Simulation-based Estimation p. 21/72 Alternative specific variance Error terms in logit are i.i.d. and, in particular, have the same variance U in = β T x in + ASC i +ε in ε in i.i.d. extreme value Var(ε in ) = π 2 /6µ 2 In order allow for different variances, we use mixtures U in = β T x in + ASC i +σ i ξ i +ε in where ξ i N(0,1) Variance: Var(σ i ξ i +ε in ) = σ 2 i + π2 6µ 2

23 Mixture Models Simulation-based Estimation p. 22/72 Alternative specific variance Identification issue: Not all σs are identified One of them must be constrained to zero Not necessarily the one associated with the ASC constrained to zero In theory, the smallest σ must be constrained to zero In practice, we don t know a priori which one it is Solution: 1. Estimate a model with a full set of σs 2. Identify the smallest one and constrain it to zero.

24 Mixture Models Simulation-based Estimation p. 23/72 Alternative specific variance Example with Swissmetro ASC_CAR ASC_SBB ASC_SM B_COST B_FR B_TIME Car cost 0 time Train cost freq. time Swissmetro cost freq. time + alternative specific variance

25 Logit ASV ASV norm. L Value Scaled Value Scaled Value Scaled ASC CAR ASC SM B COST B FR B TIME SIGMA CAR SIGMA TRAIN SIGMA SM

26 Mixture Models Simulation-based Estimation p. 24/72 Identification issue: process Examine the variance-covariance matrix 1. Specify the model of interest 2. Take the differences in utilities 3. Apply the order condition: necessary condition 4. Apply the rank condition: sufficient condition 5. Apply the equality condition: verify equivalence

27 Mixture Models Simulation-based Estimation p. 25/72 Heteroscedastic: specification U 1 = βx 1 +σ 1 ξ 1 +ε 1 U 2 = βx 2 +σ 2 ξ 2 +ε 2 U 3 = βx 3 +σ 3 ξ 3 +ε 3 U 4 = βx 4 +σ 4 ξ 4 +ε 4 where ξ i N(0,1), ε i EV(0,µ) Cov(U) = σ1 2 +γ/µ σ2 2 +γ/µ σ3 2 +γ/µ σ4 2 +γ/µ2

28 Mixture Models Simulation-based Estimation p. 26/72 Heteroscedastic: differences U 1 U 4 = β(x 1 x 4 )+(σ 1 ξ 1 σ 4 ξ 4 )+(ε 1 ε 4 ) U 2 U 4 = β(x 2 x 4 )+(σ 2 ξ 2 σ 4 ξ 4 )+(ε 2 ε 4 ) U 3 U 4 = β(x 3 x 4 )+(σ 3 ξ 3 σ 4 ξ 4 )+(ε 3 ε 4 ) Cov( U) = σ1 2 +σ2 4 +2γ/µ2 σ4 2 +γ/µ2 σ4 2 +γ/µ2 σ4 2 +γ/µ2 σ2 2 +σ2 4 +2γ/µ2 σ4 2 +γ/µ2 σ4 2 +γ/µ2 σ4 2 +γ/µ2 σ3 2 +σ2 4 +2γ/µ2

29 Mixture Models Simulation-based Estimation p. 27/72 Heteroscedastic: order condition S is the number of estimable parameters J is the number of alternatives S J(J 1) 2 1 It represents the number of entries in the lower part of the (symmetric) var-cov matrix minus 1 for the scale J = 4 implies S 5

30 Mixture Models Simulation-based Estimation p. 28/72 Heteroscedastic: rank condition Idea Number of estimable parameters = number of linearly independent equations -1 for the scale Cov( U) = σ 2 1 +σ2 4 +2γ/µ2 σ 2 4 +γ/µ2 σ 2 2 +σ2 4 +2γ/µ2 σ 2 4 +γ/µ2 σ 2 4 +γ/µ2 σ 2 3 +σ2 4 +2γ/µ2 dependent scale

31 Mixture Models Simulation-based Estimation p. 29/72 Heteroscedastic: rank condition Three parameters out of five can be estimated Formally Identify unique elements of Cov( U) 2. Compute the Jacobian wrt σ 2 1, σ 2 2, σ 2 3, σ 2 4, γ/µ 2 3. Compute the rank σ 2 1 +σ2 4 +2γ/µ2 σ 2 2 +σ2 4 +2γ/µ2 σ 2 3 +σ2 4 +2γ/µ2 σ 2 4 +γ/µ S = Rank - 1 = 3

32 Mixture Models Simulation-based Estimation p. 30/72 Heteroscedastic: equality condition 1. We know how many parameters can be identified 2. There are infinitely many normalizations 3. The normalized model is equivalent to the original one 4. Obvious normalizations, like constraining extra-parameters to 0 or another constant, may not be valid

33 Mixture Models Simulation-based Estimation p. 31/72 Heteroscedastic: equality condition U n = β T x n + L n ξ n + ε n Cov(U n ) = L n L T n + (γ/µ 2 )I Cov( j U n ) = j L n L T n T j + (γ/µ 2 ) j T j Notations: 2 = ( ) Cov( j U n ) = Ω n = Σ n + Γ n Ω norm n = Σ norm n + Γ norm n

34 Mixture Models Simulation-based Estimation p. 32/72 Heteroscedastic: equality condition The following conditions must hold: Covariance matrices must be equal Ω n = Ω norm n Σ norm n must be positive semi-definite

35 Mixture Models Simulation-based Estimation p. 33/72 Heteroscedastic: equality condition Example with 3 alternatives: U 1 = βx 1 +σ 1 ξ 1 +ε 1 U 2 = βx 2 +σ 2 ξ 2 +ε 2 U 3 = βx 3 +σ 3 ξ 3 +ε 3 Cov( 3 U) = Ω = ( σ 2 1 +σ2 3 +2γ/µ2 σ 2 3 +γ/µ2 σ 2 2 +σ2 3 +2γ/µ2 ) Parameters: {σ 1,σ 2,σ 3,µ} Rank condition: S = 2 µ is used for the scale

36 Mixture Models Simulation-based Estimation p. 34/72 Heteroscedastic: equality condition Denote ν i = σ 2 i µ2 (scaled parameters) Normalization condition: ν 3 = K Ω = ( (ν 1 +ν 3 +2γ)/µ 2 (ν 3 +γ)/µ 2 (ν 2 +ν 3 +2γ)/µ 2 ) ( ) Ω norm = (ν N 1 +K +2γ)/µ2 N (K +γ)/µ 2 N (ν N 2 +K +2γ)/µ2 N where index N stands for normalized

37 Mixture Models Simulation-based Estimation p. 35/72 Heteroscedastic: equality condition First equality condition: Ω = Ω norm (ν 3 +γ)/µ 2 = (K +γ)/µ 2 N (ν 1 +ν 3 +2γ)/µ 2 = (ν N 1 +K +2γ)/µ2 N (ν 2 +ν 3 +2γ)/µ 2 = (ν N 2 +K +2γ)/µ2 N that is, writing the normalized parameters as functions of others, µ 2 N = µ2 (K +γ)/(ν 3 +γ) ν1 N = (K +γ)(ν 1 +ν 3 +2γ)/(ν 3 +γ) K 2γ = (K +γ)(ν 2 +ν 3 +2γ)/(ν 3 +γ) K 2γ ν N 2

38 Mixture Models Simulation-based Estimation p. 36/72 Heteroscedastic: equality condition Second equality condition: Σ norm = 1 µ 2 N ν N ν N K must be positive semi-definite, that is µ N > 0, ν N 1 0, ν N 2 0, K 0. Putting everything together, we obtain K (ν 3 ν i )γ ν i +γ, i = 1,2

39 Mixture Models Simulation-based Estimation p. 37/72 Heteroscedastic: equality condition K (ν 3 ν i )γ ν i +γ, i = 1,2 If ν 3 ν i, i = 1,2, then the rhs is negative, and any K 0 would do. Typically, K = 0. If not, K must be chosen large enough In practice, always select the alternative with minimum variance.

40 Mixture Models Simulation-based Estimation p. 38/72 Taste heterogeneity Population is heterogeneous Taste heterogeneity is captured by segmentation Deterministic segmentation is desirable but not always possible Distribution of a parameter in the population

41 Mixture Models Simulation-based Estimation p. 39/72 Random parameters U i U j = β t T i +β c C i +ε i = β t T j +β c C j +ε j Let β t N( β t,σ 2 t), or, equivalently, β t = β t +σ t ξ, with ξ N(0,1). U i U j = β t T i +σ t ξt i +β c C i +ε i = β t T j +σ t ξt j +β c C j +ε j If ε i and ε j are i.i.d. EV and ξ is given, we have P(i ξ) = e β t T i +σ t ξt i +β c C i, and e β t T i +σ t ξt i +β c C i +e βt T j +σ t ξt j +β c C j P(i) = P(i ξ)f(ξ)dξ. ξ

42 Mixture Models Simulation-based Estimation p. 40/72 Random parameters Example with Swissmetro ASC_CAR ASC_SBB ASC_SM B_COST B_FR B_TIME Car cost 0 time Train cost freq. time Swissmetro cost freq. time B_TIME randomly distributed across the population, normal distribution

43 Mixture Models Simulation-based Estimation p. 41/72 Random parameters Logit RC L ASC_CAR_SP ASC_SM_SP B_COST B_FR B_TIME S_TIME Prob(B_TIME 0) 8.8% χ

44 Mixture Models Simulation-based Estimation p. 42/72 Random parameters 25 Distribution of B_TIME

45 Mixture Models Simulation-based Estimation p. 43/72 Random parameters Example with Swissmetro ASC_CAR ASC_SBB ASC_SM B_COST B_FR B_TIME Car cost 0 time Train cost freq. time Swissmetro cost freq. time B_TIME randomly distributed across the population, log normal distribution

46 Mixture Models Simulation-based Estimation p. 44/72 Random parameters [Utilities] 11 SBB_SP TRAIN_AV_SP ASC_SBB_SP * one + B_COST * TRAIN_COST + B_FR * TRAIN_FR 21 SM_SP SM_AV ASC_SM_SP * one + B_COST * SM_COST + B_FR * SM_FR 31 Car_SP CAR_AV_SP ASC_CAR_SP * one + B_COST * CAR_CO [GeneralizedUtilities] 11 - exp( B_TIME [ S_TIME ] ) * TRAIN_TT 21 - exp( B_TIME [ S_TIME ] ) * SM_TT 31 - exp( B_TIME [ S_TIME ] ) * CAR_TT

47 Mixture Models Simulation-based Estimation p. 45/72 Random parameters Logit RC-norm. RC-logn ASC_CAR_SP ASC_SM_SP B_COST B_FR B_TIME S_TIME Prob(β > 0) 8.8% 0.0% χ

48 Mixture Models Simulation-based Estimation p. 46/72 Random parameters 40 Lognormal Distribution of B_TIME Normal Distribution of B_TIME Lognormal mean Normal mean MNL

49 Mixture Models Simulation-based Estimation p. 47/72 Random parameters Example with Swissmetro ASC_CAR ASC_SBB ASC_SM B_COST B_FR B_TIME Car cost 0 time Train cost freq. time Swissmetro cost freq. time B_TIME randomly distributed across the population, discrete distribution P(β time = ˆβ) = ω 1 P(β time = 0) = ω 2 = 1 ω 1

50 Mixture Models Simulation-based Estimation p. 48/72 Random parameters [DiscreteDistributions] B_TIME < B_TIME_1 ( W1 ) B_TIME_2 ( W2 ) > [LinearConstraints] W1 + W2 = 1.0

51 Mixture Models Simulation-based Estimation p. 49/72 Random parameters Logit RC-norm. RC-logn. RC-disc ASC_CAR_SP ASC_SM_SP B_COST B_FR B_TIME S_TIME W W Prob(β > 0) 8.8% 0.0% 0.0% χ

52 Mixture Models Simulation-based Estimation p. 50/72 Latent classes Latent classes capture unobserved heterogeneity They can represent different: Choice sets Decision protocols Tastes Model structures etc.

53 Mixture Models Simulation-based Estimation p. 51/72 Latent classes P(i) = S Pr(i s)q(s) s=1 Pr(i s) is the class-specific choice model probability of choosing i given that the individual belongs to class s Q(s) is the class membership model probability of belonging to class s

54 Mixture Models Simulation-based Estimation p. 52/72 Summary Logit mixtures models Computationally more complex than MEV Allow for more flexibility than MEV Continuous mixtures: alternative specific variance, nesting structures, random parameters P(i) = ξ Pr(i ξ)f(ξ)dξ Discrete mixtures: well-defined latent classes of decision makers S P(i) = Pr(i s)q(s). s=1

55 Mixture Models Simulation-based Estimation p. 53/72 Tips for applications Be careful: simulation can mask specification and identification issues Do not forget about the systematic portion

56 Mixture Models Simulation-based Estimation p. 54/72 Simulation P(i) = ξ Pr(i ξ)f(ξ)dξ No closed form formula Randomly draw numbers such that their frequency matches the density f(ξ) Let ξ 1,...,ξ R be these numbers The choice model can be approximated by P(i) 1 R R Pr(i r), as r=1 lim R 1 R R Pr(i r) = r=1 ξ Pr(i ξ)f(ξ)dξ

57 Mixture Models Simulation-based Estimation p. 55/72 Simulation P(i) 1 R R Pr(i r). r=1 The kernel is a logit model, easy to compute. Pr(i r) = e V 1n+r e V 1n+r +e V 2n+r +e V 3n Therefore, it amounts to generating the appropriate draws.

58 Mixture Models Simulation-based Estimation p. 56/72 Appendix: Simulation Pseudo-random numbers generators Although deterministically generated, numbers exhibit the properties of random draws Uniform distribution Standard normal distribution Transformation of standard normal Inverse CDF Multivariate normal

59 Mixture Models Simulation-based Estimation p. 57/72 Appendix: Simulation: uniform distribution Almost all programming languages provide generators for a uniform U(0,1) If r is a draw from a U(0,1), then is a draw from a U(a,b) s = (b a)r+a

60 Mixture Models Simulation-based Estimation p. 58/72 Appendix: Simulation: standard normal If r 1 and r 2 are independent draws from U(0,1), then s 1 = 2lnr 1 sin(2πr 2 ) s 2 = 2lnr 1 cos(2πr 2 ) are independent draws from N(0,1)

61 Mixture Models Simulation-based Estimation p. 59/72 Appendix: Simulation: standard normal Histogram of 100 random samples from a univariate Gaussian PDF with unit variance and zero mean scaled bin frequency Gaussian p.d.f

62 Mixture Models Simulation-based Estimation p. 60/72 Appendix: Simulation: standard normal Histogram of 500 random samples from a univariate Gaussian PDF with unit variance and zero mean scaled bin frequency Gaussian p.d.f

63 Mixture Models Simulation-based Estimation p. 61/72 Appendix: Simulation: standard normal Histogram of 1000 random samples from a univariate Gaussian PDF with unit variance and zero mean scaled bin frequency Gaussian p.d.f

64 Mixture Models Simulation-based Estimation p. 62/72 Appendix: Simulation: standard normal Histogram of 5000 random samples from a univariate Gaussian PDF with unit variance and zero mean scaled bin frequency Gaussian p.d.f

65 Mixture Models Simulation-based Estimation p. 63/72 Appendix: Simulation: standard normal Histogram of random samples from a univariate Gaussian PDF with unit variance and zero mean scaled bin frequency Gaussian p.d.f

66 Mixture Models Simulation-based Estimation p. 64/72 Appendix: Simulation: transformations of standard no If r is a draw from N(0,1), then s = br +a is a draw from N(a,b 2 ) If r is a draw from N(a,b 2 ), then is a draw from a log normal LN(a,b 2 ) with mean e r e a+(b2 /2) and variance e 2a+b2 (e b2 1)

67 Mixture Models Simulation-based Estimation p. 65/72 Appendix: Simulation: inverse CDF Consider a univariate r.v. with CDF F(ε) If F is invertible and if r is a draw from U(0,1), then s = F 1 (r) is a draw from the given r.v. Example: EV with F(ε) = e e ε F 1 (r) = ln( lnr)

68 Mixture Models Simulation-based Estimation p. 66/72 Appendix: Simulation: inverse CDF 1 CDF of the Extreme Value distribution

69 Mixture Models Simulation-based Estimation p. 67/72 Appendix: Simulation: multivariate normal If r 1,...,r n are independent draws from N(0,1), and r = r 1. r n then s = a+lr is a vector of draws from the n-variate normal N(a,LL T ), where L is lower triangular, and LL T is the Cholesky factorization of the variance-covariance matrix

70 Mixture Models Simulation-based Estimation p. 68/72 Appendix: Simulation: multivariate normal Example: L = l l 21 l 22 0 l 31 l 32 l 33 s 1 = l 11 r 1 s 2 = l 21 r 1 + l 22 r 2 s 3 = l 31 r 1 + l 32 r 2 + l 33 r 3

71 Mixture Models Simulation-based Estimation p. 69/72 Appendix: Simulation for mixtures of logit In order to approximate P(i) = ξ Pr(i ξ)f(ξ)dξ Draw from f(ξ) to obtain r 1,..., r R Compute P(i) P(i) = 1 R = 1 R R Pr(i r k ) k=1 R k=1 e V 1n+r k e V 1n+r k +e V 2n +r k +e V 3n

72 Mixture Models Simulation-based Estimation p. 70/72 Appendix: Maximum simulated likelihood max θ L(θ) = ( N J ) y jn ln P(j;θ) n=1 j=1 where y jn = 1 if ind. n has chosen alt. j, 0 otherwise. Vector of parameters θ contains: usual (fixed) parameters of the choice model parameters of the density of the random parameters For instance, if β j N(µ j,σ 2 j ), µ j and σ j are parameters to be estimated

73 Mixture Models Simulation-based Estimation p. 71/72 Appendix: Maximum simulated likelihood Warning: P(j;θ) is an unbiased estimator of P(j;θ) E[ P n (j;θ)] = P(j;θ) ln P(j;θ) is not an unbiased estimator of lnp(j;θ) lne[ P(j;θ] E[ln P(j;θ)] Under some conditions, it is a consistent (asymptotically unbiased) estimator, so that many draws are necessary.

74 Mixture Models Simulation-based Estimation p. 72/72 Appendix: Maximum simulated likelihood Properties of MSL: If R is fixed, MSL is inconsistent If R rises at any rate with N, MSL is consistent If R rises faster than N, MSL is asymptotically equivalent to ML.