Generative Models, Maximum Likelihood, Soft Clustering, and Expectation Maximization
|
|
- Agnes Carroll
- 6 years ago
- Views:
Transcription
1 Geerative Models Maximum Likelihood Soft Clusterig ad Expectatio Maximizatio Aris Aagostopoulos We will see why we desig models for data how to lear their parameters ad how by cosiderig a mixture model of Gaussias ca give us a algorithm for soft clusterig. have icluded may of the derivatios that we omitted i class for those that are iterested i the details; there are a few formulae but the uderlyig ideas are simple. Feel free to skip the calculatios of Sectio 5 if you d like but make sure you uderstad the mai ideas. 1 Modelig A importat part of uderstadig our data is that of modelig. Modelig is the desig of a mathematical usually process that creates data. A good model has a few desired properties: t creates data that have similar statistical properties with the oes observed i reality. The geeratio process seems atural ad correspodig i some sese to what actually happes i reality. t is ameable to aalysis. These properties are rather vague ad ot always possible. For istace very ofte a model to create data that are realistic it has to become complicated; this meas that it will probably be hard to aalyze it. Thus we ofte cosider models that replicate oly some of the characteristics of the uderlyig data precisely the characteristics that we are iterested i uderstadig i the give time. This is why the whole process of modelig is usually hard requires experiece ad it is ofte both a sciece ad a art. Why are we iterested i desigig models for our data? First desigig a good model ca help us makig sese of our data ad the process with which the data are created. For istace the model of Watts ad Strogatz [] eve though it is ot realistic ca help us explai the small-world pheomeo i etworks. The Barabási Albert preferetial-attachmet model [1] ca explai the power law observed i the degree distributio. Secod a good model ca help us make predictios. Assume that we have a history of data say 100 days of trasportatio iformatio. What we will ofte do is desig a model i.e. come up with a family of models which are characterized by a set of parameters ad use the largest part of the past data say 90 days to trai the model that is fid the values of of the parameters of the model. The usig the part of the data that was left out we will evaluate the model: we will see if it ca create data for the 10 days that are somehow similar to what we the real data are i these 10 days. Or we will check what probability does the model give to the real data i these 10 days. This process which is ofte iterative ad usually more complicated tha what was just described allows us to evaluate the model ad see whether the family of the models that we have chose ad the parameters that we have estimated are proper. Assumig that this is the case the we ca use the model to make predictios for the future: if the model that was traied i 90 days i the past is able to predict the last 10 days the we assume hope! that it ca predict the ext 10 days as well. This of course requires several assumptios for our real-life scearios. 1
2 Takig this oe step further we ca ow take decisios for the future. For istace if the model predicts that we will have a lot of people watig to go from Colosseum to Sa Peter s Cathedral we ca icrease the umber of buses that make this route. Fially a model ca also be used to desig tools. For istace we will see that pagerak the algorithm first used by Googleamog may other features to assig scores to web pages is actually the value obtaied i the radom surfer model: this model we cosider a user who visits pages ad follows outgoig liks radomly whereas with some probability he stops ad restarts browsig at a radom page; the the pagerak score of a give web page p is the frequecy i the log term that this user visit page p. the ext sectiowewill see someofthe abovecoceptsby cosideriga verysimple model formodelig the height of a part of the populatio. This will brig us to the questio of how we ca lear the parameters of the model usig maximum likelihood. The by geeralizig this example ad iclude heights of people from differet parts of the populatio we will itroduce a more complicated model which i the process of learig its parameters will also allow us to perform soft clusterig of the uderlyig data. A simple Gaussia model To become familiar with some of the cocepts of modelig let us assume that we observe the heights of talia wome. Let x i be the height of the ith perso. A reasoable model for such a populatio is the Gaussia model: we assume that each x i is draw by some Gaussia distributio with mea µ ad variace σ each draw beig idepedet. We ca thik that whe God decides the height of the ith perso he picks a value x i distributed accordig to a ormal distributio Nµσ. Defie the etire set of data to be the vector x x 1 x...x. Recall that the Gaussia or ormal distributio symbolized by Nµσ is the cotiuous probability distributio with rage R ad probability desity fuctio give by x µ px Nx;µσ e σ πσ where the meaig of is that we defie Nx;µσ to be the expressio o the right. The expected value of the distributio is µ ad its variace is σ. Recall that whe we draw a value from a cotiuous probability distributio a distributio whose cumulative distributio fuctio CDF is cotiuous the probability that we obtai a particular value x is 0. Thus we ca characterize the distributio by its probability desity fuctio PDF which ca give us the probability that we obtai a value i a give iterval [ab]. The probability that we obtai a value i a very small iterval δx aroud x is approximately equal to pxδx ad so usig the same approach that we use whe we defie itegrals the probability that we obtai a value i the rage [ab] is Pr x [ab] b a px dx. Nevertheless it is ofte easy to thik of px as the likelihood that we will obtai the value x ad may of the properties that hold whe we talk about probabilities of discrete radom variables hold also for PDFs. For istace if two radom variables XY with a joit PDF p XY xy are idepedet the we have that p XY xy p X xp Y y where p X x ad p Y y are the PDFs of the radom variables X ad Y respectively. We call the values µ ad σ the parameters of our model. Ofte we deote the collectio of all our parameters with the vector θ so here we have θ µσ.
3 Give the data ad the model a atural questio is what are the best set of parameters for the data that we have. This is ofte called fittig the model to the data. There are differet ways to defie best ad i the ext sectio we will see oe of the most atural ad most commoly used. 3 Maximum Likelihood Assumig that the data came from a ormal distributio Nµσ the likelihood that we observe poit x i is x i µ p θ x Nx i ;µσ e σ πσ where we have put as a subscript θ to make explicit the depedece o the parameters θ µσ. Assumig also that all the x i s are joitly idepedet we have that the likelihood that we observe the values x 1 x...x is e x i µ σ p θ x i. πσ Note that this expressio is a fuctio of the data x x 1...x ad of the parameters of the model θ µσ. Because the data are give ad because we wat to fid the best value for θ we ca see it as a fuctio of θ. We thus defie the likelihood fuctio Lθ;x as Lθ;x Lµσ ;x p θ x i. Therefore the likelihood fuctio tells us what is the likelihood that we observe the data for a give set of model parameters θ. The maximum-likelihood approach to fidig the best model is to fid the value ˆθ ˆµˆσ that maximizes Lθ;x. some sese it is the most probable model for the give data amog all the models of the family that we have selected here a Gaussia distributio. t turs out that istead of maximizig Lθ; x it is more coveiet to maximize its logarithm. Thus we defie the log-likelihood fuctio LLθ;x llθ;x ad we try to maximize this oe. Because l is a icreasigfuctio the values ˆθ that maximize LLθ;x are the values that maximize Lθ;x. our case we have: LLθ;x l e x i µ σ πσ l πσ e xi µ σ 1 lπσ +l e x i µ σ 1 lπ 1 lσ x i µ σ x i µ σ lσ lπ. 3
4 To fid where LLµσ ;x attais its maximum we set the partial derivatives with respect to µ ad σ equal to 0. The we obtai LL µ 0 or equivaletly x i µ µ σ 0 x i µ σ 0 x i µ 0 µ 1 x i ad or LL σ 0 x i µ σ σ σ lσ 0 x i µ σ σ 0 x i µ σ 0 σ 1 x i µ. These give what we expect that the best values for the model s mea ad variace ˆµ 1 ˆσ 1 x i x i µ equal the sample mea ad variace respectively. Actually to show that it is a maximum ad ot a miimum or a saddle poit we have to check that the determiat of the Hessia matrix is positive ad so o; check your multi-variable calculus books if iterested. 4
5 4 Gaussia Mixture Model ad Soft Clusterig Now we will see how usig the ideas of the previous sectio we ca soft cluster a dataset. Whe we soft cluster a dataset istead of assigig each poit to a cluster we give to each poit for every cluster a probability that it belogs to that cluster. Assume that we have a populatio of wome who are either from taly or from Chia ad whose weights are give as before by a vector x x 1 x...x. We would like to try to cluster them based o their height. We will do this by usig a more complicated model for geeratig the data which will be a mixture of Gaussia distributios. The data-geeratio process for a height x i has two steps: 1. We flip a biased coi ad with probability π we choose to have a talia woma ad with probability π C a Chiese oe.. f the coi selected a talia woma the we choose the height x i distributed as Nµ σ. f the coi selected a Chiese oe the we choose the height x i distributed as Nµ C σc. Allthechoicesaremutuallyidepedet. theabovemodelwehavesixparametersθ {π π C µ σ µ CσC } ad we have the costrait that π +π C 1 so that the coi flip at the first step be properly defied. Give this model it is atural to ask the same questio as before: what is the best value of θ? We will try to apply agai the maximum-likelihood approach. For this it is coveiet to defie the hidde or latet radom variable Z i as { if the ith perso is talia ad Z i C if the ith perso is Chiese. We have p θ x i PrZ i p θ x i Z i +PrZ i C p θ x i Z i C x i µ σ x i µ C σ C π e +π πσ C e πσ C π Nx i ;µ σ +π C Nx i ;µ C σc. As before usig the fact that the differet x i s are idepedet we have that the likelihood fuctio is Lθ;x π Nx i ;µ σ +π C Nx i ;µ C σ C ad we ca also cosider the log-likelihood fuctio LLθ;x l π Nx i ;µ σ +π C Nx i ;µ C σc l π e σ πσ x i µ x i µ C σ C +π C e πσ C. 1 This is much harder to aalyze tha before because the sum prevets the logarithm to be applied to the expoetials ad we caot obtai a closed formula. priciple we ca try to maximize it umerically ad ideed this is the approach used sometimes. the ext sectio we will itroduce a alterative method which ca be used with mixture models ad more geerally whe we have to ifer the parameters of a model that cotais latet variables. 5
6 5 Expectatio Maximizatio for Gaussia Mixtures ad Soft Clusterig As we will see we will be able to ifer the values of the parameters with a algorithm similar to k-meas. For those iterested i the details have icluded the calculatios. They are actually a bit pedatic but if you follow them slowly ot too hard. Nevertheless feel free to skip them if you wat but make sure that you uderstad the high-level idea. But first let us assume that we kew the parameter vector θ. We will ow see how usig these values we could create a soft clusterig. This meas that for every value x i we will compute probabilities PrZ i X i x i ad PrZ i C X i x i which represet the probability that the ith perso is talia ad Chiese respectively coditioal that their weight is x i. We defie the radom variable X i to be the height of the ith perso whe we create the data with our model. Note that we coditio o a evet with probability 0 which eve though we have ot defied it for discrete probability spaces is fie but eeds some techical work. Nevertheless by some had-wavig argumets we have: PrZ i X i x i a PrZ i x i δ X i x i +δ PrZ i x i δ X i x i +δ Prx i δ X i x i +δ Prx i δ X i x i +δ Z i PrZ i Prx i δ X i x i +δ Prx i δ X i x i +δ Z i PrZ i Prx i δ X i x i +δ Z i PrZ i +Prx i δ X i x i +δ Z i C PrZ i C b δ p θ x i Z i PrZ i δ p θ x i Z i PrZ i +δ p θ x i Z i C PrZ i p θ x i Z i π p θ x i Z i π +p θ x i Z i C π C where i a we approximate the probability that X i x i with the probability that X i falls i a tiy iterval of legth δ aroud x i i b we approximate this probability with δ times its PDF at the ceter of the iterval whereas i the equatios i-betwee we use the defiitio of coditioal expectatio. We are provig Bayes rule which was also what Problem 1 of Homework 1 was askig you to do. Actually by takig the limit δ 0 the above approximatios hold with equality ad so we have p θ x i Z i π γ i PrZi X i x i p θ x i Z i π +p θ x i Z i C π C π Nx i ;µ σ π Nx i ;µ σ +π C Nx i ;µ C σc because assumig that the perso is talia her height is distributed accordig to Nµ σ : x i µ σ p θ x i Z i Nx i ;µ σ e. πσ Notice that if we kow θ we ca compute exactly the probability γ i PrZ i X i x i. Likewise we ca compute π C Nx i ;µ C σ γ ic C PrZi C X i x i π Nx i ;µ σ +π C Nx i ;µ C σc 3. Therefore if we fix the parameters of the model θ we ca assig a probability for height x i to be a talia or a Chiese perso ad this assigmet iduces a soft clusterig. 6
7 Now we will look at the iverse questio: Assume that we kow the probabilities γ i ad γ ic. How ca we compute the best value θ? We will try to maximize the log-likelihood fuctio of Equatio 1 as we did i Sectio 3. First we set the partial derivative with respect to µ equal to 0: which gives 0 LL µ π Nx i ;µ σ π Nx i ;µ σ +π C Nx i ;µ C σc xi µ σ γ i xi µ σ µ γ ix i γ. 4 i Similarly we get µ C γ icx i γ. 5 ic Settig the derivative with respect to σ equal to 0 we obtai: 0 LL σ 1 π Nx i ;µ σ +π C Nx i ;µ C σc π πσ π γ i 4πσ x i µ 1 σ σ 1 σ which gives γ i π σ 4πσ 4 x i µ 1 σ 1 e x i µ σ γ i x i µ γ i σ x i µ 1 σ e x πσ i µ σ 6 π σ ad similarly σ C γ ic x i µ C γ. 7 ic Fially because of the costrait π +π C 1 to maximize LLθ;x with respect to π we will use the techique of Lagrage multipliers check your optimizatio books for details if iterested. We add the costrait to the objective multiplied by λ ad we try to maximize LLθ;x+λπ +π C 1. Settig the derivative with respect to π equal to 0 we get: 0 π LLθ;x+λπ +π C 1 Nx i ;µ σ π Nx i ;µ σ +π C Nx i ;µ C σc +λ. To compute the value λ we will use the costrait π +π C 1. Multiplyig the above by π gives λ π Nx i ;µ σ π π Nx i ;µ σ +π C Nx i ;µ C σc γ i. 8 7
8 With the same approach we have λ π C Nx i ;µ C σc π C π Nx i ;µ σ +π C Nx i ;µ C σc γ ic. 9 Summig the two costraits ad usig that π +π C 1 gives λ Nx i ;µ σ π π Nx i ;µ σ +π C Nx i ;µ C σc + Nx i ;µ C σc π C π Nx i ;µ σ +π C Nx i ;µ C σc Nx i ;µ σ π π Nx i ;µ σ +π C Nx i ;µ C σc +π Nx i ;µ C σc C π Nx i ;µ σ +π C Nx i ;µ C σc. Combiig this with Equatios 8 ad 9 we obtai ad π 1 π C 1 γ i 10 γ ic. 11 Combiig these with Equatios 4 ad 5 we obtai µ γ i x i 1 π ad µ C ad combiig them with Equatios 6 ad 7 we obtai ad γ ic x i π C 13 σ γ i x i µ π 14 σc γ ic x i µ C. 15 π C Therefore we see that if we kow the values γ i ad γ ic we ca compute the best value for θ. Note that as before we eed some more work to show that the values that we computed ideed give a local maximum but we omit the details. Recall that Equatios ad 3 tell us how we ca compute the estimates γ i ad γ ic if we kow the value of θ whereas Equatios tell us how we ca compute θ if we kow the estimates γ i ad γ ic. This suggests the algorithm of Figure 1 for computig the best θ ad γ i ad γ ic which is similar to k-meas ad is called expectatio maximizatio EM. itially we pick radom values for the parameter vector θ. Give θ we estimate the probability for each height to be of a talia or a Chiese woma γ i PrZ i X i x i ad γ ic PrZ i C X i x i ; this is the expectatio step. Give these values we proceed to the maximizatio step i which we update the parameter vector θ. We repeat util we have coverged which ca be tested for istace by checkig that the value of θ does ot chage a lot from the previous roud. At the ed we obtai our estimate 8
9 1. Fuctio Expectatio Maximizatiox. itializatio step: 3. Pick say radom reasoable values for θ {π π Cµ σµ CσC} 4. while ot coverged 5. Expectatio step: 6. π Nx i;µ σ γ i π Nx i;µ σ +πc Nxi;µCσ C Equatio 7. π C Nx i;µ Cσ γ ic C π Nx i;µ σ +πc Nxi;µCσ C Equatio 3 8. Maximizatio step: 9. π 1 γ i π C 1 γ ic Equatios µ γi xi µ C π 11. σ γi xi µ σc π 1. ed while 13. Output the fial values of γ i γ ic ad θ γic xi π C γic xi µc π C Equatios113 Equatios1415 Figure 1: The expectatio maximizatio EM algorithm for learig Gaussia mixtures. for the best parameters of the model ˆθ ad our estimates for the probabilities PrZ i X i x i ad PrZ i X i x i which iduce the soft clusterig that we wated to compute i the first place. Note that the EM algorithm resembles a lot the k-meas algorithm. The expectatio step correspods to assigig poits to clusters there we oly allow γ i ad γ ic to take the values 0 or 1; istead i the EM algorithm they ca take ay value i [01] but i both cases we have the costrait that they sum to 1. The maximizatio step correspods to the step of k-meas i which we compute the best ew ceters as the ew meas give the assigmet of the poits to clusters. deed oe ca show that the k-meas ca be viewed as a special case of the EM algorithm if we put the additioal costrait that the variaces of the two Gaussia distributios σ ad σ C are very small ted to zero. this way we will ed up with a hard clusterig by performig essetially k-meas. Similartokmeaswecashowthat theemalgorithmcovergestosomevalue whichisalocalmaximum but ot ecessarily a global oe. But whereas the k-meas algorithm usually coverges fast the EM ca be much slower. Therefore oe trick ofte used whe we wat to produce a soft clusterig is to first ru k-meas ad obtai a hard cluster ad the use this assigmets to iitialize the EM algorithm. Here we saw oe particular example of the EM algorithm that of learig Gaussia mixtures actually a special case of that. But ote that the EM algorithm is much more geeral ad ca be used whe we wat to lear models that cotai latet variables. The techical details might be more ivolved but the approach is the same. Refereces [1] A.-L. Barabási ad R. Albert. Emergece of scalig i radom etworks. Sciece 86: [] D. J. Watts ad S. H. Strogatz. Collective dyamics of small-world etworks. Nature 393:
A random variable is a variable whose value is a numerical outcome of a random phenomenon.
The Practice of Statistics, d ed ates, Moore, ad Stares Itroductio We are ofte more iterested i the umber of times a give outcome ca occur tha i the possible outcomes themselves For example, if we toss
More informationECON 5350 Class Notes Maximum Likelihood Estimation
ECON 5350 Class Notes Maximum Likelihood Estimatio 1 Maximum Likelihood Estimatio Example #1. Cosider the radom sample {X 1 = 0.5, X 2 = 2.0, X 3 = 10.0, X 4 = 1.5, X 5 = 7.0} geerated from a expoetial
More information14.30 Introduction to Statistical Methods in Economics Spring 2009
MIT OpeCourseWare http://ocwmitedu 430 Itroductio to Statistical Methods i Ecoomics Sprig 009 For iformatio about citig these materials or our Terms of Use, visit: http://ocwmitedu/terms 430 Itroductio
More informationEstimating Proportions with Confidence
Aoucemets: Discussio today is review for midterm, o credit. You may atted more tha oe discussio sectio. Brig sheets of otes ad calculator to midterm. We will provide Scatro form. Homework: (Due Wed Chapter
More informationLecture 4: Parameter Estimation and Confidence Intervals. GENOME 560 Doug Fowler, GS
Lecture 4: Parameter Estimatio ad Cofidece Itervals GENOME 560 Doug Fowler, GS (dfowler@uw.edu) 1 Review: Probability Distributios Discrete: Biomial distributio Hypergeometric distributio Poisso distributio
More informationLecture 4: Probability (continued)
Lecture 4: Probability (cotiued) Desity Curves We ve defied probabilities for discrete variables (such as coi tossig). Probabilities for cotiuous or measuremet variables also are evaluated usig relative
More informationA Bayesian perspective on estimating mean, variance, and standard-deviation from data
Brigham Youg Uiversity BYU ScholarsArchive All Faculty Publicatios 006--05 A Bayesia perspective o estimatig mea, variace, ad stadard-deviatio from data Travis E. Oliphat Follow this ad additioal works
More information5. Best Unbiased Estimators
Best Ubiased Estimators http://www.math.uah.edu/stat/poit/ubiased.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 5. Best Ubiased Estimators Basic Theory Cosider agai
More information5 Statistical Inference
5 Statistical Iferece 5.1 Trasitio from Probability Theory to Statistical Iferece 1. We have ow more or less fiished the probability sectio of the course - we ow tur attetio to statistical iferece. I statistical
More informationToday: Finish Chapter 9 (Sections 9.6 to 9.8 and 9.9 Lesson 3)
Today: Fiish Chapter 9 (Sectios 9.6 to 9.8 ad 9.9 Lesso 3) ANNOUNCEMENTS: Quiz #7 begis after class today, eds Moday at 3pm. Quiz #8 will begi ext Friday ad ed at 10am Moday (day of fial). There will be
More information. (The calculated sample mean is symbolized by x.)
Stat 40, sectio 5.4 The Cetral Limit Theorem otes by Tim Pilachowski If you have t doe it yet, go to the Stat 40 page ad dowload the hadout 5.4 supplemet Cetral Limit Theorem. The homework (both practice
More informationParametric Density Estimation: Maximum Likelihood Estimation
Parametric Desity stimatio: Maimum Likelihood stimatio C6 Today Itroductio to desity estimatio Maimum Likelihood stimatio Itroducto Bayesia Decisio Theory i previous lectures tells us how to desig a optimal
More informationConfidence Intervals. CI for a population mean (σ is known and n > 30 or the variable is normally distributed in the.
Cofidece Itervals A cofidece iterval is a iterval whose purpose is to estimate a parameter (a umber that could, i theory, be calculated from the populatio, if measuremets were available for the whole populatio).
More informationMonetary Economics: Problem Set #5 Solutions
Moetary Ecoomics oblem Set #5 Moetary Ecoomics: oblem Set #5 Solutios This problem set is marked out of 1 poits. The weight give to each part is idicated below. Please cotact me asap if you have ay questios.
More information1 Random Variables and Key Statistics
Review of Statistics 1 Radom Variables ad Key Statistics Radom Variable: A radom variable is a variable that takes o differet umerical values from a sample space determied by chace (probability distributio,
More informationThe material in this chapter is motivated by Experiment 9.
Chapter 5 Optimal Auctios The material i this chapter is motivated by Experimet 9. We wish to aalyze the decisio of a seller who sets a reserve price whe auctioig off a item to a group of bidders. We begi
More information0.1 Valuation Formula:
0. Valuatio Formula: 0.. Case of Geeral Trees: q = er S S S 3 S q = er S S 4 S 5 S 4 q 3 = er S 3 S 6 S 7 S 6 Therefore, f (3) = e r [q 3 f (7) + ( q 3 ) f (6)] f () = e r [q f (5) + ( q ) f (4)] = f ()
More informationpoint estimator a random variable (like P or X) whose values are used to estimate a population parameter
Estimatio We have oted that the pollig problem which attempts to estimate the proportio p of Successes i some populatio ad the measuremet problem which attempts to estimate the mea value µ of some quatity
More informationSampling Distributions and Estimation
Cotets 40 Samplig Distributios ad Estimatio 40.1 Samplig Distributios 40. Iterval Estimatio for the Variace 13 Learig outcomes You will lear about the distributios which are created whe a populatio is
More informationx satisfying all regularity conditions. Then
AMS570.01 Practice Midterm Exam Sprig, 018 Name: ID: Sigature: Istructio: This is a close book exam. You are allowed oe-page 8x11 formula sheet (-sided). No cellphoe or calculator or computer is allowed.
More informationUsing Math to Understand Our World Project 5 Building Up Savings And Debt
Usig Math to Uderstad Our World Project 5 Buildig Up Savigs Ad Debt Note: You will have to had i aswers to all umbered questios i the Project Descriptio See the What to Had I sheet for additioal materials
More information4.5 Generalized likelihood ratio test
4.5 Geeralized likelihood ratio test A assumptio that is used i the Athlete Biological Passport is that haemoglobi varies equally i all athletes. We wish to test this assumptio o a sample of k athletes.
More informationStandard Deviations for Normal Sampling Distributions are: For proportions For means _
Sectio 9.2 Cofidece Itervals for Proportios We will lear to use a sample to say somethig about the world at large. This process (statistical iferece) is based o our uderstadig of samplig models, ad will
More informationOverlapping Generations
Eco. 53a all 996 C. Sims. troductio Overlappig Geeratios We wat to study how asset markets allow idividuals, motivated by the eed to provide icome for their retiremet years, to fiace capital accumulatio
More informationCombining imperfect data, and an introduction to data assimilation Ross Bannister, NCEO, September 2010
Combiig imperfect data, ad a itroductio to data assimilatio Ross Baister, NCEO, September 00 rbaister@readigacuk The probability desity fuctio (PDF prob that x lies betwee x ad x + dx p (x restrictio o
More informationNotes on Expected Revenue from Auctions
Notes o Epected Reveue from Auctios Professor Bergstrom These otes spell out some of the mathematical details about first ad secod price sealed bid auctios that were discussed i Thursday s lecture You
More informationStatistics for Economics & Business
Statistics for Ecoomics & Busiess Cofidece Iterval Estimatio Learig Objectives I this chapter, you lear: To costruct ad iterpret cofidece iterval estimates for the mea ad the proportio How to determie
More informationINTERVAL GAMES. and player 2 selects 1, then player 2 would give player 1 a payoff of, 1) = 0.
INTERVAL GAMES ANTHONY MENDES Let I ad I 2 be itervals of real umbers. A iterval game is played i this way: player secretly selects x I ad player 2 secretly ad idepedetly selects y I 2. After x ad y are
More information1 Estimating sensitivities
Copyright c 27 by Karl Sigma 1 Estimatig sesitivities Whe estimatig the Greeks, such as the, the geeral problem ivolves a radom variable Y = Y (α) (such as a discouted payoff) that depeds o a parameter
More informationMath 124: Lecture for Week 10 of 17
What we will do toight 1 Lecture for of 17 David Meredith Departmet of Mathematics Sa Fracisco State Uiversity 2 3 4 April 8, 2008 5 6 II Take the midterm. At the ed aswer the followig questio: To be revealed
More informationUnbiased estimators Estimators
19 Ubiased estimators I Chapter 17 we saw that a dataset ca be modeled as a realizatio of a radom sample from a probability distributio ad that quatities of iterest correspod to features of the model distributio.
More informationChapter 8. Confidence Interval Estimation. Copyright 2015, 2012, 2009 Pearson Education, Inc. Chapter 8, Slide 1
Chapter 8 Cofidece Iterval Estimatio Copyright 2015, 2012, 2009 Pearso Educatio, Ic. Chapter 8, Slide 1 Learig Objectives I this chapter, you lear: To costruct ad iterpret cofidece iterval estimates for
More informationCHAPTER 8: CONFIDENCE INTERVAL ESTIMATES for Means and Proportions
CHAPTER 8: CONFIDENCE INTERVAL ESTIMATES for Meas ad Proportios Itroductio: I this chapter we wat to fid out the value of a parameter for a populatio. We do t kow the value of this parameter for the etire
More information1. Suppose X is a variable that follows the normal distribution with known standard deviation σ = 0.3 but unknown mean µ.
Chapter 9 Exercises Suppose X is a variable that follows the ormal distributio with kow stadard deviatio σ = 03 but ukow mea µ (a) Costruct a 95% cofidece iterval for µ if a radom sample of = 6 observatios
More informationExam 1 Spring 2015 Statistics for Applications 3/5/2015
8.443 Exam Sprig 05 Statistics for Applicatios 3/5/05. Log Normal Distributio: A radom variable X follows a Logormal(θ, σ ) distributio if l(x) follows a Normal(θ, σ ) distributio. For the ormal radom
More informationMaximum Empirical Likelihood Estimation (MELE)
Maximum Empirical Likelihood Estimatio (MELE Natha Smooha Abstract Estimatio of Stadard Liear Model - Maximum Empirical Likelihood Estimator: Combiatio of the idea of imum likelihood method of momets,
More informationChapter Four 1/15/2018. Learning Objectives. The Meaning of Interest Rates Future Value, Present Value, and Interest Rates Chapter 4, Part 1.
Chapter Four The Meaig of Iterest Rates Future Value, Preset Value, ad Iterest Rates Chapter 4, Part 1 Preview Develop uderstadig of exactly what the phrase iterest rates meas. I this chapter, we see that
More informationEXERCISE - BINOMIAL THEOREM
BINOMIAL THOEREM / EXERCISE - BINOMIAL THEOREM LEVEL I SUBJECTIVE QUESTIONS. Expad the followig expressios ad fid the umber of term i the expasio of the expressios. (a) (x + y) 99 (b) ( + a) 9 + ( a) 9
More information18.S096 Problem Set 5 Fall 2013 Volatility Modeling Due Date: 10/29/2013
18.S096 Problem Set 5 Fall 2013 Volatility Modelig Due Date: 10/29/2013 1. Sample Estimators of Diffusio Process Volatility ad Drift Let {X t } be the price of a fiacial security that follows a geometric
More informationCHAPTER 8 Estimating with Confidence
CHAPTER 8 Estimatig with Cofidece 8.2 Estimatig a Populatio Proportio The Practice of Statistics, 5th Editio Stares, Tabor, Yates, Moore Bedford Freema Worth Publishers Estimatig a Populatio Proportio
More informationBinomial Model. Stock Price Dynamics. The Key Idea Riskless Hedge
Biomial Model Stock Price Dyamics The value of a optio at maturity depeds o the price of the uderlyig stock at maturity. The value of the optio today depeds o the expected value of the optio at maturity
More informationChapter 10 - Lecture 2 The independent two sample t-test and. confidence interval
Assumptios Idepedet Samples - ukow σ 1, σ - 30 or m 30 - Upooled case Idepedet Samples - ukow σ 1, σ - 30 or m 30 - Pooled case Idepedet samples - Pooled variace - Large samples Chapter 10 - Lecture The
More informationChapter 8: Estimation of Mean & Proportion. Introduction
Chapter 8: Estimatio of Mea & Proportio 8.1 Estimatio, Poit Estimate, ad Iterval Estimate 8.2 Estimatio of a Populatio Mea: σ Kow 8.3 Estimatio of a Populatio Mea: σ Not Kow 8.4 Estimatio of a Populatio
More informationFINM6900 Finance Theory How Is Asymmetric Information Reflected in Asset Prices?
FINM6900 Fiace Theory How Is Asymmetric Iformatio Reflected i Asset Prices? February 3, 2012 Referece S. Grossma, O the Efficiecy of Competitive Stock Markets where Traders Have Diverse iformatio, Joural
More informationSTAT 135 Solutions to Homework 3: 30 points
STAT 35 Solutios to Homework 3: 30 poits Sprig 205 The objective of this Problem Set is to study the Stei Pheomeo 955. Suppose that θ θ, θ 2,..., θ cosists of ukow parameters, with 3. We wish to estimate
More informationResearch Article The Probability That a Measurement Falls within a Range of n Standard Deviations from an Estimate of the Mean
Iteratioal Scholarly Research Network ISRN Applied Mathematics Volume 0, Article ID 70806, 8 pages doi:0.540/0/70806 Research Article The Probability That a Measuremet Falls withi a Rage of Stadard Deviatios
More informationInferential Statistics and Probability a Holistic Approach. Inference Process. Inference Process. Chapter 8 Slides. Maurice Geraghty,
Iferetial Statistics ad Probability a Holistic Approach Chapter 8 Poit Estimatio ad Cofidece Itervals This Course Material by Maurice Geraghty is licesed uder a Creative Commos Attributio-ShareAlike 4.0
More informationIntroduction to Probability and Statistics Chapter 7
Itroductio to Probability ad Statistics Chapter 7 Ammar M. Sarha, asarha@mathstat.dal.ca Departmet of Mathematics ad Statistics, Dalhousie Uiversity Fall Semester 008 Chapter 7 Statistical Itervals Based
More informationAPPLICATION OF GEOMETRIC SEQUENCES AND SERIES: COMPOUND INTEREST AND ANNUITIES
APPLICATION OF GEOMETRIC SEQUENCES AND SERIES: COMPOUND INTEREST AND ANNUITIES Example: Brado s Problem Brado, who is ow sixtee, would like to be a poker champio some day. At the age of twety-oe, he would
More informationLimits of sequences. Contents 1. Introduction 2 2. Some notation for sequences The behaviour of infinite sequences 3
Limits of sequeces I this uit, we recall what is meat by a simple sequece, ad itroduce ifiite sequeces. We explai what it meas for two sequeces to be the same, ad what is meat by the -th term of a sequece.
More informationST 305: Exam 2 Fall 2014
ST 305: Exam Fall 014 By hadig i this completed exam, I state that I have either give or received assistace from aother perso durig the exam period. I have used o resources other tha the exam itself ad
More informationTopic-7. Large Sample Estimation
Topic-7 Large Sample Estimatio TYPES OF INFERENCE Ò Estimatio: É Estimatig or predictig the value of the parameter É What is (are) the most likely values of m or p? Ò Hypothesis Testig: É Decidig about
More informationii. Interval estimation:
1 Types of estimatio: i. Poit estimatio: Example (1) Cosider the sample observatios 17,3,5,1,18,6,16,10 X 8 X i i1 8 17 3 5 118 6 16 10 8 116 8 14.5 14.5 is a poit estimate for usig the estimator X ad
More informationLecture 5: Sampling Distribution
Lecture 5: Samplig Distributio Readigs: Sectios 5.5, 5.6 Itroductio Parameter: describes populatio Statistic: describes the sample; samplig variability Samplig distributio of a statistic: A probability
More informationSTRAND: FINANCE. Unit 3 Loans and Mortgages TEXT. Contents. Section. 3.1 Annual Percentage Rate (APR) 3.2 APR for Repayment of Loans
CMM Subject Support Strad: FINANCE Uit 3 Loas ad Mortgages: Text m e p STRAND: FINANCE Uit 3 Loas ad Mortgages TEXT Cotets Sectio 3.1 Aual Percetage Rate (APR) 3.2 APR for Repaymet of Loas 3.3 Credit Purchases
More informationRafa l Kulik and Marc Raimondo. University of Ottawa and University of Sydney. Supplementary material
Statistica Siica 009: Supplemet 1 L p -WAVELET REGRESSION WITH CORRELATED ERRORS AND INVERSE PROBLEMS Rafa l Kulik ad Marc Raimodo Uiversity of Ottawa ad Uiversity of Sydey Supplemetary material This ote
More informationBasic formula for confidence intervals. Formulas for estimating population variance Normal Uniform Proportion
Basic formula for the Chi-square test (Observed - Expected ) Expected Basic formula for cofidece itervals sˆ x ± Z ' Sample size adjustmet for fiite populatio (N * ) (N + - 1) Formulas for estimatig populatio
More informationSampling Distributions & Estimators
API-209 TF Sessio 2 Teddy Svoroos September 18, 2015 Samplig Distributios & Estimators I. Estimators The Importace of Samplig Radomly Three Properties of Estimators 1. Ubiased 2. Cosistet 3. Efficiet I
More informationSubject CT1 Financial Mathematics Core Technical Syllabus
Subject CT1 Fiacial Mathematics Core Techical Syllabus for the 2018 exams 1 Jue 2017 Subject CT1 Fiacial Mathematics Core Techical Aim The aim of the Fiacial Mathematics subject is to provide a groudig
More informationTwitter: @Owe134866 www.mathsfreeresourcelibrary.com Prior Kowledge Check 1) State whether each variable is qualitative or quatitative: a) Car colour Qualitative b) Miles travelled by a cyclist c) Favourite
More informationSummary. Recap. Last Lecture. .1 If you know MLE of θ, can you also know MLE of τ(θ) for any function τ?
Last Lecture Biostatistics 60 - Statistical Iferece Lecture Cramer-Rao Theorem Hyu Mi Kag February 9th, 03 If you kow MLE of, ca you also kow MLE of τ() for ay fuctio τ? What are plausible ways to compare
More information1 Estimating the uncertainty attached to a sample mean: s 2 vs.
Political Sciece 100a/200a Fall 2001 Cofidece itervals ad hypothesis testig, Part I 1 1 Estimatig the ucertaity attached to a sample mea: s 2 vs. σ 2 Recall the problem of descriptive iferece: We wat to
More informationOnline appendices from Counterparty Risk and Credit Value Adjustment a continuing challenge for global financial markets by Jon Gregory
Olie appedices from Couterparty Risk ad Credit Value Adjustmet a APPENDIX 8A: Formulas for EE, PFE ad EPE for a ormal distributio Cosider a ormal distributio with mea (expected future value) ad stadard
More informationCHAPTER 8: CONFIDENCE INTERVAL ESTIMATES for Means and Proportions
CHAPTER 8: CONFIDENCE INTERVAL ESTIMATES for Meas ad Proportios Itroductio: We wat to kow the value of a parameter for a populatio. We do t kow the value of this parameter for the etire populatio because
More informationA point estimate is the value of a statistic that estimates the value of a parameter.
Chapter 9 Estimatig the Value of a Parameter Chapter 9.1 Estimatig a Populatio Proportio Objective A : Poit Estimate A poit estimate is the value of a statistic that estimates the value of a parameter.
More informationSubject CT5 Contingencies Core Technical. Syllabus. for the 2011 Examinations. The Faculty of Actuaries and Institute of Actuaries.
Subject CT5 Cotigecies Core Techical Syllabus for the 2011 Examiatios 1 Jue 2010 The Faculty of Actuaries ad Istitute of Actuaries Aim The aim of the Cotigecies subject is to provide a groudig i the mathematical
More informationAnomaly Correction by Optimal Trading Frequency
Aomaly Correctio by Optimal Tradig Frequecy Yiqiao Yi Columbia Uiversity September 9, 206 Abstract Uder the assumptio that security prices follow radom walk, we look at price versus differet movig averages.
More informationBayes Estimator for Coefficient of Variation and Inverse Coefficient of Variation for the Normal Distribution
Iteratioal Joural of Statistics ad Systems ISSN 0973-675 Volume, Number 4 (07, pp. 7-73 Research Idia Publicatios http://www.ripublicatio.com Bayes Estimator for Coefficiet of Variatio ad Iverse Coefficiet
More informationDESCRIPTION OF MATHEMATICAL MODELS USED IN RATING ACTIVITIES
July 2014, Frakfurt am Mai. DESCRIPTION OF MATHEMATICAL MODELS USED IN RATING ACTIVITIES This documet outlies priciples ad key assumptios uderlyig the ratig models ad methodologies of Ratig-Agetur Expert
More informationInstitute of Actuaries of India Subject CT5 General Insurance, Life and Health Contingencies
Istitute of Actuaries of Idia Subject CT5 Geeral Isurace, Life ad Health Cotigecies For 2017 Examiatios Aim The aim of the Cotigecies subject is to provide a groudig i the mathematical techiques which
More informationBIOSTATS 540 Fall Estimation Page 1 of 72. Unit 6. Estimation. Use at least twelve observations in constructing a confidence interval
BIOSTATS 540 Fall 015 6. Estimatio Page 1 of 7 Uit 6. Estimatio Use at least twelve observatios i costructig a cofidece iterval - Gerald va Belle What is the mea of the blood pressures of all the studets
More informationBASIC STATISTICS ECOE 1323
BASIC STATISTICS ECOE 33 SPRING 007 FINAL EXAM NAME: ID NUMBER: INSTRUCTIONS:. Write your ame ad studet ID.. You have hours 3. This eam must be your ow work etirely. You caot talk to or share iformatio
More informationChpt 5. Discrete Probability Distributions. 5-3 Mean, Variance, Standard Deviation, and Expectation
Chpt 5 Discrete Probability Distributios 5-3 Mea, Variace, Stadard Deviatio, ad Expectatio 1/23 Homework p252 Applyig the Cocepts Exercises p253 1-19 2/23 Objective Fid the mea, variace, stadard deviatio,
More informationChapter 4 - Consumer. Household Demand and Supply. Solving the max-utility problem. Working out consumer responses. The response function
Almost essetial Cosumer: Optimisatio Chapter 4 - Cosumer Osa 2: Household ad supply Cosumer: Welfare Useful, but optioal Firm: Optimisatio Household Demad ad Supply MICROECONOMICS Priciples ad Aalysis
More informationCHAPTER 2 PRICING OF BONDS
CHAPTER 2 PRICING OF BONDS CHAPTER SUARY This chapter will focus o the time value of moey ad how to calculate the price of a bod. Whe pricig a bod it is ecessary to estimate the expected cash flows ad
More informationInsurance and Production Function Xingze Wang, Ying Hsuan Lin, and Frederick Jao (2007)
Isurace ad Productio Fuctio Xigze Wag, Yig Hsua Li, ad Frederick Jao (2007) 14.01 Priciples of Microecoomics, Fall 2007 Chia-Hui Che September 28, 2007 Lecture 10 Isurace ad Productio Fuctio Outlie 1.
More informationNon-Inferiority Logrank Tests
Chapter 706 No-Iferiority Lograk Tests Itroductio This module computes the sample size ad power for o-iferiority tests uder the assumptio of proportioal hazards. Accrual time ad follow-up time are icluded
More informationHopscotch and Explicit difference method for solving Black-Scholes PDE
Mälardale iversity Fiacial Egieerig Program Aalytical Fiace Semiar Report Hopscotch ad Explicit differece method for solvig Blac-Scholes PDE Istructor: Ja Röma Team members: A Gog HaiLog Zhao Hog Cui 0
More informationNOTES ON ESTIMATION AND CONFIDENCE INTERVALS. 1. Estimation
NOTES ON ESTIMATION AND CONFIDENCE INTERVALS MICHAEL N. KATEHAKIS 1. Estimatio Estimatio is a brach of statistics that deals with estimatig the values of parameters of a uderlyig distributio based o observed/empirical
More informationAY Term 2 Mock Examination
AY 206-7 Term 2 Mock Examiatio Date / Start Time Course Group Istructor 24 March 207 / 2 PM to 3:00 PM QF302 Ivestmet ad Fiacial Data Aalysis G Christopher Tig INSTRUCTIONS TO STUDENTS. This mock examiatio
More informationQuantitative Analysis
EduPristie www.edupristie.com Modellig Mea Variace Skewess Kurtosis Mea: X i = i Mode: Value that occurs most frequetly Media: Midpoit of data arraged i ascedig/ descedig order s Avg. of squared deviatios
More informationA New Constructive Proof of Graham's Theorem and More New Classes of Functionally Complete Functions
A New Costructive Proof of Graham's Theorem ad More New Classes of Fuctioally Complete Fuctios Azhou Yag, Ph.D. Zhu-qi Lu, Ph.D. Abstract A -valued two-variable truth fuctio is called fuctioally complete,
More informationTopic 14: Maximum Likelihood Estimation
Toic 4: November, 009 As before, we begi with a samle X = (X,, X of radom variables chose accordig to oe of a family of robabilities P θ I additio, f(x θ, x = (x,, x will be used to deote the desity fuctio
More information5 Decision Theory: Basic Concepts
5 Decisio Theory: Basic Cocepts Poit estimatio of a ukow parameter is geerally cosidered the most basic iferece problem. Speakig geerically, if θ is some ukow parameter takig values i a suitable parameter
More informationCalculation of the Annual Equivalent Rate (AER)
Appedix to Code of Coduct for the Advertisig of Iterest Bearig Accouts. (31/1/0) Calculatio of the Aual Equivalet Rate (AER) a) The most geeral case of the calculatio is the rate of iterest which, if applied
More informationLecture 9: The law of large numbers and central limit theorem
Lecture 9: The law of large umbers ad cetral limit theorem Theorem.4 Let X,X 2,... be idepedet radom variables with fiite expectatios. (i) (The SLLN). If there is a costat p [,2] such that E X i p i i=
More informationA New Approach to Obtain an Optimal Solution for the Assignment Problem
Iteratioal Joural of Sciece ad Research (IJSR) ISSN (Olie): 231-7064 Idex Copericus Value (2013): 6.14 Impact Factor (2015): 6.31 A New Approach to Obtai a Optimal Solutio for the Assigmet Problem A. Seethalakshmy
More informationChapter Four Learning Objectives Valuing Monetary Payments Now and in the Future
Chapter Four Future Value, Preset Value, ad Iterest Rates Chapter 4 Learig Objectives Develop a uderstadig of 1. Time ad the value of paymets 2. Preset value versus future value 3. Nomial versus real iterest
More informationIntroduction to Econometrics (3 rd Updated Edition) Solutions to Odd- Numbered End- of- Chapter Exercises: Chapter 2
Itroductio to Ecoometrics (3 rd Updated Editio) by James H. Stock ad Mark W. Watso Solutios to Odd- Numbered Ed- of- Chapter Exercises: Chapter 2 (This versio August 7, 204) Stock/Watso - Itroductio to
More informationExam 2. Instructor: Cynthia Rudin TA: Dimitrios Bisias. October 25, 2011
15.075 Exam 2 Istructor: Cythia Rudi TA: Dimitrios Bisias October 25, 2011 Gradig is based o demostratio of coceptual uderstadig, so you eed to show all of your work. Problem 1 You are i charge of a study
More informationMath 312, Intro. to Real Analysis: Homework #4 Solutions
Math 3, Itro. to Real Aalysis: Homework #4 Solutios Stephe G. Simpso Moday, March, 009 The assigmet cosists of Exercises 0.6, 0.8, 0.0,.,.3,.6,.0,.,. i the Ross textbook. Each problem couts 0 poits. 0.6.
More informationProblem Set 1a - Oligopoly
Advaced Idustrial Ecoomics Sprig 2014 Joha Steek 6 may 2014 Problem Set 1a - Oligopoly 1 Table of Cotets 2 Price Competitio... 3 2.1 Courot Oligopoly with Homogeous Goods ad Differet Costs... 3 2.2 Bertrad
More informationSUPPLEMENTAL MATERIAL
A SULEMENTAL MATERIAL Theorem (Expert pseudo-regret upper boud. Let us cosider a istace of the I-SG problem ad apply the FL algorithm, where each possible profile A is a expert ad receives, at roud, a
More informationCAPITAL PROJECT SCREENING AND SELECTION
CAPITAL PROJECT SCREEIG AD SELECTIO Before studyig the three measures of ivestmet attractiveess, we will review a simple method that is commoly used to scree capital ivestmets. Oe of the primary cocers
More informationFOUNDATION ACTED COURSE (FAC)
FOUNDATION ACTED COURSE (FAC) What is the Foudatio ActEd Course (FAC)? FAC is desiged to help studets improve their mathematical skills i preparatio for the Core Techical subjects. It is a referece documet
More informationAnnual compounding, revisited
Sectio 1.: No-aual compouded iterest MATH 105: Cotemporary Mathematics Uiversity of Louisville August 2, 2017 Compoudig geeralized 2 / 15 Aual compoudig, revisited The idea behid aual compoudig is that
More informationAMS Portfolio Theory and Capital Markets
AMS 69.0 - Portfolio Theory ad Capital Markets I Class 6 - Asset yamics Robert J. Frey Research Professor Stoy Brook iversity, Applied Mathematics ad Statistics frey@ams.suysb.edu http://www.ams.suysb.edu/~frey/
More informationLESSON #66 - SEQUENCES COMMON CORE ALGEBRA II
LESSON #66 - SEQUENCES COMMON CORE ALGEBRA II I Commo Core Algebra I, you studied sequeces, which are ordered lists of umbers. Sequeces are extremely importat i mathematics, both theoretical ad applied.
More informationMixed and Implicit Schemes Implicit Schemes. Exercise: Verify that ρ is unimodular: ρ = 1.
Mixed ad Implicit Schemes 3..4 The leapfrog scheme is stable for the oscillatio equatio ad ustable for the frictio equatio. The Euler forward scheme is stable for the frictio equatio but ustable for the
More information= α e ; x 0. Such a random variable is said to have an exponential distribution, with parameter α. [Here, view X as time-to-failure.
1 Homewor 1 AERE 573 Fall 018 DUE 8/9 (W) Name ***NOTE: A wor MUST be placed directly beeath the associated part of a give problem.*** PROBEM 1. (5pts) [Boo 3 rd ed. 1.1 / 4 th ed. 1.13] et ~Uiform[0,].
More information