Density estimation III

Transcription

1 CS 750 Mace Learg Lecure 7 Desy esmao III Mlos Hauskrec mlos@.edu 539 Seo Square Eoeal famly Eoeal famly: all robably mass / desy fucos a ca be wre e eoeal ormal form f e a vecor of aural or caocal arameers a fuco referred o as a suffce sasc a fuco of s less mora a ormalzao cosa a aro fuco e d Oer commo form: f e

2 Eoeal famly: eamles Beroull dsrbuo Eoeal famly Parameers e???? e f e e Eoeal famly: eamles Beroull dsrbuo Eoeal famly Parameers fuco e e e f e e

3 3 Eoeal famly: eamles Uvarae Gaussa dsrbuo Eoeal famly Parameers e e???? e f ] e[, Eoeal famly: eamles Uvarae Gaussa dsrbuo Eoeal famly Parameers e e / / / 4 e e e f ] e[,

4 4 Eoeal famly For d samles, e lkelood of daa s Imora: e dmesoaly of e suffce sasc remas e same w e umber of samles e D P e e Eoeal famly e lkelood of daa s Omzg e lkelood For e ML esmae mus old e, D l 0, D l

5 5 Eoeal famly Rewrg e grade: Eoeal famly Rewrg e grade: Resul: For e ML esmae e arameers sould be adjused suc a e eecao of e sasc s equal o e observed samle sascs d e d d e e d e E E

6 Momes of e dsrbuo For e eoeal famly e k- mome of e sasc corresods o e k- dervave of If s a comoe of e we ge e momes of e dsrbuo by dffereag s corresodg aural arameer Eamle: Beroull e e Dervaves: e e e e e No-aramerc desy esmao CS 750 Mace Learg 6

7 Noaramerc Desy Esmao Paramerc dsrbuo models are: resrced o secfc fucoal forms, wc may o always be suable; Eamle: modelg a mulmodal dsrbuo w a sgle, umodal model. vs Noaramerc aroaces: Do o make ay srog assumo abou e overall sae of e dsrbuo beg modelled. Noaramerc Meods Hsogram meods: aro e daa sace o dsc bs w wds ad cou e umber of observaos,, eac b. N Ofe, e same wd s used for all bs, =. acs as a smoog arameer. Bg does o work well e a d-dmesoal sace, 7

8 Noaramerc Meods Bg does o work well e a d-dmesoal sace, M bs eac dmeso wll requre M d bs! Soluo: Buld e esmaes of by cosderg e daa os D ad ow smlar or close ey are o Eamle: Parze wdow s f we buld a b dyamcally for for wc we eed Noaramerc Meods ssume observaos draw from a desy ad cosder a small rego R coag suc a P d R P R e robably a ou of N observaos le sde R s B,N,P ad f N s large NP P R If e volume of R, V, s suffcely small, s aromaely cosa over R ad P V us P R P V Pug gs ogeer we ge: NV 8

9 9 Noaramerc meods: kerel meods Soluo : Esmae e robably for based o e fed volume V bul aroud F V, esmae from e daa Eamle: Parze wdow NV Noaramerc meods: kerel meods erel Desy Esmao: Parze wdow: Le R be a yercube cered o a defes e kerel fuco: I follows a ad ece N k N D k N NV oerwse D k 0, / /

10 Noaramerc Meods: smoo kerels o avod dscoues because of sar boudares we ca use a smoo kerel, e.g. a Gaussa N N e D / y kerel suc a k u 0 wll work. k u du acs as a smooer. Noaramerc Meods: knn esmao Soluo : Esmae e robably for based o a fed cou for a varable volume V bul aroud f, esmae V from e daa Neares Negbour Desy Esmao: Cosder a yer-sere cered o ad le grow o a volume, V*, a cludes of e gve N daa os. e acs as a smooer 0

11 Noaramerc vs Paramerc Meods Noaramerc models: More flebly o desy model s eeded Bu requre sorg e ere daase ad e comuao s erformed w all daa eamles. Paramerc models: Oce fed, oly arameers eed o be sored ey are muc more effce erms of comuao Bu e model eeds o be cked advace