Fitting Least squares

Transcription

1 3 Fttg lest sqres Fttg Lest sqres esd, /3/7 tos rgros e-ml: Now: pool of other techqes If we kow whch pots belog to the le, how do we fd the optml le prmeters? Lest sqres, Wht f there re otlers? Robst fttg, RNSC, LMedS, PSO Wht f there re m les? Icremetl fttg, K-les Or m cse std rems le fttg, bt most of the cocepts re er geerc d wdel pplcble Compttol Vso, Sprg 7

2 3 Fttg lest sqres Compttol Vso, Sprg 7 Sglr Vle Decompostο Sglr Vle Decompostο: Eer m mtr c be wrtte s the mltplcto of 3 other mtrces DV : mm mtr, D: m mtr, V: mtr he sglr les σ re determed b σ σ σ Colms of : left sglr ectors of M Rows of V: rght sglr ectors of M mm m m m m m m m σ σ σ V 3 Sglr Vle Decomposto Illstrto of SVD dmesos 4 D D

3 3 Fttg lest sqres SVD: tto. d V re orthoorml bses. Rotto, sclg, rotto D c be regrded s sclg mtr, d d V c be ewed s rotto mtrces. 3. Sglr les s sems of ellpse or ellpsod he sglr les c be terpreted s the semes of ellpse -D. C be geerlzed to -dmesol Eclde spce 5 Emple: mge mde of 4 qe row ectors. 6 Compttol Vso, Sprg 7 3

4 3 Fttg lest sqres Emple: Wht hppes f I tke the st sglr ector? Notce tht ech row of pels s the sme... jst dfferet 'brghtess' Essetll, ech row c ow be wrtte s s Rc SV where c s the scle fctor d SV s the sglr ector wth the hghest sglr le bggest cotrbto to dt 7 Emple: Wht hppes f I tke the frst sglr ectors? Now, ech row c be wrtte s the sm of two ectors Rc SV+b SV 8 Compttol Vso, Sprg 7 4

5 3 Fttg lest sqres Emple: Wht hppes f I tke the frst te sglr ectors? 9 Emple: Wht hppes f I tke the 5 sglr ectors? Compttol Vso, Sprg 7 5

6 3 Fttg lest sqres SVD: propertes. Sglrt s osglr ff ll sglr les re o zero. Rk of mtr Rk mber of ozero sglr les 3. Ierse of mtr Defed f s osglr I geerl, we defe the psedoerse of 4. Egeles d egeectors he egeles of d re σ σ > he colms of re egeectors of mm he rows of V re egeectors of DV VD + VD σ σ SVD: pplcto Lest sqres solto of ler sstem b Sole ler sstem of m eqtos wth kows m > : m mtr b m-dmesol ector Solto + b b mtr Psedoerse Psedoerse of s compted wth SVD + s er lkel to be eql to - f m > Check codto mber of Compttol Vso, Sprg 7 6

7 3 Fttg lest sqres SVD: pplcto Homogeeos ler sstem m eqtos, kows Rk - check the SVD of Oe o trl solto p to scle fctor c be obted b SVD: Proportol to the egeector of tht correspods to the zero sglr le I prctce, the egeector tht correspods to the σ wth the smllest le 3 SVD: pplcto 3 Estmto of mtr whose les re depedet. Errors de to ose:  s the estmto, Α the ctl mtr Impose costrts throgh SVD E.g., orthogolt of mtr Fd the «closest» mtr to Â, whch stsfes ectl the costrt SVD of  Note: IfΑ s orthogol, the D I the ll sglr les re eql to Solto: Sbsttte the sglr les wth the epected oes DV ˆ IV 4 Compttol Vso, Sprg 7 7

8 3 Fttg lest sqres Now: pool of other techqes If we kow whch pots belog to the le, how do we fd the optml le prmeters? Lest sqres, Wht f there re otlers? Robst fttg, RNSC, LMedS, PSO Wht f there re m les? Icremetl fttg, K-les Or m cse std rems le fttg, bt most of the cocepts re er geerc d wdel pplcble 5 Lest sqres le fttg Dt:,,,, Le eqto: m + b Fd m, b to mmze E m b, m+b E [ ] de dp p m b + p p p m b p Mtlb: p \ ; p p Modfed from S. Lzebk Compttol Vso, Sprg 7 8

9 3 Fttg lest sqres Problem wth ertcl lest sqres Not rotto-rt wh? Fls completel for ertcl les 7 otl lest sqres Dstce betwee pot, d le cosθ+sθd s cosθ+ sθ d Fd cosθ, sθ, d to mmze the sm of sqred perpedclr dstces E cosθ+sθd t orml: Ncosθ, sθ, + b d 8 Compttol Vso, Sprg 7 9

10 3 Fttg lest sqres Compttol Vso, Sprg 7 otl lest sqres Dstce betwee pot, d le +bd +b : + b d Fd, b, d to mmze the sm of sqred perpedclr dstces + d b E, +bd + d b E t orml: N, b + d b d E b d b + + N N b b E + N dn de Solto to N, sbject to N : egeector of ssocted wth the smllest egele lest sqres solto to homogeeos ler sstem 9 otl lest sqres secod momet mtr

11 3 Fttg lest sqres Compttol Vso, Sprg 7 otl lest sqres, N, b secod momet mtr, Recp: wo Commo Optmzto Problems Problem sttemet Solto s.t. mmze mmze lsq solto to trl - o.. : eg ], [ < λ λ λ Problem sttemet Solto sqres solto to b lest b \ mmze b b mtlb

12 3 Fttg lest sqres Lest sqres: Robstess to ose Lest sqres ft to the red pots: 3 Lest sqres: Robstess to ose Lest sqres ft wth otler: Problem: sqred error hel ffected b otlers 4 Compttol Vso, Sprg 7

13 3 Fttg lest sqres Robst lest sqres to del wth otlers Geerl pproch: mmze, θ σ ρ ;, θ resdl of th pot w.r.t. model prmeters θ ρ robst fcto wth scle prmeter σ m b he robst fcto ρ Fors cofgrto wth smll resdls Costt pelt for lrge resdls Slde from S. Srese Itertel re-weghted lest sqres 6 Compttol Vso, Sprg 7 3

14 3 Fttg lest sqres Choosg the scle: Jst rght he effect of the otler s elmted 7 Choosg the scle: oo smll he error le s lmost the sme for eer pot d the ft s er poor 8 Compttol Vso, Sprg 7 4

15 3 Fttg lest sqres Choosg the scle: oo lrge Behes mch the sme s lest sqres 9 Compttol Vso, Sprg 7 5