S yi a bx i cx yi a bx i cx 2 i =0. yi a bx i cx 2 i xi =0. yi a bx i cx 2 i x

Transcription

1 LEAST-SQUARES FIT (Chapter 8) Ft the best straght lne (parabola, etc.) to a gven set of ponts. Ths wll be done by mnmzng the sum of squares of the vertcal dstances (called resduals) from the ponts to the lne. Normal Equatons Let us assume that we want to ft aparabola, whose general equaton s y = a + bx+ cx 2. Mnmze (by varyng a, b and c): S y a bx cx 2 2 The three partal dervatves,.e. S a, S S b and c, must be equal to zero: 2 y a bx c =0 2 2 y a bx c x =0 y a bx c x 2 =0 Cancellng 2 and reorganzng yelds:

2 a a an+ b x + b + b x + c + c + c or (n matrx form): n x x For straght lne: n x x 4 x = = x 4 = a b = c a b = x x y x

3 For a cubc: n x x x 4 x 4 x 5 x 4 x 5 x 6 a b c d = x y etc. The smallest value of S, say S mn, s (gong back to the parabola case): y 2 abc x y Ths can serve to defnethesocalledtypcalerror: r Smn n 3 where 3 s the number of coeffcents n the ftted parabola (2 for a straght lne, 4 for a cubc, etc.).

4 Example: Ft a parabola to the followng data: x: y: Normal equatons: Wesolvethesebythesocalled Gaussan Elmnaton followed by backward substtuton. There are four elementary operatons whch can be used to modfy a lnear set of equatons wthout changng ts soluton. These are: 1. Addng (subtractng) a multple of a row to (from) any other row. 2. Interchangng two rows (.e. equatons). 3. Multplyng a row (equaton) by a non-zero number. 4. Interchangng two columns of the coeffcent matrx (wehavetokeeptrackofthese). Proceedng n a systematc way, wth the help of the frst two elementary operatons we can make all elements below the man dagonal equal to zero. It s

5 done n ths order: Howdowedealwthzeroon the man dagonal? Example: (Our normal equatons). Subtract the frst row multpled by 3 (11) from the second (thrd) row: Subtract the second row multpled by 6 from the thrd row: Startng from the bottom, we now fnd the unknowns by backward substtuton: c = 3 14 b = = a = = 7 5

6 (verfy!). s Typcal error of the ft: = Symmetrc Data When x-values are spaced symmetrcally around acenter(sayˆx), the computaton smplfes. Usng thesameexample,weft y =â + ˆb(x ˆx)+ĉ(x ˆx) 2 where bx = 3. The normal equatons now read n (x ˆx) (x ˆx) 2 â (x ˆx) (x ˆx) 2 (x ˆx) 3 ˆb (x ˆx) 2 (x ˆx) 3 (x ˆx) 4 ĉ = (x ˆx) (x ˆx) 2 But. due to symmetry, the sum of any odd power of x ˆx yelds 0:

7 n 0 (x ˆx) 2 0 (x ˆx) 2 0 (x ˆx) 2 0 (x ˆx) 4 = (x ˆx) (x ˆx) 2 whch means we can separate the odd coeffcents n (x ˆx) 2 (x ˆx) 2 (x ˆx) 4 from even ˆb (x ˆx) 2 = â ĉ = â ˆb ĉ (x ˆx) 2 (x ˆx) Example: Usngtheolddataandˆx =3yelds

8 Equatons for â and ĉ: solved by x ˆx: y: â ĉ = = The ˆb equaton s trval: 10ˆb = 3 ˆb = The best fttngparabolas: y = (x 3) 3 (x 3)2 14 (the same as before, when expanded). It also yelds the same typcal error: s = Weghted Ft Some of the x-y pars may be repeated. We canthensmplfythetablebyaddnganewrowof weghts:

9 x: y: w: Now, we mnmze S w y a bx c (each squared resdual multpled by weght). Normal equatons: w w x w w x w w x 3 a w b = w x w w w x 4 c w solved as before. The symmetry trck wll work now onlf the w cooperate. S mn s now computed by 2

10 w y 2 abc Typcal error: v u t w w x w y n Smn n 3 w Example: We ft the best straght lne to the prevous data. Normal equatons: Soluton: e = y = x

11 Typcal s error: = Lnear Models We can replace x, x 2,... of our polynomal model by any other specfc functons of x (e.g. 1 x,ex, ln x, etc.), y = af 1 (x)+bf 2 (x)+cf 3 (x) n general. Normal equatons: f 1 (x ) 2 f 1 (x )f 2 (x ) f 1 (x )f 3 (x ) f 2 (x )f 1 (x ) f 2 (x ) 2 f 2 (x )f 3 (x ) f 3 (x )f 1 (x ) f 3 (x )f 2 (x ) f 3 (x ) 2 f 1 (x ) = f 2 (x ) f 3 (x ) Ths tme, symmetrs of no help. a b c

12 Example: Let us ft our old data (wthout weghts) by y = a x + b + cx3.e. f 1 (x) = 1 x,f 2(x) =1and f 3 (x) =x 3. Normal equatons: 1 1 x 1 x 1 or, numercally x 6 x Gaussan (forward) elmnaton results n back substtuton yelds: c = and a = ,b=

13 s Typcal error: = We could easly modfy ths procedure to ncorporate weghts. Our last example s a clear ndcaton that exact computaton (usng fractons) s gettng rather tedous (or even mpossble), and that we should consder swtchng to decmals. Ths necesstates modfyng the Gaussan-elmnaton procedure, to keep the round off errors of the computaton n check..