Chapter 2 Simple Regression - An Overview. Father. Father ... Student N o 1 Student N ' s Student N Father

Transcription

1 Chaper mple Regressn - An Overvew Ths lecure s an nrducn he sascal mehdlgy called smple lnear regressn Imagne ha yu have w varables, lke yur hegh and yur faher's hegh rmally we wuld expec ha a all faher wuld que lkely gve hs ra hs ffsprng Tha means ha yu wll prbably be all als One way hnk f hs s wre u as very smple lnear equans, applcable all adul sudens, as fllws (where denes hegh) = β + β + ε uden uden' s uden aher = β + β + ε uden uden ' s uden aher = β + β + ε uden uden ' s uden aher e hw ha he ceffcens β and β apply all sudens, and n jus uden There are equans abve because we are cnsderng a sample f sudens And, here are w unknwn parameers - β and β In addn, here are randm errr erms gven by he ε ' s These errr erms are hw we allw randmness ener he mdel We wll make sme frmal assumpns abu hese errr erms laer Rgh nw jus magne hey are randm, ndependen f each her, ndependen f he hegh f yur faher, and have a zer expecan The heghs n he lef and rgh sdes f each equan are bservable, whereas everyhng else, ncludng he ε ' s are unknwn and unknwable The frmulan abve s n very cmpac Le's wre he mdel agan usng smpler nan and a sngle represenave equan, gven by = β + β + ε fr =,,,

2 where we have clearly renamed he same varables ha were n he rgnal equans abve w, le * represen he average hegh f an adul If we subrac hs ff frm bh sdes f ur represenave equan, we can elmnae β and ge ( ) = β ( ) + ε fr =,,, * * The lef-hand sde erm represens he excess f he h suden's hegh ver he adul persn's average hegh The rgh-hand sde explanary erm represens he excess f he h faher's hegh ver he average adul hegh w, suppse ha 0 < β < I shuld be clear ha f * ( ) 0 > hen he erm * ( ) 0 >, bu by n as much Als, f * ( ) 0 < hen he erm <, bu by n as much, n abslue value erms Obvusly he mvemen s * ( ) 0 wards he mean Bgger han average fahers wll prduce bgger han average chldren, bu he excess shrnks beween generans The same happens fr smaller han average fahers Ther chldren are smaller han average, bu wll neverheless be clser he average han her fahers Ths reversn wards he mean ccurs n average I s n a deermnsc phenmena I s a randm phenmena, and mus ccur f 0 < β < In 886, rancs Galn publshed an academc paper descrbng regressn wards he mean The argumen he made s smlar he ne I have made here Ths paper s where we ge he erm "regressn" Thus, yu shuld realze ha regressn has been arund fr que sme me Yu are n learnng smehng new; see regressn wrds he mean Cnsder agan he smple regressn mdel The lef-hand sde varable, = β + β + ε fr =,,, s called he dependen varable I measures he hegh f he h suden n he sample f sudens On he rgh-hand sde, he varable refers he hegh f he h suden's faher Ths varable s called eher he ndependen varable r he explanary varable I s explanng he mvemen f The fnal erm n he rgh hand Ths represenan requres ha he mean r expecan f he each f he ' s ε s equal zer Galn, (886) "Regressn wards medcry n heredary saure" The Jurnal f he Anhrplgcal Insue f Grea Bran and Ireland 5: 46 6

3 sde, ε, s called he errr erm and s assumed be a randm varable, usually dsrbued as a nrmal randm varable havng mean = 0 and varance equal σ We call hs mdel he smple regressn mdel because here s nly OE explanary varable If here s mre han ne explanary varable we call he mdel a mulvarae regressn mdel Wha s knwn frm he use Of curse we have daa n suden heghs and faher's heghs, and fr =,,, We can arrange he daa n a able as fllws Table : Table f Daa n eghs We call hs smple regressn "a regressn f n " Tha s, a regressn mdel s always a regressn f he dependen varable n he ndependen varables r example, a mre general ype f mdel can be wren as Y = α + βx + ε fr =,,,, In hs case, we wuld say ha we have a regressn f Y n X The cnsan nercep s equal α whereas he cnsan slpe ceffcen s equal β The dependen varable s Y and he explanary varable s X The randm errr erm s ε and has he assumed prperes f E[ ε ] = 0 and var( ε ) = σ The number f bservans n ur sample f daa s And, fnally, ur gal s use he Y and X daa esmae he values f αβ,, and ε fr =,,,, Our daa can be lsed n a able lke he ne abve

4 Table : Generc Table f Daa fr mple Regressn Y X Y X Y X Y X Y X The gal f regressn s esmae he unknwn parameers and ε ' s usng ur bserved daa n Y and X We als can es hypheses abu hese unknwn parameers and errr erms, whch s fen wha ccurs n scenfc sudes mple regressn has a dependen varable and OE ndependen r explanary varable Bu, hw d we knw whch shuld be he ndependen varable and whch shuld be he dependen varable? ere sascs s slen The nly way yu can ell whch s whch s by relyng upn yur hery; n sascal hery, bu yur wn subjec's hery (e ecnmcs, psychlgy, plcal scence, ec) We fen hear sad ha regressn (r crrelan) cann shw causan I can nly shw wheher r n here s a srng lnear relan beween he varables A srng crrelan beween w varables can never ell us f he w varables are n fac relaed n a causave way, n ur smple regressn mdel abve, we cann clam ha X causes Y unless we have a gd hery backng such a clam up There s a deeper prblem here, hugh Yu cann le he daa gude yur hery and hen es yur hery usng he same daa Tha s, suppse yu have daa n Y and X Yu fnd here s a clse f f he regressn and a srng relan emerges beween hese w varables Yu feel elaed Yu may have fund smehng mpran Ths leads yu frmulae a hery abu he relan beween X and Y Yu creae yur bg hery and hen use he same daa se es he hery Bng! Yur hery wrks and s upheld by very rgrus sascal ess Well, yu have made a huge errr Yu cann es yur I nce heard Lawrence ummers (ecnms and frmer Presden f arvard Unversy) nce humrusly clam ha any subjec ha has he wrd "scence" n s name s n a real scence

5 hery n he rgnal daa se ha led yu yur hery Insead, yu mus g u and ge EW daa es yur hery, Oherwse yu have sample selecn bas Yur sample becmes based wards accepng yur hery snce yu g yur hery frm he same daa se w, here s nhng wrng wh geng a gd guess r hery abu he wrld frm daa Tha s dne all he me The ruble cmes when yu use he same daa es he hery memes we can spl he daa n w pars The frs par gves us he hery, whle he secnd par can be used es he hery In all cases, he hery when beng esed mus be frm new daa, n daa we have seen and whch was used help us cnsruc ur hery Why n jus ge ls f daa and see f he varables are relaed by runnng regressns? hng wrng wh hs, f yur gal s merely see wha varables "seem" be relaed Tha s all yu can say abu hs If yu fnd a srng relan, may be jus a sascal arfac A sascal arfac s where yu fnd srucure r paerns n he daa ha seems be real and ndependen f yurself, bu such srucure s acually nly creaed by yu These paerns seem have an ndependen exsence apar frm he researcher I s as hugh yu have fund a gld mne n he daa; smehng ha exss ndependenly f wh s searchng fr gld Bu, n fac he srucure s nly n yur mnd Ths s lke seeng paerns n he cluds (a dg, a bear, he slluee f a gddess) and asserng hese have deep and lasng meanng, nly see hem dsspae n chac mvemens f he waer vapr The paerns yu see n he daa may ndeed have srucure and meanng, bu nly new daa shwng he paerns wll help cnvnce us hs s s