! Homework. ! Exploring the data " Quantitative data: e.g., reading times. ! Inferential statistics " Parametric tests

Transcription

1 Overview! Homework Introution to Psyholinguistis Leture 6 Experimentl Methos II Pi Knoeferle & M. W. Croker Deprtment of Computtionl Linguistis Srln University SS 2006! Exploring the t " Quntittive t: e.g., reing times! Brgrphs of mens & onfiene intervls! Boxplots! Histogrms: Skew n kurtosis! Testing for normlity n homogeneity of vrine! Inferentil sttistis " Prmetri tests! Compring two mens: t-test! Compring more thn two mens: F-sttisti " An exmple from the eye-trking literture P. Knoeferle 2 Homework Homework! Design n experiment " Theory1: There is proessing preferene (e.g., sujet-first) for oth miguous n unmiguous sentenes " Theory 2: Suh preferene exists only for miguous sentenes! Opertionliztion, hypotheses, esign + exmple sentenes, n lists (only the onition oing per list); metho! How mny ftors?! Assume 24 items! How mny t points per onition for 1 prtiipnt?! Type of t n nlysis?! Opertionliztion " If informtion lter in the sentene (e.g., NP2) ismigutes sentene-initil miguous NP, we shoul oserve proessing iffiulty! Hypothesis0: Suh iffiulty shoul e oserve for oth initilly struturlly miguous n unmiguous sentenes! Hypothesis 1: Suh iffiulty shoul only e oserve for initilly struturlly miguous sentenes! Metho " Eye trking (self-pe reing woul lso e possile)! Your inepenent vriles re " Wor orer (SVO vs. OVS) & miguity (miguous vs. unmig.)! Your epenent vrile is " Reing times in wor region P. Knoeferle 3 P. Knoeferle 4

2 Homework Homework! Design " (1) Die Mutter vershieet en Besuher nh er Prty. " (1) Die Mutter vershieet er Besuher nh er Prty. " (2) Der Vter vershieet en Besuher nh er Prty. " (2) Den Vter vershieet er Besuher nh er Prty.! Control " Plusiility, e.g., pretest in form of plusiility rtings on sle from 1 (very implusile) to 7 (highly plusile) " Wor length (+/-2hrs) " Frequeny of lemms (e.g., Celex)! Lists " For 2x2 esign with 2 levels for eh ftor, there re 4 exp. lists " One prtiipnt sees one list " Ltin Squre to ensure tht there is for eh list! Equl numer of trils in eh onition (24 items/4 ons: 6)! Conuting the experiment! Anlysing the t to fin out whether our mnipultion (mig. vs. unmig.) h n effet? " Exploring the t " Inferentil sttistis Item List1 List2 List3 List4 P. Knoeferle 5 P. Knoeferle 6 Exploring the t Exploring the t! Quntittive t " Compre men reing times! Br grphs with onfiene intervls (CI): 95% CIs " CIs inite the rnge within whih we expet the true vlue of the men will fll " 95% of the men vlues in our popultion fll etween the rnge inite y the onfiene intervls! So wht oes nrrow onfiene intervl inite? " The smple men is lose to the true men " Wie onfiene intervl: men oul e very ifferent from true men onfiene intervl Men regression pth urtion on NP2! Error r grphs for repete mesures esign " Stts progrms tret t s if from iff. groups " Solution! Eliminte etween-sujets vriility! Normlize prtiipnts mens! All prtiipnts hve sme men ross onitions 1. Clulte men time for eh prt. ross onitions 2. Compute grn men of ll the prtiipnts mens 3. Clulte justment ftor: just = grn men - prtiipnt mens 4. Crete juste vlues for eh vrile: Vr. + just Before normlizing the mens After normlizing the mens P. Knoeferle 7 P. Knoeferle 8

3 Exploring the t Assumptions out the t! Boxplots (ox-whisker igrms) " Qurtiles! Top/ottom qurtile " Rnge etween whih top/lowest 25% of sores fll! Interqurtile rnge " Rnge in whih the mile 50% of the sores fll! Mein " Mile sore if you rrnge the reing times in orer (! men) " Looking for outliers Mein highest sore top 25% mile 50% lowest 25% lowest sore! If we ultimtely wnte to o more thn just esriptively explore the t " We nee to eie whih test to use! For our t (reing times) we typilly use prmetri tests " Prmetri tests re se on the norml istriution " There re ertin requirements for performing prmetri tests! The t " Must e t lest intervl-sle t " Must e normlly istriute " Vrines in popultions/groups/onitions roughly equl (homogeneity of vrine)! Test for inepenent (etween-sujets) esign in ition ssume " Sores tht we ompre re inepenent (i.e., from ifferent people) " So we nee to hek first whether our t meets these requirements P. Knoeferle 9 P. Knoeferle 10 Exploring the t: skew n kurtosis Testing for normlity! In norml istriution, skew (lk of symmetry) n kurtosis (pointyness) shoul e zero " Positive vlues of skewness mens left-skewe " Negtive skewness vlues inite right-skewe " Positive kurtosis vlues inite pointy istriution " Negtive kurtosis inites flt istriution Moments! The further the skewness/ kurtosis vlues from zero, the more likely it is tht the t re not normlly istriute " Atul vlues for skew/kurtosis not informtive " z-trnsformtion z skewness = S " 0 SE skewness z kurtosis = K " 0 Mile sore if you orere the otine sores Sore tht ours most frequently in t set Z s =.312/.414 = SE kurtosis Z k = -.616/.809 = P. Knoeferle 11! Kolmogorov-Smirnov test for normlity " Shoul you test the t overll or rther for eh onition?! If the result of the K-S test re signifint you nnot perform prmetri test on tht t " Trnsform the t! E.g., log trnsformtions sqush the right til of the istriution, n n reue positive skew P. Knoeferle 12

4 Testing for homogeneity of vrine Sttistil tests! For etween-sujet esigns " Levene s test! For repete mesures " Spheriity ssumption in repete mesures nlysis of vrine (ANOVA)! One we hve explore the t in this wy " An re sure they meet the ssumptions of prmetri tests! We n test ifferenes etween the mens using inferentil sttistis! Whih test shoul we hose?! We istinguish etween prmetri n non-prmetri tests " Prmetri tests! For t tht re se on the norml istriution (e.g., intervl sle n ove)! T-Test: For 1-ftor esigns with 2 levels! Anlysis of Vrine (ANOVA) " Cn test the inepenent effet of ftor: min effet " Cn test for intertions (reltionships etween effets) " Non-prmetri tests! Do not ssume the t re from norml istriution (e.g., for tegoril t) " Chi-squre test " Log-liner moels " For our t (inspetion urtion ) we use prmetri tests P. Knoeferle 13 P. Knoeferle 14 Sttistil tests Dt olletion n vrition! So, how o test sttistis work?! Two types of vrine for oth ep./inep. esigns " Systemti vrition: result of experimentl mnipultion! E.g., SVO vs. OVS sentene onition " Unsystemti vrition: vrition ue to rnom ftors: e.g., ge, gener! Test sttistis " Disover how muh vrition there is in performne " How muh of this vrition is systemti versus unsystemti " Is there more vrition thn without the experimentl mnipultion?! In our eision tree, why o we get istintion etween tests for epenent n inepenent t olletion?! Unsystemti vrition in t iffers epening on the type of t olletion " Within-sujets (epenent) esign! One prtiipnts reeives ll onitions! So other ftors (e.g., ge, IQ et.) re onstnt ross onitions " Between-sujets (inepenent) esign! Even in the sene of n experimentl mnipultion, we woul fin ifferenes etween the groups sine these ontin ifferent prtiipnts tht iffer in gener, IQ, ge, et.! Repete mesures esigns re goo t eteting true effets " Why?! Unsystemti vrition ( noise ) is kept to minimum P. Knoeferle 15 P. Knoeferle 16

5 Minimize unsystemti vrition Compring two mens! In oth types of esign: minimize unsystemti vrition " Rnomiztion: elimintes soures of systemti vrition other thn our mnipultion! Repete-mesures " Prtie effets: fter 10 OVS sentenes, they eome esy " Boreom effets! Solution " Ensure tht these effets proue no systemti vrition etween our onitions " Counterlne the orer in whih person prtiiptes in onition! Inepenent esigns " Confouning ftors ontriute to vrition (e.g., ge, IQ), " But: ensure they ontriute to unsystemti, not systemti, vrition! Solution " Allote prtiipnts rnomly to n experimentl onition P. Knoeferle 17! Let s ssume for first test tht we h n experiment with only 2 onitions (1 ftor, 2 levels) " SVO n OVS miguous " Effet of inepenent vrile sentene type on reing times! Error rs for regression pth urtion on NP2! It looks s if the ovs-m. men is muh higher thn the svo-m. men " Test: Compring two mens " Is the ifferene ue to hne (e.g., noise) or our experimentl mnipultion? " Sttistil tests provie us with proility (p) tht the ifferene is genuine (n not ue to hne) P. Knoeferle 18 T-Test T-Test! Compring mens etween two groups/onitions " Let s look t simple test sttistis: T-Test " Inepenent mens t-test! When there re two onitions n ifferent prtiipnts ssigne to eh onition (inepenent mesures/smples t-test) " Depenent mens t-test! Sme prtiipnts took prt in oth onitions (mthepirs/pire-smples t-test)! We hve ollete t n lulte the mens! If from the sme popultion, the mens shoul e roughly equl " H0: experimentl mnipultion hs no effet on prtiipnts, n smple mens shoul e very similr! I.e., men reing time for SVO-m. is similr to OVS-m. " Mens might iffer y hne! But: lrge ifferenes shoul our infrequently y hne P. Knoeferle 19! Compre ifferene etween otine smple mens to ifferene etween mens tht you woul expet y hne " Tht mens you nee mesure of two things! How ifferent the oserve ifferene etween your smple mens is from the ifferene tht you woul expet in popultion mens (if H0 is true this seon iff. woul e 0) " We further nee mesure of unsystemti vrition (i.e., noise tht we woul get y hne) " We nee to know how likely it is tht ifferene etween the mens oul result from the ft tht for our t smple mens iffer lot lrey y hne! Rell the stnr error (SE) " Mesure of vriility etween smple mens! Smll SE: most smples shoul hve similr mens! Lrge SE: lrge ifferenes in smple mens y hne lone P. Knoeferle 20

6 Vrine T-Test! Vrine is the verge vriility in the t (spre)! meium vriility! high vriility! low vriility P. Knoeferle 21! Let s ssume " The ifferene etween our otine smples (SVO-m. & OVSm.) is lrger thn the wht we woul expet se on the SE! Smple mens in our popultion vry lot y hne & our two smples re typil of our popultion! The two smples me from ifferent popultions & re typil of their respetive popultion " Differene etween smples represents true ifferene " As oserve iff. etween smple mens gets lrger, the more onfient we n e tht the seon option is orret! The result for the t-test is t-vlue tht helps us eie whether we hve foun true ifferene or not " The igger the t, the more likely we foun true iff. t = Oserve ifferene _ etween smple mens Expete ifferene etween popultion mens (if null hypothesis is true) Estimte of the stnr error of the ifferene etween two smple mens P. Knoeferle 22 The epenent T-Test ANOVA! The t-test " Compres the men ifferene etween our smples (!D) with the ifferene we woul expet to fin etween popultions mens (µ D )! The effet of our mnipultion " Tkes into ount the stnr error of the ifferenes (s D /sqrt(n))! I.e., unsystemti vrition t = D " µ D s D / N! For our 1-ftor (2levels) exmple the result is " t(31) = -2.77, p < 0.01! But tully, for the 2-ftor exmple from your homework, we nee more omplite nlysis: repete mesures ANOVA P. Knoeferle 23! Just like T-Test, the ANOVA tells you whether " Differenes etween onitions re ue to your mnipultion " Due to unsystemti vrition " The two types of vrine llow us to rw inferenes out mens! The ANOVA n help us nlyse ifferenes etween mens in more omplite esigns (e.g., 2x2) " The result of n ANOVA nlysis is F-vlue! Rtio of the vrine ue to your experimentl mnipultion over unsystemti vrition! A high F-vlue inites lot of the vrition results from your mnipultion systemti vrition F = unsystemti vrition " This is very generl formul, n the ext lultions will iffer epening on your type of mesurement (epenent vs. inep.) P. Knoeferle 24

7 Exmple stuy Min effet n intertion Trxler, Pikering, & MElree, 2002, JML! Semnti interprettion " Vers like egin n our with NP-rguments of ifferent semnti types! Event: strt fight! Entity: strt puzzle! Vers like egin n strt pper to prefer n event s rgument " Coerion opertion tht type-shifts n entity to n event y inserting itionl semnti struture! The oy strte solving the puzzle " 2x2 esign! Ftor 1: NP type (entity, event)! Ftor 2: Ver type (entity, event) " Trget region: the fight/puzzle! Min effet " The unique effet of n inepenent vrile " Reing times for entity NP onitions re higher thn for event-type NPs " Min effet of NP type onfirms this oservtion! F1(1, 35) = 14.4, p < 0.01 F2(1, 31) = 5.74, p < 0.05 " F: signl-to-noise; the igger the F, the stronger the effet of our mnipultion " p: proility tht the finings re ue to hne Reing times in ms Seon pss times uring "the fight/puzzle" Event NP NP type Entity NP Event ver Neutrl ver P. Knoeferle 25 P. Knoeferle 26 Min effet n intertion Summry! Intertion " The omine effet of two or more inepenent vriles on the epenent vrile " The ver-type ftor ffets reing times ifferently for Entity-type NPs thn for Event NPs Reing times in ms Seon pss times uring "the fight/puzzle" Event ver Neutrl ver! Homework: experiment esign! Exploring t (here: t lest intervl-sle) " Error r grphs " Box plots " Testing for norml istriution n homogeneity of vrine! Inferentil sttistis " Compring two mens (1 ftor, 2 levels): T-Test " ANOVA " An exmple reing stuy: min effet vs. intertion 0 Event NP NP type Entity NP! Reing for next week: " Lexil proessing n the mentl lexion. In: A. Rfor, M. Atkinson, D.Britin, H. Clhsen, & A. Spener (1999). Linguistis: n introution (pp ). Cmrige, CUP. P. Knoeferle 27 P. Knoeferle 28