Teaching performance assessment (Program Standard 1.2)

Transcription

1 Teaching perfrmance assessment (Prgram Standard 1.2) Rbust assessment f pre-service teachers is vital in giving students, families and schls cnfidence that graduates frm Australian initial teacher educatin prgrams are effectively prepared fr entry int the prfessin. The accreditatin f initial teacher educatin prgrams in Australia: Standards and Prcedures requires that all pre-service teachers successfully cmplete a rigrus assessment (teaching perfrmance assessment) that cvers the breadth f teaching practices t ensure that they are classrm ready by the time they graduate. This fact sheet prvides further infrmatin n the requirements f the teaching perfrmance assessment as utlined in Prgram Standard 1.2. The fact sheet has been designed t assist members f accreditatin panels t understand and assess the requirements f Prgram Standard 1.2 and t supprt prviders in the develpment f their accreditatin applicatins Hw des the requirement fr a TPA fit within the accreditatin prgram standards? The Australian Prfessinal Standards fr Teachers at the Graduate career stage frm the basis fr accreditatin f initial teacher educatin prgrams. It is a fundamental principle that prgrams are accredited n the basis that they prduce graduates wh meet the Graduate Teacher Standards. In a teaching perfrmance assessment (TPA) a pre-service teacher illustrates their skills, knwledge and practices thrugh evidence f their perfrmance aligned t the Graduate Teacher Standards. In line with cncepts f authentic assessment (see bx 1 belw), evidence is drawn directly frm their wn practices t demnstrate: what they want students t learn hw they will facilitate this learning hw they will knw if students have achieved this learning (NBPTS 2005). The TPA is a key mechanism by which prgrams can demnstrate pre-service teachers impact n student learning (Prgram Standard 1.3). Initial teacher educatin prviders may chse t develp their wn TPA, wrk with ther prviders t develp a cmmn assessment r use/adapt an existing assessment. What are the requirements fr a TPA in Prgram Standard 1.2? Prgram Standard 1.2 requires that a TPA be situated in a classrm envirnment, in rder t demnstrate a range f authentic teaching practices. The TPA must be a requirement f successful cmpletin f the prgram, and must be cmpleted during the final year. Where pssible it shuld be included in the final prfessinal experience placement r internship, prir t graduatin. The TPA, as articulated in Prgram Standard 1.2, must have the fllwing features: be a reflectin f classrm teaching practice, prviding direct evidence f candidates' planning, teaching, assessing and reflecting (cnstruct validity see belw) be a valid assessment that clearly assesses the cntent f the Graduate Teacher Standards (cntent validity see belw) have clear, measurable and justifiable achievement criteria that discriminate between meeting and nt meeting the Graduate Teacher Standards be a reliable assessment in which there are apprpriate prcesses in place fr ensuring cnsistent scring between assessrs include mderatin prcesses that supprt cnsistent decisin making against the achievement criteria.

2 Bx 1 - Authentic assessment Darling-Hammnd and Snyder (2000, p ) identify five aspects f authentic assessment t judge teaching: The assessments sample the actual knwledge, skills and dispsitins desired f teachers as they are used in teaching and learning cntexts. The assessments require the integratin f multiple kinds f knwledge and skill as they are used in practice. Multiple surces f evidence are cllected ver time and in diverse cntexts. Assessment evidence is evaluated by individuals with relevant expertise against criteria that matter fr perfrmance in the field. The assessment practice includes multiple pprtunities fr learning and practising the desired utcmes and fr feedback and refectin, in rder t develp as well as mnitr teaching judgement and skill. Cntent validity assessment against the Graduate Teacher Standards Fr the purpses f Prgram Standard 1.2 cntent validity refers t the degree t which the TPA represents the cntent it is designed t measure. In this case, this is the degree t which it is aligned with and assesses the cntent f the Graduate Teacher Standards, and assesses a pre-service teacher s practice against these Standards. A TPA with cnstruct validity as described abve prvides pprtunities t demnstrate a wide range f the Graduate Teacher Standards. While a pre-service teacher is required t be assessed against the Graduate Teacher Standards during their prfessinal experience placement, the TPA will nt necessarily cver all f these Standards. It is up t each ITE prvider t prvide guidelines as t which Graduate Teacher Standards will be the fcus fr the TPA nting that the elements f planning, teaching, assessing and reflecting are required under Prgram Standard 1.2. Cntent validity culd be evidenced thrugh a descriptin f the prcedures fllwed t develp and test the assessment including: mapping f the assessment tasks t the skills and knwledge required in the Graduate Teacher Standards the TPA is designed t assess testing this alignment with a grup f external experts (e.g. external teacher educatrs, teachers and schl leaders) quantitative studies demnstrating cntent validity. During the transitin t the 2015 Standards and Prcedures, quantitative studies may nt have been evidenced at the pint f accreditatin. At a minimum face validity shuld be demnstrated (e.g. mapping and testing with experts). Prir t full implementatin it is expected that cntent validity wuld be quantitatively established and a Teacher Regulatry Authrity/accreditatin panel is likely t request this as subsequent evidence. Cnstruct validity alignment t the practices f teaching Fr the purpses f Prgram Standard 1.2 the term cnstruct validity refers t the degree t which the TPA measures the actual practices f teaching, including but nt limited t planning, teaching, reflecting and assessing student learning.

3 It is expected that evidence prvided in an accreditatin applicatin wuld include details f hw the TPA has been designed t require and assess pre-service teachers demnstratin f their teaching practices. Evidence f teaching practices culd include classrm bservatin f pre-service teachers alng with artefacts such as lessn plans, assessment strategies and feedback, samples f student wrk, bservatin ntes and reflectins. In assessing the cnstruct validity cnsideratin culd be given t ensuring that the TPA prvides direct evidence f planning, teaching, reflecting and assessing student learning. Fllwing the initial establishment f cntent validity, quantitative validity studies wuld prvide further evidence f the validity f the assessment tl. Achievement criteria Prgram Standard 1.2 requires that the TPA includes clear, measurable and justifiable achievement criteria that discriminate between meeting and nt meeting the Graduate Teacher Standards. Prviders are required t prvide: evidence that the standard fr successfully cmpleting the TPA is set at a level that reflects the Graduate Teacher Standards a credible prcess fr differentiating thse wh meet the standard and thse wh d nt. As the TPA is a high stakes assessment, the tls and prcesses used t determine the criteria fr meeting and nt meeting the standard are crucial. Fr example, evidence culd include reference t the use f a well regarded standard setting methdlgy t determine the passing threshld. Reliable assessment Reliable and cnsistent assessment f the TPA is a critical element in ensuring rbust initial teacher educatin prgrams that will prvide cnfidence in the readiness f their graduates. The TPA will be required t include a reliable assessment in which there are apprpriate prcesses in place fr ensuring cnsistent scring between assessrs (inter-rater reliability). T ensure reliability, a prvider shuld have prcesses and guidance in place that: prvide apprpriate knwledge and skills fr external assessrs, institutinal staff and/r supervising teachers in understanding and assessing the TPA prvide clear guidance n what evidence is required t demnstrate that a pre-service teacher s assessment meets the passing standard cnfirm accurate and cnsistent judgements against the TPA s rating scales/rubrics are ccurring, including quantitative data demnstrating the reliability f assessrs. Depending upn the TPA used an ITE prvider may als have access t lcal and refereed research analysis f inter-rater reliability fr their TPA, r plans t develp this. This culd be used t prvide additinal evidence f the reliability f the assessment. Wh assesses the pre-service teacher s TPA? Depending upn the design f the TPA the fllwing individuals (r mix f individuals) may be invlved in making the assessment f a pre-service teacher s perfrmance n the TPA: external assessrs Higher Educatin Institutin (HEI) staff supervising teachers

4 Regardless f wh is undertaking the assessment, an ITE prvider needs t demnstrate that assessr training, assessment prcedures and assurance prcesses ensure cnsistent and reliable assessment between assessrs. Fr example, if a prfessinal experience reprt is a cmpnent f a TPA a prvider needs t demnstrate hw each individual supervising teacher will be skilled in making cnsistent assessments. Ensuring that the assessr training and assessment prcedures are being applied cnsistently will be an imprtant element f a prvider s evidence fr this element f the Prgram Standard. A panel that can see clear mechanisms fr mnitring the implementatin f the TPA assessment wuld have cnfidence that pre-service teachers will be cnsistently assessed. Mderatin Mderatin prcesses that supprt cnsistent decisin-making against the achievement criteria are required t be in place t supprt the reliability f the TPA. A mderated assessment is an assessment fr which there are: prcesses in place t ensure cnsistent scring between assessrs cnsistent decisin-making against the achievement criteria. As part f their evidence fr the TPA a prvider shuld describe the mderatin prcess they will emply t ensure cnsistent assessment f the TPA. This culd include: detailing the specific prcedures t be implemented t imprve the reliability f judgements, including feedback lps t ensure nging cnsistency details f thse invlved in the mderatin prcess, fr example invlvement f supervising teachers r external experts evidence f mderatin at the pass-fail pint benchmarking with ther prgrams r crss-marking within r between institutins the review f examples f previus perfrmance, especially thse representing perfrmance at the pass-fail pint. Assessing evidence against Prgram Standard 1.2 the TPA Natinal Expert Panel AITSL is establishing a TPA Natinal Expert Panel (TPA Panel) t assist with the develpment and implementatin f teaching perfrmance assessments, including the design and successful perfrmance f TPAs. The expert panel's rle is t: Assess TPAs develped thrugh the AITSL TPA grant prgram against Prgram Standard 1.2 (visit the TPA grant prgram webpage fr mre infrmatin). Prvide advice t AITSL and Teacher Regulatry Authrities (TRAs) in regard t TPAs develped thrugh the TPA grant prgram. prvide expert advice n the develpment, validatin and/r implementatin f TPAs, including thse develped utside the TPA grant prgram. Prvide advice, when apprpriate, t initial teacher educatin prviders when develping, implementing and maintaining TPAs. The criteria fr the selectin f panellists when establishing this panel includes: experienced and expert knwledge in the design and delivery f initial teacher educatin, including in the cntext f prgram accreditatin technical expertise in statistical data analysis/psychmetrics including expertise n standard setting methdlgy and the develpment f scring strategies

5 assessment expertise and an understanding f hw t establish reliability and validity expert knwledge in the implementatin and mnitring f assessments t ensure nging fidelity The advice prvided by the TPA Panel will include cnsideratins fr implementatin and the nature f evidence that culd be expected fr each TPA ging frward. It will be crucial that cnditins in relatin t use f TPAs are understd and addressed in the accreditatin prcess. It is imprtant t nte that an accreditatin panel will always need t assess evidence against Prgram Standard 1.2 t ensure a prvider has implemented and will cntinue t implement the TPA with fidelity, that is, in the way it was designed t be used. The nature f TPA evidence Thrugh the develpment, trialling and nging implementatin f a TPA, the nature f evidence prvided and assessed as part f an accreditatin prcess will evlve. Demnstratin that a TPA meets the requirements f validity and reliability des nt ccur nly at the initial accreditatin. Ensuring nging validity and reliability will be part f the prcess fr demnstrating that a tl has been apprpriately implemented (with fidelity). As indicated in the Standards and Prcedures (p. 8) cnsideratin shuld be given as t hw validity and reliability will be evidenced fr subsequent accreditatins. In sme instances, it may be necessary fr evidence t be prvided thrughut the accreditatin perid, fr example, where a new TPA tl is being trialled and sufficient evidence cnfirming its validity and reliability has nt been prvided at the pint f accreditatin. In sme instances, a prvider culd be asked t prvide additinal infrmatin thrugh the accreditatin annual reprting prcess r have cnditins applied t their prgram accreditatin. The type f evidence required fr accreditatin will be dependent n the maturity f a TPA, fr example, initial implementatin f a TPA as ppsed t ne that has been perating fr a number f years. T give an indicatin n hw the nature f the evidence might change ver time the fllwing high level stages culd be expected (table 1). Table 1 Stages Validity Reliability Trialling r initial implementatin Evidence f face validity Plans t cllect quantitative data, fr example, calculatin f the cntent validity rati (CVR) fr the TPA Quantitative data where available Prcesses t establish reliability standard setting, effective identificatin, recruitment and training fr assessrs, assembling examples f candidate perfrmance Reliability data (where available) frm trial/initial implementatin First year f full implementatin Onging implementatin 1+ years Quantitative data and statistics demnstrating validity. As abve Additinal validity studies culd be cnsidered at this pint e.g. cncurrent validity Inter-rater reliability, cnsistency f decisins against achievement criteria Inter-rater reliability, cnsistency f decisins against achievement criteria

6 Mature 3+ years Lng term As abve Additinal validity studies e.g. predictive validity Additinal evidence ver time culd include: - Time series data n achievement n the TPA - Analysis frm lngitudinal research Outcmes f mderatin activities including crss-institutinal As abve *Nte these stages are nt intended t represent all pssible stages/appraches t develping, implementing and evidencing a TPA. They are prvided t illustrate hw the expectatins arund evidence fr a TPA will evlve. It als des nt represent all evidence expected in relatin t Prgram Standard 1.2 What data frm the TPA has t be reprted? Prgram Standards 6.2 and 6.3 utline the data that needs t be cllected, reprted and used t supprt a prgram s next accreditatin applicatin. As such, it may cntribute t the suite f evidence that a prvider will cllect t demnstrate the impact f their prgram, including impact n pre-service teacher learning and impact f pre-service teachers n schl student learning (Prgram Standard 6.2 and 6.3). Are there transitin arrangements fr meeting Prgram Standard 1.2? Transitin arrangements have been agreed fr Prgram Standard 1.2. Under these arrangements applicatins against Prgram Standard 1.2 will be required t prvide a descriptin f the strategy fr meeting the TPA requirements, and evidence f any wrk undertaken t date. It is expected TPA s will be available fr implementatin frm The descriptin f the strategy t implement the TPA shuld include: where the TPA will be placed within the prgram structure evidence regarding prperties f the assessment tl in line with the requirements f the standard (r plans t demnstrate this in the future), including prcesses fr assessr training and mderatin timeframes fr implementatin. The strategy shuld give cnfidence t the accreditatin panel that, when implemented the TPA will meet all requirements f the Prgram Standard and within the required time frames. ITE prviders shuld wrk with their relevant teacher regulatry authrity t discuss any issues in relatin t the implementatin f a TPA. References Darling-Hammnd, L., & Snyder, J. (2000). Authentic assessment f teaching in cntext. Teaching and Teacher Educatin, 16(5-6), Natinal Bard fr Prfessinal Teaching Standards (NBPTS) and Educatinal Testing Service (ETS) 2005, The take ne activity bk: An evidence-centered apprach t accmplished teaching, NBPTS & ETS, USA.