Framework for interim reporting

Transcription

1 Framewrk fr interim reprting Prpsal by GAML Secretariat Glbal Alliance fr Mnitring Learning Furth meeting Nvember 2017 Madrid, Spain GAML4/18 Headline 1 Overview

2 2 GAML4/18 Framewrk fr interim reprting Overview Depending n the indicatr, a few cuntries - nt all regins represented - cnduct crssnatinal surveys/assessments. Nt participating in crss-natinal assessments/surveys after 2017, a large number f cuntries d nt measure learning. Sme assessment/surveys are nt peridical. There is a number f cuntries that have their wn natinal initiatives/assessments. Hetergeneus characteristics f assessments: N agreed standards with respect t cntents and data quality; There are varius natinal and crss natinal framewrks; There are different benchmarks fr every assessment; Tls have different scpe and cverage; Different cverage in terms f dmain; Different mdes f assessment (paper/cmputer based). Identifying the prblem The main issue in reprting at this level seems t be cmparability acrss systems and languages. D r shuld assessments at this level allw fr cmparability in reprting? There is a need fr an initial list f criteria fr data and measures. It is imprtant t acknwledge that measurement cannt be dne withut parallel measurement f cntextual determinants. There is a need t identify if further methdlgical wrk is required. There is a need t identify if further data cllectin tls are required. There is a need t include bth wrk n existing datasets and the develpment f new samples. There is a need t encurage/induce sme cnvergence f tls in rder t achieve mre cmparability. There is a need t cnsider alignment f skills between pre-primary, primary and yuth and adults skills. An interim strategy needs t take advantage f existing effrt. Defining principles fr reprting Universal defining criteria nt using a unique apprach r tl unless agreed glbally. Desirably lng-term view given that many cuntries will chse their wn tls t reprt. A definitin f cnstructs desirable in a framewrk t achieve glbal cmparability, r have hks that allw cmparability. Guide the best pssible, cst effective measurement, nt nly reprting t SDGs.

3 3 GAML4/18 Framewrk fr interim reprting Have principles that are as pragmatic as pssible and as rigrus as needed: Based n lng-term view. Accmmdate a wide range f perfrmance/cntexts: Allw acrss all grades/ages including early childhd, and Include ut-f-schl children, if relevant. Useful t cuntries given natinal bjectives (cnsistent with what cuntries are wrking tward). Depart frm lng-term view: Establishing a cmmn framewrk fr reference that defines the cnstructs t be evaluated acrss all cntexts; Guide the best pssible, cst-effective measurement, nt nly reprting t SDGs; List the set tls that culd serve t infrm the target; Evaluate t develp a set f purpse-built tls that cuntries can draw n/adapt. Identify criteria fr reprting in three areas: Des the measure cver the necessary dmains? What are the prperties f the tl? What are the prperties f the data? Interim strategy An interim strategy prmtes the highest level f participatin and reprting by priritising a fit-fr-purpse apprach. The fcus wuld be t take all tls, regardless f whether they meet these criteria, and reprt thse using anntatins fr thse that d nt meet all the criteria. Nn-ideal measures wuld be accepted; Reprt data with anntatins; Natinal data t be reprted; Natinal benchmarks t be utilized; and Slutins will be wrked ut with gvernments. Data gaps are ging t be filled with available data, prvided the fllwing are given t judge alignment: Data n the indicatr; and Infrmatin abut prcedural decisins. Interim reprting prcess Identify ideal criteria fr data and measures (and dcument them in writing and with examples). Evaluate existing data surces against thse criteria and integrate criteria int the Catalgue f Learning Assessments (CLA) and ther mechanisms.

4 4 GAML4/18 Framewrk fr interim reprting Outline a reprting system with tw pssible appraches: Cnceptual alignment; and Pssible empirical appraches including linking. Indicatr Results frm all assessments are accepted, whether schl-based r nt. The name f the assessment and year are required. Results fr +1/-1 grade are accepted, except fr lwer secndary, and the cuntry is t identify if it is reprting in the exact grade r nt. Results can r cannt include ut-f-schl children. The cuntry needs t clarify if there are ther exclusins. The cuntry needs t add a clumn with the percentage f ut-f-schl children and the number f years f the relevant ISCED level, if it represents the end f cycle. Results are accepted with the assessment s wn minimum level benchmark with plicy descriptrs: This will allw alignment with harmnized levels; and It will be pssible t highlight the relatin between the minimum level benchmark and the glbally recmmended ne. The cuntry needs t reprt data generating prcedures. If fllwing data alignment criteria at least in three main dimensins: Fitness fr purpse; Representativeness; and Translatin. Where results are lngitudinally equated r nt. Indicatr Define develpmentally n-track: Criterin referenced. Measure learning in a hlistic way: Health, psychscial well-being, learning. Depart frm a lng term view: Describe the learning dmain and its ties t ther dmains general areas f early language/literacy, early numeracy, scial/emtinal, physical. The results can be ppulatin-based. The surveys can be cnducted n a representative sample. Prvide tls that are useful t cuntries given natinal standards (cnsistent with what cuntries are wrking twards).

5 5 GAML4/18 Framewrk fr interim reprting The results need t be glbally cmparable, r have hks that allw ne t determine their cmparability. The assessments need t be administered at a variety f ages. The assessments need t have a well-defined reprting framewrk. The assessments need t fllw the standards in the Gd Practices in Learning Assessment (GP-LA). Indicatrs and Draw n the assessment framewrks and tls, and reprt n them with the apprpriate ftnting. Quality standards t be used as ftnting: Definitin f literacy that invkes cntinuum; Assessment that cvers a full range f skills; Representative sample; Frm f administratin f assessment (paper r cmputer based); Direct r indirect reprting; Natinal r Crss Natinal assessment; Ideally cver in terms f age grups; and Use f adaptive tl r nt. Indicatrs and Results frm all assessments are accepted, whether schl-based r nt. The name f the assessment and year are required. The grades crrespnding t the ages need t be reprted. Results can r cannt include ut-f-schl children. The cuntry needs t add a clumn with the percentage f ut-f-schl children and the number f years f the relevant ISCED level, if it represents the end f cycle. Results are accepted with the assessment s wn minimum level benchmark with plicy descriptrs. The cuntry needs t reprt data generating prcedures. Align with the manual and cde f gd practices The results needs t align with the manual and cde f gd practices. The results need t fllw the data alignment criteria at least in three main dimensins: Fitness fr purpse; Representativeness; and Translatin.

6 6