Milestone Messenger. Selected Response. Constructed Response. Cobb County School District Middle School November Edition

Transcription

1 Milestone Messenger Cobb County School District Middle School November Edition Selected Response Constructed Response Updates: Training on Common Formative Practices focusing on Selected Response items has been provided to Assessment Assistant Principals. Materials from this training can be accessed through Blackboard. A Module on writing Constructed Response questions has been shared with Principals. CTB Online Test Environment DEMO NOW AVAILABLE! Expose students to the online platform so that they are familiar with the online environment before testing. A demonstration of CTB s online platform can be accessed at Office of Office of Assessment and Personalized Learning Address Questions to dianna.denton@cobbk12.org Middle School Assessment Supervisor

2 Office of Assessment and Personalized Learning Address Questions to Middle School Assessment Supervisor Georgia Milestone th Grade End of Grade Assessment Assessment Development Cycle Writing Multiple Choice Items Constructed Response Mathematics Sample Student Work Sample Overview of Writing Constructed Response Items Office of Assessment and Personalized Learning 2

3 Assessment Development Cycle Planning Stage: Determine who will use the assessment results and how will they use them. Is the planned use Formative or Summative? o The assessment instrument is designed to align directly with the content standards to be learned. o The assessment is able to provide information on specific problems/misconceptions so that teachers can make good decisions about actions to take. o All the assessment s items/tasks match what has been or will be taught. o The results should be available in time to provide opportunity to assist students. o Both teachers and students must actually take action based on the results. 2 Planning Stage: Identify the learning targets to be assessed o List the learning targets to be assessed along with their classification (knowledge, reasoning, skill or product). o Deconstruct the target for clarity. o Specifying the intended targets is important because the depth and breadth of the target will determine how much time should be devoted to instruction and how many questions should be on the assessment. o Clarifying the targets is important because different target types require different assessment methods. 3 Planning Stage: Select the appropriate method or methods o Once the learning targets have been classified by type, decide on the assessment method/s. o Adhere to the guidelines for each method. 4 Planning Stage: Sample size o There should be enough questions to determine how each student performed in mastering a particular standard or target. o A learning target that is broad in scope indicates a need for a greater number of questions. o The more important the target is as a foundation to later learning, the more questions there should be. o The more important the decision to be made based on the results, the larger the sample must be. Office of Assessment and Personalized Learning 3

4 5 Development Stage: Develop or select items, exercises, tasks and scoring procedures o Refer to Classroom Assessment for Student Learning. Doing It Right Using It Well. Stiggins, R. et al 6 Development Stage: Review and critique the overall assessment for quality before use o Consider sources of bias and distortion common to all assessment methods. o There are several student issues that can impact the assessment such as language, emotional upset, physical handicap, lack of motivation at the time of the assessment, lack of personal confidence leading to test anxiety, a lack of understanding on how to take tests etc. o There can be issues at the time of the assessment such as insufficient time in which to take the test, noise, poor light, discomfort, a lack of rapport with the assessor, a lack of proper equipment etc. o The assessment itself can cause issues such as poor directions, poorly worded questions, misleading layout, missing information, poor reproduction of the test etc. 7 Use Stage: Conduct and score the assessment o The purpose of the assessment it must be considered when reporting results. o If the user is the teacher then what type of breakdown of results will best serve the formative/summative purposes? o If the user is the student how will the results be reported to communicate strengths and areas for improvement? o The results should be timely. 8 Use Stage: Revise as needed for future use o Did the assessment do what it was intended to do? o Were the results able to be used for the decisions that were intended to be made? o Were the students able to use the results to understand their strengths and identify the areas needing attention? o Was there a complete alignment between was taught and what was assessed? o Were parts of the assessment a surprise to the students? o Did a source of bias or distortion creep in and affect results? Office of Assessment and Personalized Learning 4

5 Writing Selected Response Items Multiple Choice items work best when: Able students who know the material get the answer right Challenged students who work hard get the answer right Students who do not prepare do not get the answer right This means that the question should not be guessable by someone who has not worked on the material. The question should require specific knowledge rather than the ability to use logic in generic situations. (Questions that test that are usually found on IQ tests). Goal of Test Items Should be about what the students know Measure the required skill or information (Learning Targets) Focus on important content Be written at an appropriate level of difficulty Be clear and concise. Use vocabulary that is age and grade-level appropriate. Use technical vocabulary when assessing the students knowledge of the meaning of the word/phrase Be bias free Test Vocabulary Stem the initial part of the item in which the task is given Answer Choices the correct answer and the distracters in a MC item Distractors incorrect answers in a Selected Response item. o A good distractor looks like the correct answer but is not. At least one distractor is obviously wrong so a student who knows something will have an increased chance of guessing the right answer by process of elimination. The other distractors should be reasonable responses but clearly incorrect to someone who understands the material. The other distractors may in fact be true statements but not relevant to understanding the question being asked. Stimulus passage, picture, graph, map, chart or quotation needed to interpret the question Key the correct answer in a MC item Creating Multiple Choice Items The questions should: directly align to one or more content standards ask enduring or essential information have a stem have 3 answer choices in Grades K, 1 and 2 have 4 answer choices in Grades 3-5 have only one correct answer keep the focus on finding what students know rather than determining whether they can decipher the question Office of Assessment and Personalized Learning 5

6 The Stem should be narrowly focused Ask a complete question. Direct questions are best but incomplete statements are sometimes necessary to avoid awkward phrasing or convoluted language Are clearly expressed so the student knows what s expected Avoid starting a question with Which of the following Avoid all of the above, none of the above, always, never, best, most, least, and worst Avoid a question stated in the negative Avoid the word you Do not use a graphic, map etc. unless it is necessary to answer the question Make sure the grammar in the stem does not give clues to the answer Answer choices Teachers should always consider the following when designing answer choices: Provide a single response that is agreed to be the best Should write the correct answer before the distractors Must be plausible and attractive to those that lack knowledge Simulate likely errors and misunderstandings Should be about the same length and parallel in grammatical structure Should not include silly or absurd choices Should not be partially correct or overlap with the correct answer Arrange alternatives in a logical order Avoid all of the above, none of the above, always, never, best, most, least, and worst Office of Assessment and Personalized Learning 6

7 Constructed Response Mathematics Sample The figure shows the dimensions, in centimeters (cm), of a cube. Part A Explain why the expressions representing the dimensions are valid edge lengths of the cube. Show your work. Part B If x = 2, determine the volume of the cube in cubic centimeters. Show your work. Be sure to complete ALL parts of the task. Write your answer and show your work on the paper provided. Do NOT type your answer in the text box below. Answer to obtain a Score 4 Office of Assessment and Personalized Learning 7

8 Student Work Sample Score 4 Note: During the Georgia Milestone Assessment, the rubric used for Constructed Responses will not be available for either teachers or students. The task-specific rubric for the piloted task above has been included for teachers to examine in order to have a general idea of what might be expected. Office of Assessment and Personalized Learning 8

9 Not Addressed Not Addressed Not Addressed Not Addressed MILESTONE TASK-SPECIFIC RUBRIC At this time, learning how to work through constructed responses with the use of rubrics should be encouraged. A Cobb County rubric is provided for this purpose. COBB COUNTY RUBRIC Representation Does the representation accurately reflect the important mathematical concepts in the problem? Strategies & Reasoning Is there evidence that a strategy was chosen using logical mathematical reasoning? Successful Progressing Limited NA The representation(s) reflect complete understanding of all the important mathematical components presented in the task. Strategy/strategies applied is/are efficient and appropriate for the task. The representation(s) reflect partial understanding of some of the important mathematics in the task. Strategy/strategies applied is/are partially correct because only part of the task is addressed. The representation(s) are incomplete or do not accurately reflect the important mathematics in the task. The strategy/strategies used is/are not appropriate for the task. Computation & Solution Is the computation accurate and complete with the correct solution(s) for the problem? The computation is accurate and leads to a correct solution(s) that addresses all the mathematical components in the task. The verbal/written explanation is well organized and easy to follow from one step to the next. The computation is accurate but does not lead to a complete or correct solution(s) to the task. The computation is incorrect or there is no solution, or the solution has no relationship to the task. Communication Does the verbal and/or written explanation clearly communicate the student s reasoning? The verbal/written explanation is somewhat clear but inferences are necessary. The verbal/written explanation is incomplete or difficult to understand. Office of Assessment and Personalized Learning 9

10 Over view of Writing Constructed Response Items W h at i s a C R I te m? C o n st r u c t e d - r e s p o n s e q u e st i o n s a r e a s s e s s m e n t i t e m s t h at a s k st u d e n t s t o a p p l y k n o w l e d g e, s k i l l s, a n d c r i t i c a l t h i n k i n g a b i l i t i e s t o re a l - w o r l d, st a n d a rd s - d r i ve n p e r fo r m a n c e t a s k s. A d va ntage s : B e t t e r i n f e r e n c e s a b o u t s t u d e n t u n d e r s t a n d i n g A l l o w s f o r h i g h e r D O K q u e s t i o n i n g C h a l l e n g e s : D i f f i c u l t t o w r i t e T a k e s l o n g e r t o s c o r e S t u d e n t w r i t i n g p r o f i c i e n c y m a y i m p e d e s t u d e n t s a b i l i t y t o s h a r e t h e i r k n o w l e d g e C h a l l e n g e t o s c o r e c o n s i s t e n t l y P r o c e s s fo r Wr i t i n g C R I te m s : B e g i n w i t h t h e s t a n d a r d o r s t a n d a r d s. W h a t i s i t t h a t y o u w a n t t o a s s e s s? D e t e r m i n e t h e l e v e l o f t h i n k i n g t h e s t u d e n t s n e e d t o a t t a i n ( B l o o m s, D O K o r R i g o r a n d R e l e v a n c e ). C h o o s e q u e s t i o n t y p e ( 5 t y p e s ). D r a f t t h e q u e s t i o n. A s k a p e e r t o r e v i e w. C r e a t e a s c o r i n g r u b r i c. T i p s fo r Wr i t i n g C R I t e m s : M a k e s u r e t h e q u e s t i o n i s a s s p e c i f i c a s p o s s i b l e. U s e s i m p l e a n d d i r e c t l a n g u a g e. U s e w o r d s f r o m t h e p a s s a g e i n t h e q u e s t i o n ( i f a p a s s a g e i s a p p r o p r i a t e ). U s e s i m p l e v o c a b u l a r y u n l e s s y o u a r e t r y i n g t o a s s e s s s t u d e n t s k n o w l e d g e o f t h e m e a n i n g o f c o n t e n t v o c a b u l a r y. U s e l a n g u a g e t h a t i s b o t h a g e, g r a d e - l e v e l a p p r o p r i a t e. A v o i d b i a s. I f t h e r e a r e p a r t s, l a b e l o r b u l l e t t h e m. Office of Assessment and Personalized Learning 10

11 Types of Constructed Response Questions Response to Provided Information Student Choice Two or More Independent Components Single Dimension Scaffolded Adapted from the State of Kentucky DOE Office of Assessment and Personalized Learning 11