Request for Proposals for Providers of Principal Professional Development to Participate in an Impact Evaluation

Transcription

1 Request for Proposals for Providers of Principal Professional Development to Participate in an Impact Evaluation September 2014 RFP # Mathematica Policy Research American Institutes for Research Social Policy Research Associates Pemberton Research Important dates: RFP release date September 3, 2014 Questions due September 16, 2014 Answers posted September 24, 2014 Intent to bid due September 26, 2014 Proposals due October 14, 2014 In-person finalist presentations November 17 18, 2014 Expected contract award December 5, 2014 Copyright 2014 Mathematica Policy Research & American Institutes for Research. All rights reserved.

2 CONTENTS Chapter Page OVERVIEW OF REQUEST FOR PROPOSALS... 1 A. BACKGROUND... 2 A.1. Project rationale... 2 A.2. Theory of action... 2 A.3. Training and ongoing support... 3 A.3.1. Content... 3 A.3.2. Structure and duration... 4 A.3.3. Participants... 4 A.3.4. Intensity... 4 A.3.5. Capacity... 5 A.3.6. Replicability/consistency of implementation... 5 A.3.7. Evidence of effectiveness... 5 A.4. Description of impact evaluation... 5 A.5. General requirements... 7 A.5.1. Study team responsibilities... 7 A.5.2. Professional development provider responsibilities... 8 B. TECHNICAL SPECIFICATIONS... 9 B.1. Statement of work... 9 Task 1. Project kickoff... 9 Task 2. Site recruitment assistance... 9 Task 3. Finalize training and professional development materials (as needed) Task 4. Finalize staffing, training, and coaching plans Task 5. Professional development implementation and monitoring Task 6. Project management C. PROPOSAL REQUIREMENTS C.1. General instructions C.1.1. Questions C.1.2. Intent to bid C.1.3. Proposal submission ii

3 C.1.4. Minimum qualifications C.1.5. Technical proposal review criteria C.1.6. Proposal review process C.1.7. Budget and contract structure C.2. Technical proposal requirements C.2.1. General instructions C.2.2. Content and organization of technical proposal C.3. Business proposal requirements C.3.1. General instructions C.3.2. Content and organization of business proposal REFERENCES ATTACHMENT A. BUDGET TEMPLATES Budget Template 1. Payment Schedule By Task and Deliverable Budget Template 2. Cost Per Principal Budget Template 3. Itemized Budget with Staffing ATTACHMENT B. CONTRACT FLOWDOWNS iii

4 OVERVIEW OF REQUEST FOR PROPOSALS Mathematica Policy Research, Inc. invites providers of principal professional development to submit proposals to implement their program in multiple school districts across the country as part of a large-scale, randomized controlled trial evaluation funded by the Institute of Education Sciences in the U.S. Department of Education. The evaluation will provide rigorous evidence on the effectiveness of principal professional development and practical guidance to policymakers, states, and school districts. The impact evaluation will be conducted by Mathematica, with its partners American Institutes for Research (AIR), Social Policy Research Associates (SPR), and Pemberton Research (collectively, referred to hereafter as the study team ), under U.S. Department of Education Contract No. ED-IES-14-C Proposals shall be evaluated on the basis of the quality of the proposed principal professional development program and associated evidence of program (or its underlying components) impact or influence on relevant outcomes. Because the program will be implemented in 50 elementary schools spread across 10 U.S. school districts selected by the study team, evidence of organizational capacity to provide professional development across multiple school districts will be required. After a review of written proposals, highly rated offerors will be invited to Washington, DC, to present their program and training plans. The study team, in consultation with ED, will then select one offeror s program for inclusion in the study through a firm fixed-price subcontract to Mathematica. 1

5 A. BACKGROUND This portion of the RFP provides information on the motivation for the evaluation of principal professional development programs, the professional development program desired for inclusion in the study, and the evaluation design that will test the effectiveness of the implemented program. A.1. Project rationale Research suggests that school-level leadership is the second most important school-level factor influencing student learning (Hallinger and Heck 1998; Harris et al. 2010; Knapp et al. 2006; Leithwood et al. 2004), and teachers commonly identify the principal as a strong factor influencing their instructional and career decisions (Ingersoll 2001). Studies point to principals roles as talent managers, organizational change agents, and instructional leaders as influencing teaching and learning (Porter et al. 2008; Walters et al. 2004). Research also indicates, however, that few principals actually engage in practices consistent with these roles for example, spending minimal amounts of time on instructional leadership activities (Grissom et al. 2013; Loeb et al. 2010; May and Supovitz 2011; May et al. 2012; Murphy 1990). Furthermore, few principals are trained to coach teachers on instructional improvement, evaluate teacher progress, or use value-added growth measures and other teacher effectiveness data for human capital decisions (Goldring et al. 2012; Murphy et al. 2013). Unfortunately, little is known about the impact of principal professional development programs on principal practice, school culture, teaching, and learning (Huff et al. 2013). This evaluation is designed to inform policymakers and stakeholders about promising practices to improve principal performance. A.2. Theory of action The implementation of principal professional development programs is intended to improve student outcomes (see Exhibit 1). The logic model begins with the intervention itself (Box A). This professional development program may vary in structure (such as formal group training, one-on-one coaching, and the use of supplemental resources) and content (such as aspects of instructional leadership, human capital management, and organizational leadership). The receipt of the professional development program should lead principals to adopt or improve a broad set of leadership practices (Box B), such as improved instructional leadership, organizational leadership, and human capital management. Principals improved leadership practices should lead to teacher changes (Box C), such as teachers receipt of high quality professional development and feedback. Principals improved leadership practices and teacher changes together should bring about schoolwide changes (Box D), including improvement in school climate and culture. The teacher and schoolwide changes together should improve certain teacher outcomes (Box E), including instructional quality and retention of effective teachers. Schoolwide changes and teacher outcomes may directly influence student outcomes (Box F), including student achievement, behavior, and attendance. Finally, contextual factors (Box G), such as district policies, principal autonomy, and resource constraints, may influence inputs and outcomes in this logic model. 2

6 The logic model shown here is for illustrative and research purposes, and should not be interpreted as indicating specific requirements for the professional development provider selected for this evaluation. The next section (A.3) describes the specific requirements for the principal professional development to be provided in this evaluation. Exhibit 1. Logic model for principal professional development A.3. Training and ongoing support We are seeking principal professional development program providers that (1) have a program that provides intensive, comprehensive formal group leadership learning experiences and individual coaching; (2) have displayed a capacity to implement the program across multiple school districts with fidelity, and (3) can demonstrate a research-based rationale for their program s design or previous evidence of their program s effectiveness. All proposed curriculum and other supportive materials must already exist and must have been previously implemented; no funding will be provided for curriculum, technology, or other substantial program development. However, to ensure that the program is cohesive and meets the needs of the evaluation, the RFP will allow for modest customization or adjustment of training materials. A.3.1. Content An eligible program must include content from at least two and preferably all three of the following content areas: Instructional leadership. This area focuses on aspects of principals leadership practices that help teachers improve instruction. Such support might focus on performance data use, teacher evaluation, feedback and coaching on instruction, instructional planning support, curriculum materials selection, and curriculum alignment with state standards. Human capital management. This area focuses on principals roles in developing and managing the school s workforce, which includes teachers and other staff. Such support might include recruiting, hiring, and retaining effective teachers; selecting professional development tailored to teachers needs; effectively assigning teacher talent to students; and establishing pathways for developing teacher leaders and assistant principals as instructional leaders. Organizational leadership. This area focuses on principals roles in advancing organizational development. Such support might focus on setting a shared school mission; creating a safe and respectful environment for learning; improving school climate and 3

7 culture; fostering communication and collaboration among teachers and parents; distributing leadership responsibilities; ensuring efficient use of available funding and instructional time; and deploying resources aligned with strategic goals. A.3.2. Structure and duration The professional development program must provide two types of learning and development opportunities for current school principals over a 12- to 16-month period: (1) formal group training, and (2) one-on-one individual coaching. The formal group training shall be delivered in person before the start of the school year and in person and/or online during the school year (and possibly also during the school year, depending on the proposed length of the program). These formal group training sessions must be based on a defined curriculum. The proposal must include a rationale for the mix of online and in-person group sessions proposed and how each learning modality allows for optimal outcomes and engagement. The program also must provide one-on-one coaching 1 throughout the school year to support principals as they apply their professional development to the specific needs of their schools. Coaching also must provide opportunities for principals to receive individual feedback on their leadership practice and on the implementation of strategies learned in the group sessions. In addition, providers may offer supplemental supports and resources. Similar to the required components, any supplemental supports must be fully described including a rationale for the activities proposed. A.3.3. Participants Offerors programs shall be provided to 50 elementary school principals in districts selected by the study team (NOTE: selection criteria include schools that are not currently providing extensive professional development to principals. We anticipate that a mix of novice and experienced principals may participate). Although the professional development provided as part of this evaluation shall focus on principals and improving their practices, the study team recognizes that some professional development programs are typically offered to a school leadership team. Therefore, if the offeror proposes to include support for nonprincipals (for example, school leadership team members, assistant principals) in the professional development program (in addition to principals), the offeror must make a compelling case for the proposed approach. The study, however, will focus on the impact of the program on principals, teachers, schools, and students. A.3.4. Intensity The desired program for this evaluation must be intensive. To that end, the program must include: 1 For the purposes of this evaluation, coaching is defined as supporting a specific set of leadership practices addressed by the principal professional development program through one-on-one feedback. 4

8 A minimum of four days of in-person formal group training for each principal, delivered in summer 2015 A minimum of 50 hours of one-on-one leadership coaching for each principal, delivered throughout the school year (and possibly also extending into the school year, depending on the proposed length of the program) A minimum of four formal group trainings provided in person or online, delivered in the school year (and possibly extending into the school year, depending on the proposed length of the program) These requirements are minimum requirements, and additional in-person or online resources and support also may be proposed. Preference will be given to programs with greater intensity, within the current study resource constraints. Offerors must justify the intensity of their program, and describe how the professional development can be provided realistically within the context of principals schedules and how the intensity is intended to lead to the desired outcomes. These minimums should not be interpreted as signifying the desired level of intensity, because we recognize that many existing programs exceed these minimums. A.3.5. Capacity The offeror must demonstrate a minimum of three years of experience providing intensive professional development and coaching to principals. The offeror also must have available, trained staff with professional development expertise capable of providing professional development and coaching to a minimum of 50 principals across approximately 10 districts geographically dispersed across the US. The offeror must have processes for hiring, training, monitoring, and supporting trainers and coaches to ensure a high quality program. In addition, the offeror must have the ability to coordinate services with school districts and the study team and be willing to work in the context of an evaluation. A.3.6. Replicability/consistency of implementation The offeror must demonstrate the ability to deliver its program with fidelity in varied locations including, availability of a written curriculum and coaching model, written training materials/processes, and the use of quality assurance processes for evaluation of personnel (such as coaches/trainers). A.3.7. Evidence of effectiveness Given the limited research on principal professional development programs and coaching, the most promising professional development program will be selected. The offeror must demonstrate a research-based rationale for their program s design and/or empirical evidence supporting the promise of the current professional development program for positively impacting principals, teachers, schools and/or students. A.4. Description of impact evaluation This evaluation is designed to provide rigorous evidence on the effectiveness of the selected principal professional development program. According to the study design, the study team will randomly assign participating study schools to a treatment group whose principals are offered the 5

9 intensive professional development program or to a control group whose principals are not offered the professional development program. A comparison of school, principal, teacher, and student outcomes between the treatment and control groups will provide rigorous evidence of the effects of the selected professional development program on intermediate and longer-term outcomes. To ensure that the study provides useful context for policymakers and school districts, findings will include detailed analysis of the fidelity with which the principal professional development program was implemented and challenges encountered in its implementation. The study will include 100 elementary schools within high-need districts. 2 There will be approximately 10 districts, and random assignment will occur within each district. In total, 50 schools will get the treatment (approximately five per district), and 50 schools will comprise a business-as-usual control group (approximately five per district). Although many districts provide some professional development to principals, the evaluation will seek to include only districts and schools that currently provide minimal support and funding for principal professional development. The study team will recruit the districts and schools to participate in the evaluation by May The study team will work closely with school districts when recruiting them to ensure that participant learning conditions (for example, time for professional development, autonomy to apply learning) are consistent with the program s goals and the evaluation. To help ensure a high quality implementation, the study team has resources to independently monitor implementation throughout the evaluation and will work closely with the provider to discuss and address implementation challenges identified by the provider or the study team. Exhibit 2 shows the study s research questions, as well as the data sources and analysis methods planned to address each question. The study team will collect data for school years and We anticipate that recruiting for the study will focus on districts that have at least 20 high-poverty elementary schools (defined as those with at least 40 percent of students eligible for free or reduced-price lunch). However, schools already receiving School Improvement Grant (SIG) program funding would not be included in the eligible sample of schools, because it would not be possible to disentangle the effects of SIG from the effects of the principal professional development provided as part of this evaluation. 6

10 Exhibit 2. Research questions, data sources, and analysis methods for the proposed study Research questions Data sources Analysis methods 1. What are the principal professional development experiences and initial impacts on school climate and educator behaviors (for example, hiring and recruitment activities, staffing decisions and strategies, interactions with teachers and leadership in other schools and the district, educational support and activities provided for teachers)? 2. What are the impacts of the program on teacher retention, the effectiveness of instructional staff, and student achievement? Professional development participation forms Coaching logs Professional development observations End-of-session principal surveys Annual principal surveys Principal time use logs Annual teacher surveys Teacher assignment data Teacher and principal performance data Student test scores Descriptive analysis of principal professional development activities, experiences, and challenges Regression-adjusted differences in school climate and educator behaviors between treatment and control group schools Regression-adjusted differences in teacher retention, teacher effectiveness, and student achievement between treatment and control group schools Methods that account for principal turnover A.5. General requirements This section presents a general overview of roles and requirements. A.5.1. Study team responsibilities Support of district and school staff time. The study team will be responsible for the cost of any district or school staff time associated with evaluation activities (for example, survey completion and survey administration). District and school recruitment and random assignment. The study team will recruit and select the districts and, within districts, randomly assign participating schools and their principals to either receive the professional development program or continue with typical activities (business as usual). At this time, we do not know which districts will participate in the study. Therefore, offerors shall assume that the participating districts will be geographically spread across the United States. Data collection for the evaluation. The study team will develop the data collection instruments described in Exhibit 2 and collect all outcome and implementation data for the evaluation. This includes independent data on the fidelity of implementation of the professional development activities. District support. The study team will work with district administrators to ensure that principals can attend the study s professional development and implement lessons from it to the greatest extent possible throughout the intervention. 7

11 A.5.2. Professional development provider responsibilities Professional development. The selected provider shall finalize training activities and schedules; prepare and provide all training, curriculum, and coaching materials; and provide formal group training, ongoing coaching, and/or other support. The provider shall be responsible for providing all professional development staff and ensuring that all professional development staff (including trainers and coaches) have appropriate credentials, experience, and preparation. The provider also shall closely monitor implementation fidelity of training and coaching. The provider shall be responsible for providing all professional development materials and online access to materials. The provider shall pay for the cost of transportation and lodging of its own training staff, as well as any provider staff time. Formal group training can be provided within each district or at a central location. If formal group training is provided at a central location, at least two sessions on different dates must be made available to participants to maximize attendance. As needed, the provider shall pay for the cost of air, ground transportation, lodging, and meals for all participants attending formal group trainings. Providing information to districts. As needed, the provider shall help the study team share information on the program with potential districts and schools during recruitment. Coordination. The provider shall coordinate professional development activities across participating school districts, in collaboration with identified district staff and staff from the study team. This coordination shall include scheduling activities related to professional development, locating and contracting facilities, inviting participants, and collaborating with the study team staff to collect information on participation in all professional development activities. 8

12 B. TECHNICAL SPECIFICATIONS This portion of the RFP provides information on the tasks to be undertaken and their anticipated timeline. It also presents requirements for the content, organization, and formatting of proposals. B.1. Statement of work The selected provider shall begin professional development during summer 2015 and provide coaching, other group trainings, and/or other supports across the school year. Professional development activities may extend into summer/fall 2016, depending on the length of the intervention proposed and start date, although all programs must be between 12 and 16 months long. The tasks and deliverables outlined in this section shall be the responsibility of the professional development provider. (See Exhibit 3 for a full list of anticipated deliverables and due dates, and Exhibit 4 for a detailed timeline.) Task 1. Project kickoff Senior provider staff shall meet with study team members and staff from ED within 14 working days of the award of the contract (anticipated award on December 5, 2014). The meeting shall be held in Washington, DC. The purpose of the meeting shall be to review the contract activities and timeline and discuss any areas of concern. The study team shall provide an agenda for this meeting. At least six working days before the meeting (within eight working days of contract award), the provider shall deliver to the study team five copies of all existing program-related materials, such as scope and sequence documents, brochures, agendas for formal group trainings and coaching sessions, coaching manuals (or topics, if manuals are unavailable), and training presentation materials. These materials must provide the study team and ED with comprehensive information on the professional development program and training steps. 3 Deliverables: Training and coaching materials Task 2. Site recruitment assistance The provider shall provide two documents that the study team will use during the recruitment process: (1) a program brochure that describes the features, content areas addressed, and implementation requirements of the professional development program; and (2) a frequently asked questions (FAQ) document to address questions that district staff and principals may have about the professional development program. The provider shall submit a draft of these materials to the study team within eight working days of the contract award date. The study team and ED will work with the provider to finalize. Although the study team will be responsible for recruiting districts and schools for participation in the impact evaluation, districts and schools probably will have questions regarding the principal professional development program. When requested by the study team, 3 Sample program materials appended to offerors technical proposals (see section C.1) are intended to provide the review panel with snapshots of the content of the program. 9

13 the provider shall review recruitment materials developed by the study team and participate in recruitment calls made by the study team to districts and schools to further explain the program. For budgeting purposes, offerors shall estimate that they will participate in approximately 20 one-hour recruitment calls. Deliverables: Program brochure FAQ Participation in recruitment calls (up to 20) Task 3. Finalize training and professional development materials (as needed) ED intends to study an existing professional development program. However, there may need to be some adjustment of the training materials to ensure that the overall training is cohesive and appropriate for the evaluation. If needed, the provider shall make adjustments to the existing training and professional development materials so that they meet the requirements of the evaluation and reflect any modifications to the training or materials during the negotiation process and discussions among the provider, the study team, and ED. The provider shall submit revised materials by April 1, Deliverables: Revised training materials Task 4. Finalize staffing, training, and coaching plans 4.1. Staffing plan. Although the provider s proposal prepared in response to this RFP must detail its overall management and staffing plans, the provider shall submit a final staffing plan which updates the original plans once the locations of the study districts are known (and indicates whether and how the selected locations affect their original staffing plan). The staffing plan shall describe which trainers and coaches shall be assigned to each planned formal group training session and individual principal (for coaching). The plan shall also describe who provides supervision to trainers and coaches. Should the provider s proposal include intervention supports beyond the school year, the final staffing plan must give details for the full term of the intervention. As district participation is confirmed, the provider shall submit districtspecific staffing plans on a rolling basis. The provider must submit a final and complete staffing plan including information for all districts by June 15, The provider must communicate any changes in key staffing that may be necessary to the study team at least two weeks before the change being implemented and must receive approval for the changes from the study s director. Deliverables: Final staffing plan 4.2. Formal group training and coaching plan. Although the selected provider s proposal developed in response to this RFP must detail how the formal group trainings and coaching are to be conducted in the districts (i.e., how many formal group training and coaching sessions will be held, the sequence and timeline for sessions, and the content of sessions), the provider shall submit a final plan that fleshes out the specific dates or date ranges and proposed locations for specific training activities once the locations of the study districts and names of the treatment principals are known. In addition, the final training plan shall reflect any changes in the training plans as a result of the negotiation process stemming from responses to this RFP and discussion 10

14 among the selected provider, the study team, and ED. As district participation is confirmed, the provider shall submit district-specific formal group training and coaching plans on a rolling basis. The provider must submit the final training plan and materials, which should include the final dates for any formal group trainings to take place in summer 2015, to the study team by June 15, Deliverables: Final formal group training and coaching plan Task 5. Professional development implementation and monitoring To evaluate the effectiveness of the principal professional development program when implemented as intended, the program must be implemented with fidelity in each of the districts and schools participating in the impact study. As part of this task, the provider shall conduct any orientation and training necessary for its own staff (e.g. trainers, coaches) to ensure that they are fully qualified to provide formal group training and coaching and to ensure that they understand the study. This task (and subtasks) also consists of providing the personnel, materials, and support necessary to carry out the professional development program, in accordance with the plans provided in the proposal and finalized in Task 4. The provider shall conduct professional development and coaching support at locations convenient for school principals and other staff as needed Provide professional development in summer The provider shall conduct formal group training sessions to provide introductory professional development support to principals (and other relevant staff, if applicable) in summer 2015 (for example, through a multiday summer institute). Study team staff will observe formal group training sessions. Deliverables: Conduct formal group summer training session(s) (summer institute or other) 5.2. Provide ongoing formal group training and one-on-one coaching support. The provider shall conduct formal group training sessions and ongoing one-on-one coaching during the school year. If the provider s planned intervention extends beyond the end of the school year (for example, if the proposed intervention is 16 months long), the provider also shall provide supports through the indicated time period. The provider shall monitor the implementation of the program to ensure implementation fidelity and use results of the monitoring to improve implementation fidelity as needed. The study team will also systematically monitor coaching to ensure it is being provided consistently and with high quality to all treatment principals across all participating school districts. The provider s coaches shall fill out and submit web-based coaching logs developed by the study team after each one-on-one coaching session to document coaching activities conducted. It is possible that the provider has a web-based coaching log system of its own that would address this evaluation data requirement. The provider may propose to provide information from such an existing data system. If so, the provider must include a description of this system and the data elements it collects. At a minimum, the study requires information about the planned dates for coaching sessions, actual dates on which the coaching occurred, length of each coaching session, topic of each coaching session, and next steps identified during each session. 11

15 Deliverables: Conduct formal group training sessions Provide coaching support Completed weekly coaching logs Task 6. Project management To support effective implementation and comply with necessary study reporting requirements, the provider shall participate in regular, weekly meetings with the study team during the implementation period. The provider shall also provide monthly progress/exception reports. These reports must (1) summarize the major activities and accomplishments of the previous month, comparing progress with the planned schedule and describing any problems encountered; (2) describe activities planned for the next month; (3) identify any significant deviations from the implementation plan; (4) describe plans for addressing any exceptions to the implementation plan; and (5) outline any decisions that may be needed from the study team or ED. Upon award, the study team will provide more information on the required format for the monthly progress/exception reports. Deliverables: Monthly progress reports Weekly meetings during implementation period 12

16 13 Exhibit 3. Deliverables Task Subtask Deliverable Description Due 1. Project kickoff Training and coaching materials Five copies of training materials; includes existing scope and sequence documents, brochures, agendas for summer institutes and other formal group trainings, agendas (or topics, if agendas are unavailable) for coaching sessions, and training presentation materials Within eight working days of the contract award date 2. Site recruitment assistance Program brochure FAQ Includes description of program, time requirements, benefits Document providing answers to FAQs regarding professional development program Within eight working days of the contract award date Within eight working days of the contract award date Review of select recruitment materials and participation in recruitment calls as needed Offeror shall budget for participation in up to 20 onehour calls. Ongoing through May Finalize training and professional development materials (as needed) Revised training materials Includes existing scope and sequence documents, brochures, agendas for summer institutes and other formal group trainings, agendas (or topics, if agendas are unavailable) for coaching sessions, and training presentation materials April 1, Finalize staffing, training, and coaching plans 4.1. Staffing plan Final staffing plan Includes number of current training and coaching staff, summary of staff qualifications and staff resumes, plans for hiring and training additional staff as needed, and assignments of staff to training locations and coaches to specific principals 4.2. Formal group training and coaching plan Final formal group training and coaching plan Includes description of which school-level staff must attend training, specific dates of training, and formats of training June 15, 2015 June 15, 2015

17 14 Task Subtask Deliverable Description Due 5. Professional development implementation and monitoring 5.1. Provide professional development in summer 2015 Conduct formal group training sessions (summer institute or other) In accordance with training plans and materials TBD 5.2. Provide ongoing formal group training and one-on-one coaching support Conduct formal group training sessions In accordance with training plans and materials TBD 5.2. Provide ongoing formal group training and one-on-one coaching support Provide coaching or other support In accordance with training plans and materials TBD 5.2. Provide ongoing formal group training and one-on-one coaching support Completed weekly coaching logs Includes the planned dates for coaching sessions, actual dates on which the coaching occurred, length of each coaching session, topic of each coaching session, and next steps identified during each session Weekly during implementation 6. Project management Monthly progress reports Includes the status of implementation activities, deliverables completed, current or foreseeable challenges, and proposed solutions Within 10 working days of the first of each month Weekly meetings By telephone during implementation period Weekly

18 15 Exhibit 4. Timeline Tasks and Deliverables Nov De c Ja n Fe b Ma r Apr Ma y Jun Jul Aug S e pt Oc t Nov De c Ja n Fe b Ma r Apr Ma y Jun Jul Aug S e pt Oc t Nov De c Task 1: Project kickoff Training and coaching materials Task 2: Site recruitment assistance Program brochure FAQ Review of select recruitment training materials and participation in recruitment calls as needed Task 3: Finalize training and professional development materials Revised training materials Task 4: Finalize staffing, training, and coaching plans Final staffing plan Final formal group training and coaching plan Task 5: Professional development implementation and monitoring 5. 1 Provide professional development in summer 2015 Conduct formal group training sessions 5. 2 Provide ongoing formal group training and one-on-one coaching support Conduct formal group training sessions Provide coaching or other support Completed weekly coaching logs Task 6: Project Management Monthly progress reports Weekly meetings Note: Depending on the length of the program, Tasks 5 and 6 may extend into the summer or fall of 2016; this optional period is represented by the yellow shading.

19 C. PROPOSAL REQUIREMENTS C.1. General instructions Providers are encouraged to submit their best offer, as they may not have an opportunity to revise their proposal, and awards may be made without discussion. Each offer must consist of two separately packaged, bound proposals: a technical proposal and a business proposal. All information necessary to judge the technical soundness, staffing, and management capabilities of the vendor must be contained in the technical proposal. The technical proposal must not refer to pricing data. The technical proposal must include enough detail so that an expert review panel can effectively assess the technical quality of the offeror s professional development program. Simply restating the requirements of this RFP will not be sufficient. The offeror s proposal must clearly describe: the covered content areas of the professional development program as indicated in section A.3.1 (instructional leadership, human capital management, and organizational leadership), including illustrative examples of professional development materials; how the program integrates content across the various program activities (initial and ongoing formal group training and individualized coaching); how the particular focus and curriculum will alter leadership practices and improve school, teacher, and student outcomes (see Exhibit 1). The technical proposal also must demonstrate that the provider has the knowledge, capacity, and experience necessary to provide high quality training, coaching, and support to ensure implementation fidelity on the scale the study requires. Specifically, the offeror must describe: how coaches are selected, trained, and monitored to ensure that high quality coaching is provided; how coaching sessions are designed and monitored to ensure consistency across coaches. C.1.1. Questions A copy of the solicitation will be available online at and at Questions regarding this RFP shall be sent to Ben Martinez at AIR by (bmartinez@air.org) no later than September 16, Responses to technical questions shall be provided online at and no later than September 24, It is the responsibility of offerors to check this site regularly to see if questions and answers have been appended to the solicitation. If you would like to receive an notifying you when responses to questions have been posted, please let Ben Martinez know at the address above. Submitting questions represents an opportunity for vendors to clarify information; however, there is no requirement to submit questions. 16

20 C.1.2. Intent to bid For study team planning purposes, offerors shall submit notification of intent to bid to Ben Martinez at AIR by no later than 5:00 p.m. Eastern Daylight Time on Friday, September 26, Failure to submit an intent to bid will not disqualify providers from consideration; however, submission of an intent to bid is strongly encouraged. C.1.3. Proposal submission An original and two copies of the written technical proposal and an original and two copies of the business proposal, as well as electronic versions of the technical and business proposals, must be submitted no later than 5:00 p.m. Eastern Standard Time on October 14, Offers received after the deadline for proposal submission will not be considered. The hard copies of the proposal (technical and business) must be mailed to the following address: Ben Martinez American Institutes for Research 1000 Thomas Jefferson Street, NW Washington, DC (Phone: ) The electronic copy of the proposal (technical and business) must be ed to Ben Martinez at: The anticipated award date is December 5, C.1.4. Minimum qualifications Only complete proposals received by 5:00 p.m. Eastern Daylight Time on October 14, 2014 will be reviewed. Proposals that do not include all of the requested technical and financial information will not be considered. C.1.5. Technical proposal review criteria Technical proposals that meet minimum requirements will be evaluated against the criteria specified in Exhibit 5. Each criterion will be weighted as indicated, for a total of 100 points. 17

21 Exhibit 5. Selection criteria Criteria Description Points Quality of proposed program Corporate capability Management approach and staffing plan Staff qualifications Evidence of promise and effectiveness Proposals will be evaluated according to the quality of the proposed program and ongoing support activities and their relationship to intended outcomes. Proposals will be evaluated on the basis of evidence of having provided similar training and support activities in school districts and corporate capacity to provide the proposed activities at the scale required by the study. Proposals will be evaluated on the basis of the inclusion of a credible plan for management and oversight of tasks, as well as inclusion of a welldeveloped staffing plan which demonstrates that proposed commitments of staff are adequate for the proposed work and that the offeror has sufficient staff or capability to obtain needed staff to provide professional development and coaching in all districts. Proposals will be evaluated on the extent to which staff qualifications match staff responsibilities and staff members have adequate breadth and depth of experience. Proposals will be judged on the basis of the program design and evidence of program impact or influence on relevant outcomes C.1.6. Proposal review process Proposals will undergo a three-stage review process. In the first stage, the study team will review proposals to ensure that they are complete and meet the stated requirements of the RFP with respect to budget and technical material. In the second stage, based on the criteria shown in Exhibit 5, an expert review panel will review all complete proposals and recommend up to three proposals for final consideration. The third stage will consist of written responses to questions and oral presentations by the finalists, after which the expert review panel will make a final recommendation. The in-person presentations will be held in Washington, DC and the study team will cover travel costs for up to two staff from each finalist. The final selection of the provider will be made by the study team in consultation with ED and the expert review panel. Costs will be evaluated for congruence with the quality of the proposed work, as long as the total budget is within the $4 million allocated for this scope of work. Strong proposals must have a realistic budget for each of the tasks, with a detailed budget justification. Price will be a factor in selection; however, program quality will also be an important factor in the selection process. The study team will assess proposals for the best program quality and pricing combination. C.1.7. Budget and contract structure This study is funded by the U.S. Department of Education under Contract No. ED-IES-14- C-0028 (the Prime Contract). The successful offeror will be issued a subcontract by 18

22 Mathematica, which will be governed by the applicable Federal Acquisition Regulations (FAR) and the required terms and conditions of the Prime Contract. The terms and conditions of the Prime Contract are appended to this RFP. The contract will be structured as a firm fixed-price contract (with payments tied to deliverables). The maximum funding available for the proposed work is $4 million. C.2. Technical proposal requirements C.2.1. General instructions The technical proposal (text plus all figures, charts, tables, and diagrams) is limited to 30 single-sided pages, not including the title page and table of contents. Technical proposals must be bound separately from business proposals. Specified appendices will not count as part of the 30 pages. All text must be double-spaced, 12-point Times New Roman font with standard character spacing; exhibits can use 10-point Times New Roman (or larger) font, but should remain clear and easy to read. Pages should be 8.5 by 11 inches, with a 1-inch margin along all four sides. Page numbers must be included. Any materials submitted outside of the 30-page limit (other than the exceptions noted) will not be reviewed. C.2.2. Content and organization of technical proposal The technical proposal shall include the following content and be organized into the following sections: Title page. The title page must include (at a minimum) the name of the provider; the name, title, and contact information of the proposal author(s); and contact information for a person with authority to negotiate for the provider. Table of contents. The table of contents shall provide an easy means to locate each section of the proposal. Introduction. The introduction shall briefly present the provider s approach and understanding of the goals, processes, and products of the project. Description of program. This section shall describe in detail the key components of the proposed existing program, clearly indicating its content, intensity and duration, structure, and participants in the program (all program features described earlier in this RFP) and how the program delivers its content. The offeror must provide a convincing rationale for how the proposed program is intended to affect the outcomes to be measured in the study. The rationale must be research based and reference relevant research and evaluation studies in support of the program s promise/effectiveness. This section also shall include a logic model or theory of action (with an illustration) for the proposed program. The logic model shall be a well-specified conceptual framework that stipulates the following: - The key components (activities and inputs delivered by the provider and considered by the provider to be essential in implementing the program) of the proposed program, and how the key components connect to form a coherent program - The target participants 19

23 - The expected pathways/mediators for effects on teachers and students - The specific mechanisms of change (how aspects of the proposed program, such as content, organization, length, and intensity, will improve relevant outcomes) This section must specify whether and how all elements of the program (formal group trainings, coaching, etc.) have been previously implemented together. Any materials or products associated with the implementation of the program shall be described in this section, and any examples of these products/materials shall be included in Appendix A. This section shall describe each role required of district and school staff (for example, principals, leadership team, principal supervisors) in implementing the program. Description of evidence of effectiveness. This section shall include a brief summary of any previous research on the proposed program, or descriptions or citations for available empirical evidence to support the theoretical linkage between at least one critical component of the program and at least one relevant outcome presented in the logic model. If direct research on the proposed program is available, providers also shall include in Appendix B copies of up to five of the most relevant publications and reports they wish to be used to evaluate the evidence of effectiveness. Statement of work. This section shall provide a detailed discussion of how each task outlined in Section B.1 of this RFP shall be completed. It shall expand on each of the tasks, including a discussion of procedural issues related to completing each task. The description of activities must be supported by as many examples of materials (including agendas, schedules, training materials, and manuals) as necessary to demonstrate the quality of the activities and materials. As noted, all sample materials shall be included in Appendix A. The section on Statement of work shall also describe how the professional development program shall be implemented consistently and with fidelity across school districts. The plans for supporting implementation fidelity must describe the information that the provider shall use to monitor training and coaching components, determine if the training and coaching goals have been met, and, if not, what activities shall be used to remedy the relevant problems. Specifically, the description of Task 4 shall include details on any processes to monitor, support, and evaluate the quality of training and coaching. The offeror shall discuss how participant attendance and engagement in professional development (formal group training and coaching) will be maximized, and how non-attendance is typically handled. The description of Task 5 shall include the purpose, type, sequence, and timeline of the proposed professional development, coaching, and supports. Management approach and staffing plan. o Management approach. In this section, providers must describe how they shall successfully execute the tasks described in the statement of work. This section shall detail the provider s overall management plan, as well as lines of authority, coordination, and communication within the provider organization, and between the provider organization and the study team. This section also shall include a project management chart showing the timelines for all major tasks and subtasks, with start and completion dates for each task, as well as intermediate dates for any 20