Early Outcome Evaluation of Bay Area Schools Israel Synergy (BASIS): The Israel Education Project in the San Francisco Bay Area Jewish Day Schools

Transcription

1 Early Outcome Evaluation of Bay Area Schools Israel Synergy (BASIS): The Israel Education Project in the San Francisco Bay Area Jewish Day Schools April 28, 2011 P19153 Prepared for: Charles Edelsberg, Executive Director Jim Joseph Foundation Submitted by: Shari Golan Marjorie Wechsler Tracy Huang Nicole Fabrikant Lynn Newman

2 Contents Executive Summary... i Introduction... 1 Overview... 1 Data Collection Methods... 2 The BASIS Model... 5 Implementation of Project Activities... 7 Activity 1: Provide Professional Development... 7 Activity 2: Update Israel Education Vision Statements and Develop Goals Activity 3: Develop Long-term Israel Education Plans Activity 4: Implement New and Enhanced Israel Education and Engagement Strategies with Students Educator Short-Term Outcomes Student Short-Term Outcomes Student Learning Student Connections to Israel and the Jewish Community Other Outcomes Integration of Israel Education into School Infrastructures and Cultures Establishment of a Network Across Jewish Day Schools Discussion Conclusion Appendices A: Schools Israel Visions and Enduring Understandings B: Sample Curriculum Maps C: Teacher Survey Results D: Student Post-Survey Results

3 Executive Summary The Bay Area Schools Israel Synergy (BASIS), the Israel Education Project funded by the Jim Joseph Foundation (JJF), is an innovative initiative to improve and institutionalize Israel education in Bay Area Jewish day and high schools. BASIS strives to strengthen students knowledge of and connection to Israel by incorporating best practices in Israel education and engagement through a process of school improvement and change. The JJF has funded the San Francisco Bureau of Jewish Education (BJE) to provide management and professional consultation for 11 BASIS project partner schools (on 12 campuses). BASIS encourages the schools to design and put into place a comprehensive long-term plan to incorporate Israel education into every aspect of the school s community students, faculty, and administration to think critically about the Israel education programs they currently offer, to determine how current programs fit into that long-term plan, and to design new programs, as needed, to meet the goals of the new plan. SRI International was contracted by JJF to conduct an evaluation of BASIS. This report presents our early summative findings, including the extent to which BASIS has fully implemented the project activities and achieved the short-term outcomes posited in its logic model. The findings are based on data from school site visits and faculty interviews, student and teacher surveys, school-based assessments of student learning, interviews with BASIS team members and consultants, and school progress reports and proposals. A December 2010 report, The Israel Education Project in the San Francisco Bay Area Jewish Day Schools: Summary of School Site Visits, presented formative findings about project management and support, the most useful resources and supports for schools, and challenges schools encountered in planning and implementation. These are the key findings of the evaluation to date: Professional development, coaching, and consultation and cross-school collaboration were instrumental in helping schools focus on Israel education and learn about resources and strategies to improve their Israel education. Activities were coordinated and enhanced by the part-time coordinator at each school. Schools are planning Israel education in ways that are more focused and outcome-based than they were before BASIS. Schools have implemented new and enhanced student-focused Israel education and engagement strategies. Many of the teachers (especially the 44% directly involved in BASIS) have increased their confidence, skills, access to resources, and frequency of instruction in Israel education. Students have experienced increases in connections with people living in Israel. Israel education has become more a part of the school culture for all the schools. Most schools are still developing their scope and sequence for Israel education so more time will be needed to see whether these plans lead to coordinated and comprehensive Israel education in a sustainable way. The progress schools made on project implementation and the early outcomes achieved to date are summarized below and detailed in the full report. i

4 Implementation of Project Activities The BASIS team led by the BJE provided school staff with professional development on how to design and implement high-quality Israel education. The BASIS team supported the professional development of staff at each of the Jewish day and high schools through multiple workshops, forums, and consultation from a team of Israel education experts. Workshops and forums. A total of 10 workshops and forums were held for school staff to learn new pedagogical skills and teaching content, hear about new resources, strengthen their teams commitment to Israel education, and work on their individual school plans. These professional development opportunities included local and Israelbased forums and themed workshops on specific Israel education strategies and curriculum development. Consultation. A team of consultants from the BJE with expertise in arts and culture, technology, family education, and twinning supported the schools individually in planning and implementing their Israel education strategies. In addition, schools received one-on-one consultation from a curriculum coordinator and curriculum coaches (manchim). Manchim are Israel education experts from universities and other educational institutions who were carefully matched with schools based on similar perspectives of Israel education and Jewish identity. They support schools with developing a curricular scope and sequence. Manchim have been credited with keeping schools on task and focused on identified goals, supporting staff as they work through challenges encountered when developing a schoolwide sequenced Israel education curriculum, modeling effective Israel education practices, organizing and facilitating collaborative faculty meetings, developing systems to gather input, and analyzing current Israel education activities in reference to schools new Israel education visions and goals. Community of Practice meetings. Networking and sharing of learning was facilitated through 17 Community of Practice (CoP) meetings for BASIS coordinators that were held between November 2008 and February School-based professional development. School-based and school-designed professional development workshops and teacher trips to Israel also were used to increase teachers knowledge and excitement about Israel. Participation in BASIS helped schools update their Israel vision statements and develop goals for the types of engagement they want students to have with Israel. To develop a long-term plan for Israel education, schools developed Israel education vision statements and clarified and defined specific learning goals. Vision statements. As of October 2010, all schools had developed a final or draft Israel education vision statement, and most had developed a set of enduring understandings and essential questions to guide their curriculum work. Interviewed school staff members report using these documents to guide their Israel education planning. See Appendix A for examples of vision statements and enduring understandings. Goals and learning objectives. Schools have clarified and defined their goals for Israel education, which has enabled them to develop more intentional, coordinated, and comprehensive plans for Israel education and engagement. Interviewed school staff ii

5 reported that working backward from the learning goal (the desired enduring understanding) and selecting or developing Israel activities that will produce those learning outcomes has required a paradigm shift in how school faculty members think about Israel education. They learned this approach from the BASIS professional development programs and consultants. Participation in BASIS helped schools to systematically make progress toward the development of long-term Israel education plans that are outcome-based, comprehensive, and sequenced. To develop long-term Israel education plans, schools mapped their existing Israel education activities and lessons, compared their existing program with their newly refined Israel education statements and goals, identified existing lessons and activities that were still appropriate and those that were not, and identified gaps where additional lessons needed to be developed. Example curriculum maps are presented in Appendix B. Mapping. As of October 2010, most of the schools had inventoried their Israel education and engagement lessons and activities and entered their mapping information into the Atlas curriculum mapping software. Through these activities, schools identified some of their implicit priorities for student learning, redundancy in current activities, alignment of current activities with their identified enduring understandings, and gaps in coverage of these important ideas. Scope and sequence. As of October 2010, a few schools had begun to outline their scope and sequence and associated curricular units. Most schools planned to work more intensively on their scope and sequence from January to June The planning process for developing a scope and sequence has changed how teachers in the schools are collaborating. For many schools, BASIS has inspired links across faculty groups such as Judaic and general studies teachers who previously functioned more independently. Through BASIS, schools have implemented new and enhanced student-focused Israel education and engagement strategies. Schools are using a variety of strategies twinning and partnerships programs, Israel trips, family education, shlichut (Israeli emissaries), arts and culture, and technology to engage students in Israel education, with the majority of schools using multiple strategies. Implementing more types of strategies is not inherently better, however, because some strategies are more comprehensive than others. Further, determining the number and type of strategies for schools to use is a work in progress as schools think about how to use these strategies to support their new scope and sequence for Israel education. Twinning and partnerships. Eight of the schools are working on developing or enhancing their twinning and partnerships strategy for Israel education and engagement, including coordinated activities and visits with their twin/partner school in Israel. Other student travel to Israel. Two schools have developed or enhanced trips to Israel to increase students knowledge of and connections with Israeli culture and Jewish Peoplehood. Family education. Eight of the schools are conducting family education activities, such as schoolwide events celebrating arts and culture, to support Israel education and engagement. iii

6 Shlichut. Three schools have shlichim, Israeli citizens who serve as a cultural and educational link between the local school communities and the people of Israel. Arts and culture. Nine schools are implementing new and enhanced arts and culture activities, such as creating public art on school campuses or exploring aspects of Israeli arts and culture, to support Israel education and engagement. Technology. Four schools are using technology to develop new and enhanced ways to build students knowledge of and connections with Israel. Short-Term Educator Outcomes Through a teacher survey that collected data from 251 teachers across the 11 schools (a 68% response rate), SRI assessed teachers involvement in BASIS as well as short-term teacher outcomes: changes in teachers knowledge about, connection to, and engagement with Israel and teachers capacity to design, plan, and teach about Israel. Although a correlation does exist between some positive changes and BASIS, we are unable to attribute reported changes specifically to BASIS. Almost half (44%) the teachers surveyed reported being involved or very involved in at least one of six BASIS activities. The most common BASIS activities surveyed teachers reported involvement in were mapping the school s current Israel education activities and objectives (30%), planning or implementing BASIS student activities (26%), developing a teaching plan for Israel education across all grades (23%), and participating on the BASIS school leadership team (22%). Given the findings, there appears to be a critical mass of teachers who are involved and can change the way Israel education is carried out in their schools. However, more than half the teachers reported no involvement in BASIS, indicating there is room to encourage broader involvement. Teachers involved in BASIS activities tended to have the following characteristics more often than their less involved peers: Teach Jewish/Hebrew studies (59% vs. 9%) Have more than 5 years of teaching experience (63% vs. 46%) Have visited Israel (90% vs. 50%) Were born in Israel (34% vs. 5%). Analysis of the teacher survey indicated that teachers who were involved in BASIS activities showed the greatest increase in their knowledge of, connection to, and engagement with Israel as well as in their capacity to plan and incorporate Israel education in their curriculum. Thus, if the goal is to integrate Israel education into all subjects in the school, more involvement of general studies teachers will be needed. Schools will need to systematically plan how to gain greater involvement from general studies teachers who have less experience with Israel. Findings about changes in educators are described below. Faculty support for teaching about Israel is very high, and it has increased during the BASIS project. Overall, 91% of surveyed teachers reported medium or high support for teaching about Israel among their faculty. Teachers involved in BASIS, compared with teachers not involved, perceived even more support (97% vs. 86%). Many surveyed teachers reported that iv

7 faculty support for teaching about Israel increased in the past year, with those involved in BASIS more often reporting increases than those not involved (57% vs. 36%). Teachers reported having increased their knowledge of, interest in, and connection to Israel since fall These increases were greater for teachers involved in BASIS than for teachers not involved in BASIS. Teachers involved in BASIS, compared with teachers not involved, more often had medium to high levels of interest in teaching about Israel (98% vs. 54%), knowledge and understanding about Israel (94% vs. 63%), attachment to Israel and Israeli culture (87% vs. 51%), and connection to teachers in Israel (56% vs. 14%). This may be related to their more often being Jewish/Hebrew studies teachers and teachers with more connections to Israel. Teachers involved in BASIS, compared with teachers not involved, more often reported increases in these areas: interest in teaching about Israel (49% vs. 18%), knowledge and understanding about Israel (52% vs. 27%), attachment to Israel and Israeli culture (36% vs. 14%), and connection to teachers in Israel (29% vs. 8%). Teachers who were involved in BASIS were more likely to attend professional development related to Israel education and positively rate the workshops or conferences they attended. Overall, 61% of teachers participated in at least one professional development activity related to Israel education. Participation in school workshops was higher for teachers involved in BASIS compared with those not involved in BASIS (75% vs. 36%). Satisfaction with these workshops was also higher among attendees who were involved in BASIS compared with those who were not (74% vs. 37%). Given that so few of the teachers who were not involved in BASIS reported having participated in professional development related to Israel education, this might be a key way to begin their involvement in the project. Teachers who were involved in BASIS reported having more access to resources and materials for teaching about Israel and finding them more useful. Teachers involved in BASIS, compared with those who were not, had greater access to books and magazines (54% vs. 14%), computers and other technology (49% vs. 17%), visual art (46% vs. 13%), maps (45% vs. 12%), and movies (38% vs. 8%). Furthermore, teachers who were involved in BASIS and had access to materials, compared with teachers who were not involved but had access to similar materials, more often reported that the materials were useful or very useful in their classrooms: books and magazines (91% vs. 44%), computers and other technology (77% vs. 64%), visual art (84% vs. 65%), maps (98% vs. 63%), and movies (90% vs. 55%). As a result, teachers involved in BASIS increased their access to Israel education resources more than their noninvolved peers, even though they probably had greater access to these resources to start with. Capacity to teach about Israel was higher and increased more for teachers involved in BASIS than for teachers not involved. More teachers involved in BASIS, compared with teachers not involved, reported high alignment of Israel education across grades (85% vs. 50%), amount of Israel education planning done with other teachers at the school (72% vs. 14%), awareness of resources to help develop lessons about Israel (85% vs. 40%), confidence in teaching about Israel (84% vs. 38%), skills to effectively teach about Israel (87% vs. 38%), and familiarity with organizations in Israel (73% vs. 27%). v

8 Even though they already have a stronger capacity to teach about Israel, more teachers involved in BASIS, compared with teachers not involved, reported increases in capacity, including alignment of Israel education across grades (46% vs. 20%), amount of Israel education planning done with other teachers at the school (31% vs. 6%), awareness of resources to help develop lessons about Israel (30% vs. 11%), confidence in teaching about Israel (27% vs. 9%), skills to effectively teach about Israel (26% vs. 7%), and familiarity with organizations in Israel (17% vs. 5%). Because the skills and capacity to teach about Israel were much higher for teachers involved in BASIS than for peers not involved, building capacity for uninvolved teachers will require planning and support. Teachers increased the frequency with which they taught about Israel from the to school years. Overall, 4% more teachers taught about Israel frequently (i.e., at least once a week) in than they did in (27% vs. 23%, respectively). Teachers who were involved in BASIS taught about Israel more frequently, and their frequency of teaching increased more after a year of BASIS compared with that of teachers who were not involved (40% vs. 8% in and 45% vs. 12% in ). The teachers involved in BASIS have increased the frequency with which they do certain activities to teach about Israel. More than one-third of the teachers involved in BASIS reported increases since fall 2008 in using technology to teach about Israel (44%), providing experiences with Israeli artists or performers (41%), teaching about the cultures in Israel (40%), and displaying Israel-related posters, exhibits, and maps (38%). This finding may be related to the increased access to resources and growth in interest and skills among the teachers involved in BASIS. Short-Term Student Outcomes SRI assessed changes in two short-term student outcomes: (1) increasing student learning about Israel (e.g., its geography, its history, its people and their cultures, its geopolitical issues, its cultural/religious issues) and (2) increasing student connection to and engagement with Israeli people and the role Israel plays in the Jewish community worldwide. SRI used three measures to asses these short-term outcomes school assessments of student learning objectives, student surveys, and teacher surveys that included items about student interest in learning about Israel. As with the teacher findings, the evaluation design enabled us to demonstrate a correlation between some student outcomes and BASIS, but we were unable to attribute those changes to BASIS. Each school identified five learning objectives (60 in total), and all but two were assessed. In the cases where objectives covered multiple topics, we considered each topic separately. As a result, our analysis included 58 learning objectives across the 12 campuses. Schools used curricular and project-based assessments such as quizzes, projects, and papers to assess 298 students and determine the percentage of students who mastered each objective (e.g., received at least a grade of B on the assessment task). The focus of learning objectives varied widely across schools, but the level of mastery was high for all of them. Overall, of the 58 objectives schools assessed, more than half (58%) were reported to have been mastered by 100% of the students. Given the very high mastery rate, it appears that schools were very explicit in teaching the information and skills identified in their learning objectives. Teaching to the test can have a positive outcome when learning objectives vi

9 focus on important information. Exhibit ES-1 indicates the mastery level students achieved for each of the topic areas assessed. Exhibit ES-1. Average percentage of students who mastered objectives, by topic Topics of learning objectives Percentage of objectives focused on this topic Percentage of students who mastered learning objectives Israel s history Israel s geography, present and/or past Israeli culture, literature, art, or music Israel s current events and/or politics Zionism: history, philosophy, culture Connection with Israel and its role in the Jewish community Hebrew proficiency 5 97 Teachers reported increases in students learning about Israel. More than a quarter (27%) of the teachers surveyed reported increases in the past year in students ability to think about Israel from different perspectives, students knowledge about Israel (e.g., culture, politics, history, geography), and students interest in learning about Israel. Pre- and post-surveys indicated changes in students attitudes, connections, and behaviors toward Israel and Jewish Peoplehood. Surveys were administered to students in all 11 BASIS schools in fall 2009 and again in spring or fall 2010 (after a year of BASIS participation). A total of 583 students completed pre-surveys and 630 students completed postsurveys (an overall response rate of 75%). We matched 416 students pre- and post-surveys. The survey indicated a consistent pattern of increases in students connections to people living in Israel, as described below. More students visited Israel. The percentage of students who had visited Israel in the past 2 years increased from 49% at the pre-survey to 66% at the post-survey. Much of this increase was due to 26% more students having taken a school trip to Israel between their pre- and post-surveys (15% to 41%). The vast majority of students who took a school trip to Israel were in eighth grade. The fact that many eighth-grade students took a school trip to Israel may be a major factor in why the grade 7 and 8 students tended to experience greater increases in connections to people living in Israel than the grade 4 through 6 and grade 9 to 12 students. More students feel very much connected to the Jewish community and the role Israel plays in it. After a year of BASIS, students overall were more likely to report that they feel very much like they are part of the Jewish people (56% to 62% from pre-survey to postsurvey). In addition, more grade 7 and 8 students were more likely to feel very much that visiting Israel is an important part of being Jewish (42% to 51% from pre-survey to post-survey) and that vii

10 caring about Israel is an important part of their life (32% to 36% from pre-survey to postsurvey). Teachers also reported changes in students attitudes toward Israel. More than one-third (34%) of surveyed teachers reported that students level of positive attitudes toward Israel and connections to Israel had increased since fall More students feel very much connected to people living in Israel. Overall, after a year of BASIS, students were more likely to report that they feel very much connected with people living in Israel (32% to 36% from pre-survey to post-survey). Students overall also were more likely to report having a friend in Israel (60% to 71% from pre-survey to post-survey). In addition to being more likely to have a friend in Israel post-survey, students also tended to have more friends in Israel. More students expressed interest in communicating with people in Israel. After a year of BASIS, students overall were more likely to report having a very high interest in communicating with family and friends living in Israel (31% vs. 36% from pre-survey to postsurvey), and these increases were seen at both the grade 4 through 6 and grade 7 and 8 levels. Similarly, more students overall were more likely to report ing, writing letters, or calling someone living in Israel at least three times a year (36% to 45% from pre-survey to post-survey). More students reported frequent participation in some Israel-focused activities. After a year of BASIS, more students overall reported that they hear guest speakers talk about Israel frequently (i.e., at least three times a year) (37% to 44% from pre-survey to post-survey). Students in grades 4 through 6 also were more likely to report frequently going to Israel celebrations (29% to 38% from pre-survey to post-survey) and to museum exhibits focused on Israel (13% vs. 15% from pre-survey to post-survey). The frequency of student engagement in other types of Israel-focused education activities did not change between the pre- and postsurveys. Decreases occurred in learning about Israel outside school. Interestingly, at a point when Bay Area day schools are more focused on increasing students knowledge about Israel, students reported less often learning about Israel frequently (i.e., at least three times a year) from other Jewish organizations, including synagogues, temples, and shuls (50% to 45% from pre-survey to post-survey); Jewish youth groups (26% to 25% from pre-survey to post-survey); and at Jewish camps (30% to 27% from pre-survey to post-survey). Thus, increasing the amount of Israel education in schools may be of even greater importance if time devoted to it is being decreased in other educational settings for Jewish children. Other Outcomes BASIS is changing how Israel fits into the school culture and how Jewish day schools interact with each other. Israel education is now a more established component of schools cultures. Many interviewees reported that participation in BASIS had increased the presence of Israel education in their schools, from the creation of Israel education vision statements to more awareness and discussion of Israel among their faculty. There is more visual evidence of the importance of Israel in schools now. Several interviewees reported that BASIS activities have increased the visual presence of Israel as viii

11 schools now display Israel-focused art including permanent murals and temporary exhibits, informational bulletin boards, photos from student trips, and maps. Parent support for teaching about Israel has increased and is strong. Overall, surveyed teachers (89%) reported medium or high levels of support from the parent community for teaching about Israel. Also, nearly a quarter of teachers (23%) reported that parent support for teaching about Israel has increased since fall Schools have begun to proactively think about how they will sustain their BASIS activities, and all schools recently submitted sustainability reports to the BJE and JJF. Many heads of schools said during interviews that sustaining BASIS activities in the absence of BASIS funds is a concern, especially given that school budgets are often tight and schools have many competing priorities for limited funds. Therefore, they reported they are building support for their increased focus on Israel education among faculty, school trustees, and the broader school community and identifying ways to cover project costs. BASIS has developed a network that supports collaboration across the Jewish day schools in the greater Bay Area. BASIS coordinators reported during interviews that they are sharing resources and ideas across schools through discussions at CoP meetings and informally outside CoP meetings. Conclusion The evaluation data from all sources interviews, document reviews, and teacher and student surveys suggest that all participating Jewish day schools are going about Israel education in new ways and that educator and student short-term outcomes are moving in the desired direction posited in the logic model. More time will be needed, however, to determine whether BASIS results in schools developing a comprehensive scope and sequence of Israel education and a set of sustainable Israel education strategies and what the long-term impacts of these instructional changes are on students. ix

12 Introduction SRI International was contracted to support the Jim Joseph Foundation (JJF) by conducting an evaluation of Bay Area Schools Israel Synergy (BASIS), the Israel Education Project in 11 San Francisco Bay Area Jewish day schools. The JJF has funded the San Francisco Bureau of Jewish Education (BJE) to provide management and professional consultation for these BASIS project partner schools. The goal of this evaluation is to provide JJF and BJE with useful information on the implementation and accomplishments of BASIS. Overview BASIS is an innovative initiative to improve and institutionalize Israel education in Bay Area Jewish day schools. It strives to strengthen students knowledge of and connection to Israel by incorporating best practices in Israel education and engagement through a process of school improvement and change. BASIS encourages the 11 schools to design and put into place a comprehensive long-term plan to incorporate Israel education into every aspect of the school s community students, faculty, and administration to think critically about the Israel education programs they currently offer, and to determine how they fit into that long-term plan. The 11 BASIS project partner schools are Brandeis Hillel Day School (BHDS), with San Francisco and Marin campuses Contra Costa Jewish Day School (Contra Costa) Gideon Hausner Jewish Day School (Hausner) Kehillah Jewish High School (Kehillah) Jewish Community High School of the Bay (JCHS) Oakland Hebrew Day School (OHDS) Ronald C. Wornick Jewish Day School (Wornick) Shalom School (Shalom) South Peninsula Hebrew Day School (SPHDS) Tehiyah Day School (Tehiyah) Yavneh Day School (Yavneh). A December 2010 report, The Israel Education Project in the San Francisco Bay Area Jewish Day Schools: Summary of School Site Visits, presented formative findings about project management and support, most useful resources and supports for schools, and challenges schools encountered in planning and implementation. This report presents early summative findings, including the extent to which BASIS has fully implemented the project activities and achieved the short-term outcomes posited in its logic model (Exhibit 4). The findings are based on all the data collected, including data from school site visits and faculty interviews, student and teacher surveys, school-based assessments of student learning, interviews with BASIS team members and consultants, and school progress reports and proposals. 1

13 This report is organized as follows: Description of the BASIS logic model Description of the extent to which schools were able to implement the BASIS project activities outlined in the logic model, including providing teachers with professional development, developing Israel education visions and goals, developing long-term Israel education plans, and implementing new and enhanced Israel education strategies with students Findings on changes in educators knowledge about, connection to, and engagement with Israel, as well as their capacity to design, plan, and teach Israel education Findings on student learning about Israel and changes in students connections to and engagement with Israeli people and the role Israel plays in the Jewish community worldwide Description of how school cultures and infrastructures are being impacted by BASIS Discussion of the main findings and conclusion. Data Collection Methods The 18-month evaluation drew on the multiple methods described below. Document reviews. SRI helped design the format of end-of-year and midyear progress reports so that they would be informative to the evaluation. We used the progress reports to gather information about the strategies being used and results of schools internal assessments of student learning about Israel. We also reviewed all schools proposals to receive BASIS funding. School site visits. A fall 2010 site visit to all 12 school campuses provided an opportunity to see how the planning and implementation of project strategies were progressing and what additional supports schools required as they moved forward with their plans. It also was an opportunity to capture descriptions of successful or promising strategies. During the visits, SRI staff members conducted interviews with the heads of school, BASIS coordinators, and two to three teachers (at least one Judaic studies and one general studies teacher) on each school s BASIS Leadership Team. Interviews with BJE. SRI interviewed BJE personnel involved in BASIS about project accomplishments, challenges, and lessons learned. Interviews were conducted in March 2010 and again in January Observations. SRI observed four Community of Practice (CoP) meetings, the Post-Summer Forum, and the San Francisco-based Summer Forum to learn firsthand how schools are networking, exchanging resources and ideas, accessing information about resources from the BJE, and working together in other ways. Teacher survey. In November 2010, SRI conducted a survey with all teachers in the 11 schools who have direct teaching responsibilities (i.e., plan and implement lessons). Assistant teachers and preschool teachers were not surveyed. The survey asked about teaching responsibilities, attitudes and practices regarding teaching about Israel, and the extent to which teachers have been involved in BASIS planning and program implementation. The overall response rate for all teachers across all schools was 68% (251 teachers total), but response rates across schools 2

14 ranged from 48% to 92%. Exhibit 1 presents the respondent counts by type of teaching responsibility and grade level. To test for differences between teachers who were involved in BASIS and those who were not, we used the chi-square test. Survey items with observed statistical significant differences at the p <.05 level are reported. Exhibit 1. Teacher survey respondent counts Teach Judaic/Hebrew Studies 50% or more Teach general studies more than 50% Teach mostly other subject(s) Elementary school (grades K 5) Middle school (grades 6 8) High school (grades 9 12) Total All Note: Excludes four surveys with missing grade and/or subject information. Teacher assessments of student learning outcomes. SRI worked with each school to identify a set of five learning objectives (knowledge-focused objectives) for an appropriate grade level that school staff assessed using curricula-based assessments. Schools were encouraged to identify learning objectives for their highest level grade (e.g., grade 5, 8, or 12) in fall Schools completed the assessments of the learning objectives in spring 2010 and reported the percentage of students achieving various learning objectives in their July 2010 progress reports. Student survey. SRI designed and administered a student survey to assess changes in student attitudes, connections, and behaviors toward Israel and Jewish Peoplehood in the fall of 2009 (pre-survey, baseline) and in 2010 (post-survey, after 1 year of BASIS). Five of the schools administered the post-survey in the spring of the same grade year, and six of the schools administered it the next fall. A total of 583 students completed pre-surveys and 630 students completed post-surveys (Exhibit 2). The overall response rate for student surveys was 75% and ranged from 68% to 92% across the 11 schools. Many surveyed students (214) had only a postsurvey. This was often because students started the school between the pre- and post-surveys. We matched 416 students pr and post-surveys to assess changes over time (Exhibit 3). To test for differences between student pre- and post-survey responses, we used the t test for matched samples. Survey items with observed statistical significant differences at the p <.05 level for students overall and by grade levels are reported. 3

15 Exhibit 2. Student survey respondent counts Post-survey Grade Pre-survey (fall 2009) (spring 2010/ fall 2010) Unknown 0 3 Total Exhibit 3. Matched student survey respondent counts Grade at Matched pre-/postsurveys pre-survey Total 416 4

16 The BASIS Model The BASIS team at the BJE describes the overarching goal of BASIS as follows: BASIS strives to bring Jewish students and their families closer to Israel, and to strengthen the connection of youth to Israel and the Jewish people by making Israel a core part of every school's academic program and culture. The BASIS team has also identified the following subgoals: Increase student knowledge about, attitudes toward, and engagement with Israel Build the professional capacity of Bay Area Jewish day schools to offer a robust, sequenced, and coordinated Israel education program to their students and families Institutionalize Israel education in the schools through capacity building, sustainability, and strategic planning Develop model(s) for Israel education that can be applied in Jewish day schools across the nation. The BASIS logic model (Exhibit 4) articulates how a group of Jewish day and high schools with a set of supports are expected to develop sustainable infrastructures and curricular and cocurricular activities that increase students, teachers, and community members connection with Israel. Each school has a BASIS coordinator who spends at least 2 days a week managing project tasks, organizing school-level efforts and planning, and coordinating with other coordinators and the BJE through regular Community of Practice (CoP) meetings. All schools also have school leadership teams composed of school administrators, the BASIS coordinator, both Jewish studies and general studies faculty, and often school community or board members that meet regularly to provide guidance to the project and support the project s broader implementation in the school. Each school also has a curriculum design team that has been overseeing the mapping of current Israel education activities and the development of a curricular scope and sequence. The primary focus of BASIS in was to have each school develop a curricular scope and sequence based on its Israel education vision statement. The BASIS model involves a set of supports that help schools develop their staff capacity to collaborate on, plan, and implement more intentional and coordinated Israel education and engagement activities. Each school receives funding based on student enrollment ($360 per K 12 student) and access to a group of consultants who have been hand selected and are managed by the BJE. Each school is working with a curriculum coach (manche or mancha) from an Israel education organization and receives consultation, as requested, on Israeli arts and culture, twinning and partnering with Israeli schools, and curriculum mapping software. Schools also support each other through sharing resources and participating in joint-training sessions at CoP meetings of BASIS coordinators. The BJE provides the leadership, coordination, and management that make all the project supports possible and that keep schools focused on project deliverables and goals. 5

17 Exhibit 4. Logic model for BASIS evaluation INFLUENTIAL FACTORS RESOURCES ACTIVITIES OUTPUTS Effectiveness of BJE will provide Schools, in partnership with BASIS # of teachers BJE s example curricula and development team, will provide trained management models, experts, professional development for and structure, trainings, Community coordinate with teachers on how to design Teacher communication of Practice meetings, and provide high-quality Israel education satisfaction with with schools, Israel Forum training clarity of workshops to support Schools will update their visions and expectations, and schools planning mission statements and their logic models timeline to help as needed to guide their ongoing schools reach strategies project goals Timeliness of receiving JJF grant funds Teacher and community buy-in for project at each of the schools BJE will link with other Israel education organizations to expand available training and resources to the schools JJF will provide $360 per student; a basket of consultants; access to icenter resources; and timely payment for project management, educators professional development, and BASIS evaluation Schools will develop goals for the types of student engagement they desire students to have with Israel Schools will develop knowledge- and skillbased learning objectives for students and will assess progress on those objectives Schools will develop long-term Israel education plans, based on the identified objectives and goals, that are comprehensive and sequenced by starting with a plan for one or a few grade levels and expanding the plan to include all grade levels by the end of the 4 years Schools will implement the new and enhanced student-focused strategies outlined in their plans: curricula, school events, shlichim, family education, twinning and partnerships programs, Israel trips, technology, and arts and culture BJE = The Bureau of Jewish Education Schools = The 11 Jewish day and high schools participating in BASIS Amount of teachers who include Israel education in their instruction # of students who participate in various Israel education strategies Amount of time students spend on Israel education Student satisfaction with Israel education instruction/ activities they receive SHORT-TERM OUTCOMES (within 1 year) Increases in educators knowledge about, connection to, and engagement with Israel Increases in capacity of teachers to design, plan, and teach Israel education Increases in student learning about Israel (e.g., its geography, its history, its people and their cultures, its geopolitical issues, its cultural/religious issues) Increases in student connection to and engagement with Israeli people and the role Israel plays in the Jewish community worldwide INTERMEDIATE-TERM OUTCOMES (within 2-4 years) Increases in families knowledge about, connection to, and engagement with Israel Increases in the visibility of Israel in school environments, including surroundings (portraiture), programming (adult education), place of Israel in the overall school life beyond celebratory Israel days Increases in resources relating to Israel (e.g., purchase of books for libraries, resources for music education, creation of learning centers, and other resources directed to Israel related topics) Increases in schools use of sustainable resources within the community to support and fund Israel education capacity building and programming IMPACT (sustainable changes) School mission statements and planning and professional development structures will incorporate Israel education as an essential piece of Jewish education Schools will have developed sustainable Israel education plans and strategies by incorporating Israel education into their budgets and ongoing fundraising efforts Schools will have more available resources and capacity to support ongoing Israel education Israel education will play a more instrumental role in school life and curriculum 6

18 Implementation of Project Activities In this section, we discuss the activities schools and the BASIS leadership team have undertaken as part of BASIS and the accomplishments schools were able to make toward the BASIS goals. The logic model identifies the following activities that schools are expected to accomplish with support from the BASIS team to strengthen their capacity to provide high-quality Israel education and engagement activities: Provide school staff with professional development on how to design and implement high-quality Israel education Update the schools Israel vision statements as needed to guide their ongoing strategies and develop goals for the types of student engagement they desire students to have with Israel, including knowledge- and skill-based learning, and how to assess progress on those objectives Develop long-term Israel education plans, based on the identified objectives and goals, that are comprehensive and sequenced by starting with a plan for one or a few grade levels and expanding to include all grade levels by the end of the 4 years Implement the new and enhanced student-focused strategies outlined in their plans: twinning and partnerships programs, Israel trips, family education, shlichut, arts and culture, and technology. The accomplished activities are described in this section. Activity 1: Provide Professional Development The BASIS team supported the professional development of staff at each of the Jewish day and high schools through multiple workshops, CoP meetings, and consultation from a team of Israel education experts. Workshops The BASIS team held professional development workshops for school staff to learn new skills, hear about new resources, strengthen their teams commitment to Israel education, and work on their individual school plans. Summer forum in Israel (July 2009). All schools sent teams to Israel in July 2009 as part of the initial project-wide summer forum and to a pre-summer forum meeting in San Francisco to prepare for the trip in June Attendees reported numerous benefits from the forum, including that it clarified project expectations, helped schools develop cohesive teams before beginning BASIS work, highlighted the existence of diverse perspectives about Israel within and between schools and opened a dialogue to discuss them, provided an opportunity for first-time visitors to experience Israel so they can be more connected and passionate about Israel education, exposed participants to ways of teaching and discussing Israel that could be replicated (especially in high schools), and allowed for staff from some schools to meet with twin schools or purchase materials. Post-summer forum (October 2009). BASIS school teams participated in a post-summer forum to help them begin to refine their schools visions for Israel education and identify the related goals and outcomes those visions would suggest. This forum also enabled schools to explore 7

19 various Israel education strategies, including twinning and partnership as well as Israeli arts and culture. Themed seminars. The BASIS team offered four themed seminars in coordination with CoP meetings. Whereas CoP meetings were typically for BASIS coordinators, themed seminars were open to any interested school staff member. The seminars were designed to support school staffs in developing or expanding their Israel education strategies and activities with students. The seminars covered the most common strategies being used by schools: technology, arts and culture, partnerships and twinning, and curriculum development. Typically, some members of the school leadership team or curriculum design team attended these workshops along with the BASIS coordinators. Atlas Curriculum Mapping training (May 2010). All the schools participated in 1 day of training on the Atlas Curriculum Mapping system to facilitate the creation of their scope and sequences and design of curricular units. Schools will use the Atlas tool to organize their curricular planning and sharing across grades, subjects, and schools. Summer forum in San Francisco (June 2010). Attendees of the 2010 San Francisco summer forum learned about Understanding by Design (UbD), the curriculum planning approach chosen by the BASIS team at BJE. 1 In this process, educators first define enduring understandings the ultimate learning goals. Then they engage in backward design guided by essential questions, choosing activities that will guide students to the goal. Most attendees reported benefits from the forum, including the opportunity to work with curriculum design professionals and manchim, time to start drafting big ideas and vision statements specific to their schools, and the opportunity to collaborate with other schools and hear about their plans and activities. Winter forum and Caravan of Learning (January 2011). Ten schools (11 campuses) participated in a winter forum that allowed participants to select a variety of academic presentations to attend. All led by Israel education and engagement experts from around the world, sessions included presentations on Israel-related subject matter, presentations on different pedagogic approaches to Israel education, and hands-on participation in Israel education activities. The winter forum also included an evening arts and culture program: Hatikvah (HOPE) a Hymn Is Born. After the 1-day winter forum was the 2-day Caravan of Learning that allowed schools to select from a menu of professional development sessions to be held at their school sites. All but one BASIS school took advantage of this opportunity. Exhibit 5 presents the numbers of participants at each of these workshops. 1 G. Wiggins and J. McTighe. 1998, Understanding by Design, Alexandria, VA: Association for Supervision and Curriculum Development (ASCD). 8