The Impact of Chicago’s Excellence in Teaching Project on Student Performance

In 2006, Chicago Public Schools (CPS) and the Chicago Teachers Union (CTU) partnered to improve classroom instruction by developing the Excellence in Teaching Project (EITP). The objective of the EITP was to evaluate teachers’ performance and provide feedback to improve their lessons, in an effort to advance student learning. Principals were responsible for rating teachers and their instruction as unsatisfactory, basic, proficient, or distinguished. Additionally, principals were responsible for providing their teachers with feedback on how to improve their lessons. Teachers were evaluated through classroom observation metrics based on classroom environments and instruction, two of the four domains of the Danielson Framework.

Before the EITP was implemented, Chicago teachers were evaluated based on several classroom practices: instruction, school environment, and professional and personal standards. Under this system, the school principal observed teachers’ lectures and rated them using a binary scale across 19 categories. Unfortunately, this system failed to differentiate between good and bad teaching practices.

In the 2008-2009 school year, 44 elementary schools (Cohort 1) were randomly selected to participate in the EITP pilot. In the 2009-2010 school year, an additional 49 elementary schools (Cohort 2) were selected to participate. Matthew Steinberg and Lauren Sartain published an October 2015 study in which they followed and compared these two cohorts over time to measure the impact of the EITP on student learning.

In order to measure student performance, the researchers used the Illinois Standards Achievement Test (ISAT) in reading and mathematics. Before the EITP began, there were no statistically significant differences between Cohort 1 and Cohort 2 schools on measures of sex, ethnicity, proportion of students receiving a free or reduced-price lunch, or math and reading achievement (ISAT scores). In the 2008-2009 school year, Cohort 1 schools were exposed to the project, while Cohort 2 schools were not. Therefore, comparing the results of the ISAT for the treatment group, Cohort 1, and the control group, Cohort 2, allowed the investigators to identify the unbiased effects of the EITP.

The results show that Cohort 1 scored better in reading than the Cohort 2 schools, after controlling for constant effects related to student enrollment, race, sex, and geographic area. Specifically, students’ reading scores improved by 0.10 standard deviations. An effect of this size is equivalent to closing the gap of reading scores by 25 percent to 50 percent, between “weak” (schools at the 10^th percentile of the achievement distribution) and “average” schools (schools at the 50^th percentile of the achievement distribution). As for math, the results suggest that the EITP had no statistically significant effect.

Furthermore, the intervention was more successful in schools that had higher performing students (better ISAT results) prior to the intervention. The EITP was also associated with a larger effect on reading scores at low-performing schools, relative to high-performing schools. Breaking down poverty levels further, there was a greater effect from the EITP on reading scores in schools with low poverty levels. As for high poverty schools, the EITP had no effect.

Although the project had a positive impact on reading achievement, the results suggest that this impact is mainly obtained in high-performing and low-poverty schools. Moreover, Steinberg and Sartain did not find evidence of an effect on math achievement. Still, the main contribution of their study is the isolation of the effect of classroom observation metrics to evaluate teachers on student achievement.

Today, classroom observation metrics are a key part of teachers’ evaluation frameworks in the United States. The results of this study suggest that these metrics have an impact on student reading achievement when the schools are higher performing or when they have low poverty levels. However, the EITP’s classroom observation metrics can still be improved when it comes to their effect on students’ math achievement. Moreover, since the EITP had no impact in schools with high levels of poverty, it is advisable to consider other strategies to strengthen teachers’ evaluation frameworks.

Article Source: Steinberg, Matthew P., and Lauren Sartain. “Does Teacher Evaluation Improve School Performance? Experimental Evidence from Chicago’s Excellence in Teaching Project,” Education Finance Policy 10, no. 4 (2015) 535-572.

Featured Photo: cc/(US Department of Eduction)