Introduction. Proficiency Student growth Growth gap reduction Graduation rates

Transcription

1 MMR Growth Description of the MinnesotaTestdata.com Interactive Graphical Tool Jim Angermeyr, Ph.D. May, 2014 Data on the web site have been amended to include the 2014 MMR scores and associated 2014 F/R Lunch data. Note that the 2014 MMR calculations were included rather than the Designation 2014 scores. Since these latter measures are a combination of both 2013 and 2014 scores, the actual earned 2014 scores were deemed more appropriate to make comparisons of change over the previous year. Introduction The Minnesota Department of Education (MDE) publishes its annual Multiple Measurement Rating (MMR) as the primary accountability system for Minnesota schools. The MMR was developed as part of the state s NCLB waiver approval, and was designed to give a more nuanced look at the performance of schools than was the case with the original Adequate Yearly Progress (AYP) measures. The MMR provides schools with indicators in up to four areas: Student growth Growth gap reduction Graduation rates The proficiency score represents the weighted proportion of AYP cells that reach the annual measurement objectives and, with small exceptions, is essentially the same metric as the old AYP indicator. The graduation measure applies only to high schools. However, the growth and growth gap reduction measures are new and different ways of looking at the performance of schools. The MMR Growth Report on the minnesotatestdata.com web site was designed to show school performance on these measures, specifically a means to study the relationship between growth and gap reduction, which is where it differs from the MMR reports available on MDE s Report Card site ( ). Measures and Data Source The MMR Report uses the standardized z-scores for both student growth and achievement gap reduction (AGR). For a complete description of the growth and gap reduction measures, see the ESEA Flexibility Request from the MDE 1. Briefly, the two measures rely on a standardized growth calculation across two years of state tests. Student Growth: A standardized growth score (z-score= student score expected score / standard deviation of expected score) is calculated for each student with a prior year s valid test score. The resulting z-scores are averaged for a school, and this mean z-score represents the degree to which students in that school grew more (or less) than expected. Growth Gap Reduction (AGR): Within each school, separate growth z-score means are calculated for students eligible for free or reduced price lunch, English learners, special education students, and any student classified as non-white. The statewide mean growth for the corresponding higher-performing group (i.e. regular-priced lunch, non-esl, etc.) is then subtracted from the school's mean growth on the lower-performing group. Weighted averages are calculated to give each school s overall growth gap effect size. Negative values indicate the gap is closing and positive values indicate a widening gap. 1 d&rendition=primary

2 Data for these two measures was obtained for all schools by downloading the 2013 Multiple Measurement Workbook for State from the MDE website. The Summary tab from that spreadsheet contains the growth z-score average and the gap reduction AGR score for each eligible school. Note that the z-scores are calculated separately for the elementary middle and high school distributions. In addition, the spreadsheet Enrollment by Special Populations from MDE was downloaded to obtain the proportion of students receiving free and reduced-price lunch at each school. The lunch data was linked to the MMR data by school and district ID number. Using the MMR Growth Report Web Tool Go to and click on the State Data menu tab. If not already selected, change the Report dropdown box to MMR Growth. The default chart shown below in Figure 1 will load and the associated data appears in the table below the chart. Fig 1. MMR Report for Region 11 This default data represent the scatter plot of the school s growth z-score (x-axis) and AGR z-score (y-axis) for elementary schools in economic development region 11 (Twin Cities metro). The plot is divided into four quadrants and labeled according to the two z-score distributions. The upper left quadrant indicates schools with below average growth and below average AGR scores, while the lower right quadrant are those schools with above average growth and above average AGR scores. The remaining quadrants are similarly labeled. Users can change regions, and school grade levels using the relevant drop downs. Users can also filter the data by the schools free and reduced lunch percentage ranges. 2 Finally, the schools from any available district can be selected in the District drop down. The schools from the selected district will be highlighted with a different colored marker, and data for those schools will be listed first in the table. As the scatter plot in Figure 1 suggests, there is a strong functional relationship between the growth and AGR measures. This should not be surprising, since the average school growth is integral to the AGR measure. For example, if a school 2 The schools were divided into three categories of free/reduced lunch: less than 33%; between 34 and 66%; and more than 66%.

3 has low overall average growth, it would be difficult to have above average growth reduction on the AGR measure. This would explain the very few schools that fall into the lower left quadrant. On the other hand, there are schools falling into the upper right quadrant, which suggests that school can have above average growth without necessarily closing the gap. Table 1 shows the correlation coefficients for the two z-score measures, as well as the correlation for the overall MMR percentile rank and for the school s proficiency levels on the 2013 state tests. Not surprisingly, the two growth measures are quite (negatively) correlated. Note that the school s proficiency rates are more correlated with its free and reduced lunch proportion than with the growth indicators. Poverty is also more closely related to the overall MMR percentile than to the growth indicators, probably since the portion of MMR still is based on overall achievement, and thus poverty levels. Nonetheless, a school s poverty level still impacts the growth and AGR z-scores. This can be seen graphically by changing the F/R Lunch proportion drop down and watching the impact on the scatter plot. High poverty schools at most levels cause the plot to move higher and to the left, suggesting a decrease in both the overall growth and fewer school narrowing the growth gap. Summary As with the Challenge Index report on the minnesotatestdata.com web site, the MMR Growth report is designed to help schools evaluate their own performance relative to other schools, and to help identify schools which may be similar in geographic location and poverty levels, but performing at higher rates on the growth and AGR domains.

4 Table 1. Correlation Matrix Type Elementary FRLPct Growth Zscore AGR Z- Score MMR FRLPct N Growth Zscore N AGR Z-Score N MMR N N N 835 Middle FRLPct N Growth Zscore N AGR Z-Score N MMR N N N 225

5 Type High FRLPct Growth Zscore AGR Z- Score MMR FRLPct N Growth Zscore N AGR Z-Score N MMR N N N 408