Today, Wisconsin’s DPI released the 2023-24 Forward Exam scores for schools across Wisconsin.  In the past, WILL has referred to these scores–and the report card built from them–as creating a “Lake Wobegone” effect where everyone is above average.  But the changes made to the report card this year make the previous version of report cards seem like the pinnacle of accountability.  In this blog post, we will compare the new report card to the old and highlight how DPI’s unilateral power to make changes to the state’s school accountability system must be checked.

What has Changed?

We wrote about some of this in a previously when the changes DPI was planning first became public. In a closed-door meeting that required the signing of a non-disclosure agreement to get into, DPI altered both the labels for categories of Forward Exam achievement and the cut points for those categories.  The new labels are perhaps meant to be less “triggering” for kids, but they are also far less informative for parents. “Below Basic” is now “Developing.” “Basic” is now “Approaching.” “Proficient” is now “Meeting.” And “Advanced,” mercifully, will remain the same.  But far more insidious than changing the names of the categories were changes to what “proficiency” itself means. DPI took it upon itself to alter proficiency standards that have been in place in Wisconsin since 2012.  The new bar was expected to be lower, and that clearly has proven to be the case based on results released today.

Implications of Changes

Wisconsin’s Forward Exam scores were useful in part because they were tied to the national standards of the National Assessment of Educational Progress (NAEP).  The NAEP is taken in schools around the country and provided a useful proficiency benchmark for education policy wonks to compare Wisconsin to the rest of the nation.  Now, the Forward Exam will no longer be useful in making such an assessment.

A second problem is that changing the proficiency standards breaks the long-term trend of student performance in Wisconsin schools.  No longer can we assess how students are doing relative to their counterparts from more than a decade ago. Indeed, the WiseDash website now includes the disclaimer, “In 2023-2024, the standards for ELA and Mathematics were changed, as a result you will not see results for 2023-2024 in the trends graph for Forward in these two subjects.” DPI may have gotten tired of being dinged for decades of high spending that has produced low proficiency, but the answer is not to change the standard of judgement.

The final problem is that changing proficiency levels will have an impact on the rating of schools and school districts on the state report card.  We were already in a situation where districts like Milwaukee with proficiency rates in ELA of around 15% were being rated as 3 stars, or “Meets Expectations.”  What is likely to happen now is that very few districts at all will be less than three stars.  This robs parents of an important tool for assessing how well schools are performing when they’re evaluating where to send their kids.

Results: Statewide Proficiency

The figure below shows the trend in ELA proficiency on the Forward Exam over the past five years (note that the 2019-20 school year is missing because testing didn’t take place).  The trend until this year is relatively clear-a large dip during COVID followed by gradual improvement.  Yet the state had yet to reach the proficiency levels of 40% that we observed prior to the pandemic. Until this year, that is.  Proficiency shot up by more than 10% in ELA in just one year thanks to the changes DPI implemented.

The changes in math are very similar, as shown below.  The increase thanks to DPI’s changes was even greater, shooting up more than 12% between 2022 and 2023.

 

Of course, we know that this is not the case.  But quantifying exactly what change has occurred (up or down) is now nearly impossible. And districts will no doubt tout their dramatically improved proficiency, ignoring the changes.

Changes in Proficiency within Districts

The figure below shows the shift in proficiency between 2022 and 2023 for the largest twenty districts in the state.  As of this writing, DPI has not put out the Excel files yet that include every district, so each of these was manually calculated via Wisedash. The average district among the top twenty saw an increase in proficiency of about 11.4%.  It is possible that districts like Oshkosh that saw a more sizeable increase have actually seen a meaningful increase in student performance.  By the same token, it is possible that districts on the lower end like Milwaukee may have seen a decline in student outcomes.  But the reality is that we will never know, and that is a problem.

Table 1. Proficiency Shift, ELA 2022-23

Conclusion

DPI has proven that it cannot function as a trustworthy administrator of school accountability in the state. Even Governor Evers, who was Superintendent when the link to NAEP was put in place, has said “I don’t think we should be lowering (the standards).” The legislature should move to reverse these changes so that school performance can be compared over time, pre- and post- COVID closures.  Moving forward, the legislature should require than any changes to the accountability system in the state be put through the required rule-making process.  It is vital that Forward Exam scores be a tool to inform parents, taxpayers, and policymakers about how Wisconsin schools are doing.  Only by reversing what DPI has done here can that remain the case.