On the Value of Cold, Hard Data
Or, bursting the bubble of test happy edu-policymakers
Policymakers spend a lot of time talking about “good schools” and “pro-schools” policies. Many like to reference data points based on good, fair tests. These tests provide the evidence needed to advance a range of policy goals - from more school funding to new teacher evaluation schemes to privatizing the whole system.
Rarely, though, does anyone suggest we address the root causes of education challenges - systemic poverty. Access to healthcare. Hunger.
Peter Greene breaks through the noise a bit with a piece he wrote back in 2019 about the NAEP - National Assessment of Education Progress.
So what’s the one actual lesson of NAEP? One continuing belief for some students of education policy is that if we just had some cold, hard data, we could really get some stuff done. We could settle arguments about curriculum and pedagogy and policy, and by making data-driven decisions, we could steer education into a new golden age.
Well, here’s our regular dose of cold hard data. It hasn’t settled a thing.
That’s the one actual lesson of NAEP; the dream of data-informed, data-driven decision making as a cure for everything that ails us is just a dream. Data can be useful for those who want to actually look at it. But data is not magical, and in education, it’s fruitless to imagine that data will settle our issues.
Yep.
As I noted in a 2015 piece on NAEP data:
After analyzing the Tennessee results and putting them in context with national results (both of which essentially remained steady from 2013) , I noted:
It’s also worth noting that states that have adopted aggressive reforms and states that haven’t both remained flat. The general trend was “holding steady,” and it didn’t seem to matter whether your state was using a reform agenda (charters, vouchers, value-added teacher scores in teacher evaluations) or not.
Again, this makes it difficult to suggest that any one or even a package of educational practices drives change.
We do a lot of testing. We do a lot of talking about the testing. And we typically do little to actually address the foundational issues - every child should be fed, have access to housing, and receive health care.
Testing continues to be excessive. We hear a lot about making learning "engaging" for students, but then we test them, and the tests are about as unengaging as you can get.
This year- the first week of August 2025- I and other teachers had to give a standardized test in English class...on the second day of school! How well do you think they did? What kind of first impression do you think this made on them? Whatever happened to spending the first two weeks getting to know your students?
The testing material was about as boring as you can imagine. It was comprised of cold reads presented without context.
We are supposed to give this exact same test at the end of the 9 weeks. I forgot to mention that these tests were to be graded as a Formative assessment, which doesn't affect a student's grade at all. It counts for 0%. If you knew that a boring test would count for zero percent, how hard would you try?