One of the shortcomings of the data-driven reform movement is its reliance upon standardized test data to draw meaningful inferences about darn near everything: teacher quality, school quality, student quality, school district quality. Comparisons are invariably made between the high and low performers. The usual chorus of “apples to oranges” drifts into the night sky. Then “similar” schools are compared according to demographics and size. The high performers within these subgroups become the watermark for our schools, the examples of schools doing it right.
The unmet challenge in all of this is trying to meaningfully compare different populations while accounting for very nonstandard learning experiences. So when your school gets compared to another somewhere in your state because you have kinda sorta similar poverty rates and ballparkish demographic numbers, it’s difficult to say “I am confident that these students have had standardized instruction in and out of schools from the time of their birth to the time they took this test.”
Therein lies the horrible thing about making decisions based on tests alone. We do not account for the lack of standard learning experiences children have. We don’t make all that much of an effort frankly. Common core standards are offered as a solution, but I’m not convinced that teaching the same thing to the same age peers everywhere in the country will overcome the differences children have in the privacy of home.
Standardized tests with limited stakes can give us an idea of what kids may or may not know. Attaching ever-higher stakes to them is just thoughtless.