The Hanover Collaboration

Standardized Testing in General and No-Child-Left-Behind in Particular are Destroying our Schools

You are doubtlessly aware that there has been some controversy regarding the effects of the historic “No Child Left Behind” law, created in 2002. In fact, the “controversy” is the political kind. Amongst teachers there is a robust consensus: It’s hamstringing them.

The point of the law was to provide accountability in education by means of testing. The NCLB requires that students be tested regularly and that schools be held accountable for the results. The goal was to ensure that by 2014 no child be “left behind,” i.e. that 100% of students demonstrate some sort of adequacy.

In principle this seems like an excellent idea, don’t you think? One of the main problems with the educational system is that teachers teach in a vacuum. They are completely on their own and are free to fail in secret. While I would say the problem is lack of “support” rather than of “oversight” or “accountability,” the distinction is largely semantic. The system is generally incapable of recognizing good or bad teaching, and therefore can’t leverage the former against the latter. Some “good hard data” would be useful, wouldn’t it? The NCLB law is in some sense the culmination of a much larger trend to provide the teaching profession with analytical tools.

I’m thinking the scientist types in the room are still nodding, thinking this is a good idea. If so, you missed the clue. We want 100% of students to pass the test. There is only one way to make that happen, and it has nothing to do with teaching. It means making the test stupidly easy.

But of course the schools couldn’t get away with having a truly trivial test and so we’ve got the next best thing: multiple-choice tests that require nothing more than memorization to pass. Schools that are concerned with their results now spend the entire year drilling for these tests. In fact, many are getting rid of recess, arts, gym and electives to spend more time on test prep. For many students, their learning experience has consisted only of drill. I find this terrifying.

One might think that in better schools the problem is less prevalent. Sadly, even in the best public schools, months are set aside for test preparation. Moreover, since the standards are so low, the entire curriculum is dragged down. Even private schools that are not obligated to follow the NCLB are not immune.

Consider: recently, many colleges have stopped accepting AP credit, because success on the AP exam is a poor indicator of success in the equivalent college class. How can this be, when the AP exam is ostensibly based on the college curriculum? Simple. It’s because the AP classes are simply drilling for the exam. The “equivalent” college courses (typically) require a much deeper understanding.

The problem is that far too many people think that having “learned” is synonymous with “knowing stuff.” Or equivalently, that the only measure of “learning” is the quantity of factual knowledge. This fallacy is wrongly driving many people, but the ones who matter are the ones creating the tests. The very premise that one could make

$ a multiple-choice test
$ that could be completed in two hours or less
$ and would suffice to demonstrate that someone had mastered a college subject

is dubious at best, even for the most mechanical subject. For foreign language I suppose, it would suffice.

Here’s a personal example. I have never enjoyed biology, and I’ve always been somewhat ashamed of the fact. It seemed to me that before you got to the interesting stuff, you simply had to know too much trivia. I believe now that my assessment was wrong—It only takes a small set of principles to come up with extraordinarily rich models. It was the curriculum and the teaching that I couldn’t enjoy.

One might argue that drilling is all that schools are capable of, or that many of the skills we would like are innate and unteachable. There is data to refute this. First anecdotal:

Courtney is currently teaching 5th grade in an esteemed Massachusetts public school. In this school they spent a month memorizing the phases of the moon, because that will be on the test.[1] Not a single kid seems engaged at all; the kind of effort they put into their assignments is extremely disheartening. In particular, the lack of depth, thesis, and voice in their writing is especially troubling. They approach writing in the same mindset that they approach their worksheets and multiple-choice tests.

¹ Courtney is the only teacher in the grade who has even tried to explain that the phases came from orbits and rotation. And that only happened because I made up a demo activity for it.

The previous year she taught in a private school that took a very strange approach. They deliberately taught very little content—only one topic (e.g. China for the 5th graders) for an entire year—but they could afford to treat it with great depth. By the end of the year, every single student could write a decent essay on that topic. The teacher could ask questions that required synthesis or creativity and expect the students to provide it.

From more systematic data, we can see that many other countries have revamped their tests so that they are more indicative of abilities other than recall. For instance compare the International Baccalaureate chemistry exam to the AP. Or more starkly, we could look at the difference in how we assess fifth graders (keeping in mind that the MCAS is one of the hardest tests in the country). Other countries place different emphasis on breadth versus depth of understanding. On the PISA (careful, that is a safari crasher) tests, we came finished below the median while our per pupil spending is in the top decile. Perhaps unsurprisingly, we are above the mean for proficiency levels 3 and 4, while below the mean for levels 1,2,5, and 6.

It has taken me a long time to formulate an opinion on how this applies to math. Learning math is inherently sequential. In some sense you must master a certain volume of earlier material before you move on. Can we afford to minimize content in this same way? I think the answer is, “yes.”

Consider the fact that once you’ve mastered a concept, it becomes abstract. We’ve argued before over the degree to which mechanical mastery is necessary, and I seem to recall that my previous position (i.e. “not very”) was not the strongest. That said, I think you’ll agree that in order to “move on,” what matters is the conceptual rather than mechanical mastery. Suppose that instead of drilling an operation as long as we do, we spend the first half of that time doing it the same as we do now (i.e. worksheets, mechanical problems), but then spend the second half asking essay questions. Can you believe what you’re reading? This is Tom here, advocating essay questions in math class. Sure the students will make more mechanical mistakes, but when it comes time to learn the next abstraction, I’m confident that the students will be better off. Furthermore, I think this will make math as a whole a lot less intimidating for a lot of students.

I was going to talk about the ramifications of NCLB for the very worst schools, the ones that it has apparently been helping, but this post is getting long. Suffice it to say that it really has helped and done so in a socially significant way. However I think the academic gains are ephemeral, due to the reasons already described.

What do I advocate? Obviously, cut back on the testing. When we do test kids, we need to be much smarter about the tests that we create. We should do our best to ensure that “teaching to the test” is fine—great even—since that is what we want taught. Look at some of the countries that are succeeding and copy them. For example, in some successful systems, the standard for assessment is a personal interview, or a portfolio of work, or a large (group) project—all of which are dramatically more work than multiple choice.

More importantly, curriculum, lesson and test design desperately needs attention. As it stands now, every teacher has to reinvent the wheel every single day. There are a few private companies providing well-designed lesson plans. Sadly, these materials tend not to mesh precisely with the standardized tests which are designed ad-hoc by state boards.[3] The private market is largely a failure—it is only successful in early grades for which there are fewer tests. Even then it is usually only the good teachers who are aware of (and who often spend their own money on) these materials.

Nonetheless, there are no great technical hurdles to providing every teacher with experimentally vetted sets of lesson plans designed by qualified people. In fact, they could be given the opportunity to choose between many such lesson plans. Furthermore, these sets of plans could explicitly relate to a broader curriculum and easily include meaningful measures of assessment. The International Baccalaureate program would be a great start.

The only big (non-political) obstacle is the teacher’s time. How much time per day can the teacher devote to preparing new material? How much time can the teacher spend per student, or even per group, during class? These are the bottlenecks. Hiring more teachers seems to be the only answer.[4]

³ For instance, by designing multiple choice tests based on highlighted vocabulary from a textbook (that was chosen because of bribes).

⁴ We don’t necessarily need to build more classrooms, though. Unlike in the “interdisciplinary” classes at Dartmouth, Courtney has seen instances in which two teachers have split a large class and it works surprisingly well.