UNDERNEWS

Undernews is the online report of the Progressive Review, edited by Sam Smith, who covered Washington during all or part of one quarter of America's presidencies and edited alternative journals since 1964. The Review has been on the web since 1995. See main page for full contents

September 13, 2009

HOW TEST OBSESSION IS HURTING LEARNING

This is one of the best pieces we've seen on the topic. From a paper presented at the International Conference on Redesigning Pedagogy, National Institute of Education, Nanyang Technological University, Singapore, June 1, 2009.

David C. Berliner - America being a practical country, a land for the promotion of Yankee ingenuity and dominated by the interests of business, never was in awe of the liberal arts or the humanities. There never was a golden age in America in which the liberal arts and humanities flourished while the servile arts were looked down upon. In fact, for at least the first hundred years of widespread public schooling in the US, the practical was deemed appropriate to teach most middle-class white students and was considered even more desirable as the curriculum for the poor and minorities, were they afforded any education at all. The exception to this focus on the practical was for a small and select group of wealthy, white, male students who did get to study the liberal arts in upper high school grades and college.

Today may actually be worse for poor children in the US than at any time in the last half century. This is because the lower classes are being kept from the liberal arts and humanities curricula by design. Using the argument that we must get their test scores up, we in the US are designing curriculum for poor children, often poor children of color but certainly, numerically, for poor white children, that will keep them ignorant and provide them with vocational training, at best. Their chances of entrance to college and middle class lives are being diminished, and this is all being done under the banner of "closing the gap," a laudable goal, but one that has produced educational policies with severe and negative side effects. . .

Data show that changes in the time allocated for teaching reading and mathematics in elementary schools were quite dramatic between 2002 and 2007. These are the years of the NCLB act and mandated high-stakes testing. The time allocated to reading has been increased, on average, over two and a third hours a week, while mathematics time has been increased, on average, about an hour and a half a week. What needs to be kept in mind when interpreting this table is that the "average" masks relevant information. It is likely that many school districts increased time in these subjects a great deal more than the average, because the average includes districts serving high-income children, who typically score well on the tests used to satisfy NCLB requirements. Those districts probably changed their time allocations very little. On the other hand those serving low-income students probably changed their time allocations a lot. . .

If reading and English language arts consists of too much phonics practice; too much drill and test preparation; too many worksheets for practicing reading skills; not enough writing to express complex thoughts; not enough reading for enjoyment; and not enough reading of academic material to increase vocabulary in order to aid comprehension; then the reading is more to foster the goal of basic literacy and not literacy for its pleasure, or for its value in exploring the arts, the sciences and the humanities. . .

Sadly, evidence exists to support the hypothesis that the increased time spent on reading and mathematics is not helping us make better readers and mathematicians.

A second look at reading achievement and the effects of greatly increased reading instruction on the performance of the various US states comes from the Educational Testing Service. What is obvious is that average scores are not increasing, and many states have actually done worse since the enactment of NCLB. . .

The evidence is that the schools with the poorest children, and therefore the schools with the greatest likelihood of being sanctioned under NCLB, are those where the reading curriculum in now often of the most basic type. While such a level of literacy might have been good enough at the beginning of the 20th century, it is hard at the beginning of the 21st century to defend the forms of instruction used and the kinds of literacy attained by the children in many of our poorest schools.

We now know that for many children the motive to engage in activities found pleasurable for their own sake is diminished when those same tasks are rewarded. This suggests that a significant number of poor and minority children who really do enjoy reading for pleasure and edification are much more likely to be turned off of reading because reading has become a task governed by extrinsic rewards. In many schools with the poorest students stars are awarded for rather trivial multiple-choice questions answered correctly about books just completed. . . Other schools have class parties for high numbers of books read collectively per unit of time. None of these approaches is wrong from a behaviorist theory, yet all of these short-term motivational strategies are likely to have a negative influence on continuing motivation to read. We don't know this, of course, because we usually do not study the long-term effects of these programs. But there is good reason to believe that continuing motivation to read will suffer under some of these instructional programs.

There is another theory in our field that comes to mind when looking at these data. It is related to time and learning. I did some of that research myself. From all the research, and from the common sense that is found in the humblest of homes, we have been able to derive a sound educational law, namely, that the more time students spend studying in some area of the curriculum, the more likely they will have learned more in that area. Time and learning are believed to be, and are empirically found to be, causally related. But this principle of learning is directly challenged by the reading data we have. Significantly more time spent in reading is leading to less improvement on the high quality assessments of reading that are used, the NAEP tests. This suggests that students may be studying the wrong things, or that their motivation is being undermined, or both. This is not good.

Much more


9 Comments:

Anonymous Mairead said...

Sam, please consider changing your headline. I suggest "Bean counters are destroying education".

There's nothing wrong with being keenly concerned about test results. The sine-qua-non requirement for improvement is to know where the soft spots are. Only testing will reveal them.

But not all testing is created equal. As Einstein is alleged to have said (it sounds more like Theo Geisel): not everthing that counts can be counted, nor does everything that can be counted, count.

Many of the most important adult-human qualities --integrity, love, wisdom, etc.-- are not quantifiable in any meaningful way. Which doesn't mean they can't be tested, but it does mean that any hard value put on the results -even pass/fail- is silly at best and criminal at worst. Those qualities exist on a continuum, as any clinical psychologist knows, and the only meaningful way to test them is full-life observation, with a week or more of all-day conversation between tester and subject running a distant second in validity.

It's a rock-solid premise in psychology that adults will only learn instrumentally. We have so many demands on our time that anything that's not useful doesn't get any "brain-time".

As kids mature toward adulthood, they begin to evaluate the utility of what we demand they learn, and by the time they hit adolescence, the smarter ones (I wasn't one of them, I'm saddened to say) have figured out that most of it is dreck and they spend as little time as possible on it.

We stuff them with dreck --when did Columbus discover America, what's 13x12, who was the 19th president, where is Samarkand, identify the three major themes in Finnegan's Wake, what is the principal export of Australia-- purely and simply because the learning of dreck is easy to test. It's quantifiable!

And it'd still be dreck even if we could test to 3-decimal-places accuracy with complete reliability.

Who deify numbers? Bean counters. They're not concerned with whether the numbers mean anything, only that they're there. Their lives are spent adding, subtracting, averaging, collating, and writing-down. Understanding? Not their job, mon.

So do please change the headline, Sam. You're skewing the meaning.

(Berliner must not have spent a lot of time on his prospective sample questions, because if he'd been in grad school with me, he'd have been walking around with his arse in a sling after our Methodology prof had been at those questions. "Compare and contrast" indeed! A question like that guarantees a superficial, mechanical, bean-countery answer from everyone but the obsessives in the class.

A better question might be something like "to what extent were the people in the breakaway states during the US Civil War like the people in the Thirteen Colonies during the Revolution?" By only asking for similar, you get dissimilar for free - the Tao shows the way again)

September 13, 2009 6:36 PM  
Blogger m said...

Mairead, its not just the bean counters. Somehow I suspect its far worse than that. Bush43 could not be considered a bean counter by any stretch of the imagination. But he, and others like him, strongly supported NCLB.

The old chestnut about learning what your enemy is afraid of, by listening to what he threatens you with, really shines here. Bush43 loved to talk about standardized testing for children. I suspect that it was his hostility towards the younger generation, and his own fears of testing generated failure.

Many have a variety of needs to separate themselves from other generations. Length of hair, beards, skirt lengths, accusations of unbridled sexuality and promiscuity, laziness amongst others. I suspect that the pro heavy testing crowd is just another group that encompasses sublimated sadists or has additional needs to differentiate the generations.

In other words, it is not just a search for a simple metric to measure the quality of education. But the drive to excess and invalid testing also stems from emotional weaknesses and needs in people controlling the educational system. This latter cause is by far the most pernicious, in that no amount of logic can ever persuade such individuals to rectify their behaviors.

September 13, 2009 9:35 PM  
Anonymous Mairead said...

I dunno, M, I don't think the descriptors are mutually exclusive, or contradictory.

The worst bean counters are authoritarian, and authoritarians are like people with paranoia in having a LOT of fear and hostility. They have a big need to keep the world under control, which fits with your characterisation. And numbers are a wonderful way to keep control, especially for shallow thinkers like Bush.

The numbers don't have to mean anything, it's enough that they exist and can be ordered arbitrarily into Good and Bad groups. To people who need them to be, they're complete in themselves, their inherent orderability carrying all the meaning and power anyone like that can want.

Marshall said "the power to tax is the power to destroy". But it's equally true that the power to label is the power to destroy. And numbers are the perfect tool to support labeling. They're a wonderful threat because they can't really be gainsaid - the fact that they refer to nothing important is ignored or denied by the bean counters. They exist and have an ordering; ipso facto, they are important.

So I suspect that, as usual, we're actually in agreement on this, just expressing ourselves differently.

September 14, 2009 6:30 AM  
Anonymous Mairead said...

(I intended to make clear, but didn't, that "bean counter" isn't a synonym for "accountant" where I come from, professionally. In those communities it refers to anyone who overvalues numbers in an unthinking way. Another term sometimes used for the bean-counting mentality is "physics envy"-- the neurotic need to quantify the unquantifiable.)

September 14, 2009 6:36 AM  
Anonymous pj said...

Alfie Kohn is the great pioneer on this subject. No one exposes the damaging downside of standardized testing better. If you've read his books, this article looks like a runner-up re-hash. Mr. Kohn is a better writer, in my opinion. He is very easy to read and to comprehend, and he goes deeper to expose many of society's most fundamental misconceptions - and how they harm education and our schools and our children.

September 14, 2009 8:09 AM  
Anonymous Mairead said...

An example of ...something.

On Saturday I'd a circulation test done on my feet (I'm at that age, but I'm now nearly positive it was nothing but a money-maker for him).

So it finishes (it's all done by a machine) and produces this neat little laser printout. He says my circulation is normal, pointing to the numbers which were in the 70s. He then pointed to the bit in the legend where it says that numbers in the 30s are not good.

So, what would you ask in that circumstance? Probably what I asked: what's the upper bound? I'm trying to find out how much goodness there is in those numbers by, first, finding out if its a 0-80 scale or a 0-100 or 25-250 or what. My next question would have been whether it was linear and if not, what. But he shut me down with the egregiously dismissive claim that "there is no upper bound". So I just shut up, gritted my teeth, smiled, and escaped on my bicycle as soon as I could.

I've no hesitation in calling him a bean-counter at heart. Someone doing science or medicine would have been eager to send me away with a useful level of understanding. But he only cared about telling me the raw numbers, as though they themselves meant something.

In a world where money is god, numbers are the catachism.

September 14, 2009 3:43 PM  
Blogger m said...

Mairead, I'm OK with your connotations, and have used them as well.

Given the import of this topic, I wanted to differentiate between "those who know the price of everything and the value of nothing", and the truly malicious.

September 14, 2009 5:02 PM  
Anonymous Mairead said...

I did get what you were saying, M, and was trying to strongly agree with you. But I seem to have lost the ability to communicate. :(

Any disagreement I have is with Sam, who seems intent on maintaining the pearls-before-swine model of corporate newspaper journalism even though that model is a big part of why we're circling the drain. Those buggers are in it up to their oxters, and can't be put out of business soon enough (imo).

September 15, 2009 12:35 PM  
Anonymous Mairead said...

(Insomnia-exhaustion strikes again, muddling what passes for my mind these days: in saying, after likening Sam's publication model to that of corporate newspapers, that the corpo newspapers should go out of business, I wasn't intending to suggest that prorev should, too. I just got derailed and changed topics in a non-sequitur way *sigh*)

September 15, 2009 3:28 PM  

Post a Comment

<< Home