Intelligence Squared debate: Don’t end the tyranny of the test

On Thursday I spoke at an Intelligence Squared debate called ‘Let’s end the tyranny of the test: relentless school testing demeans education’. Together with Toby Young, I spoke against the motion; Tony Little and Tristram Hunt spoke for it.

There were a number of important points of agreement between the two sides. Tony Little told the story of Tom, a brilliant history student who got a U in his History A-level because his argument was essentially too sophisticated for the narrow exam rubric. I’ve known Toms in my time teaching, and I’ve also known the opposite – the student who gains top grades in exams through a mastery of the exam technique, as opposed to the subject itself. I completely agree with Tony Little that this is a real problem in our exam system. I’ve written about this in this collection of essays here, where I am critical of narrow exam syllabuses and textbooks that encourage a focus on exam structures, rather than the topic itself. For example, in this exam textbook on Germany 1919-1945, there are large sections called ‘Meeting the Examiner’ where the technicalities of the ‘8 mark question’ are discussed, but there is no detail at all on any aspect of German history outside that narrow period. In this textbook, it is more important to Meet the Examiner than it is to meet Bismarck. If you also have concerns over this phenomenon, I’d recommend reading Heather Fearn’s blogs on the subject here.

We also agreed with the proposition that blunt government targets were problematic and that teaching to the test was not a good thing (I have written more about this here), and they agreed with us that exams should always play an important part in education. However, where I felt there were central areas of disagreement were in the attitudes to exams. The proposition viewed them as essentially a necessary evil, which were in many ways inimical to good education, and whose role and impact needed to be reduced as far as possible. Tristram Hunt said that our real focus should not be on exams, but on ensuring equity in the early years, and on teacher quality. Tony Little said that our focus should not be on exams, but on ensuring a love of learning; he also said that exams ‘atomise knowledge’. It was clear that for both of them, exams were not always helpful for these other aims. Whilst I accept that in our current system this may be the case, I am convinced that exams have an important and indeed indispensable role to play in achieving these aims.  I don’t think you can improve equity, teacher quality and a love of learning without some form of reliable feedback – and exams are basically the best and most accurate method of gathering feedback that we have. In many different areas of life, improvements and innovations are often brought about by improvements in measurement and feedback. Exams are our measurement system in education, and that in some ways we misuse our measures at the moment is not an argument against measurement: as Sir Philip Sidney said, ‘shall the abuse of a thing make the right use odious?’

Where exams have gone wrong – as I fully accept they have – we have to reform them, not abolish them. And in many ways, the ways they have gone wrong is that they have drifted away from being pure exams. For example, for me, one of the problems with the kinds of history exams Tony Little rightly criticised is that they are too dependent on human judgment against abstract criteria, which we know is a very ineffective method of assessment.  I would like to see an element of multiple choice questions in such exams, which would help to eliminate such problems – but of course, multiple choice questions are exactly the type of question which receive even more criticism for ‘atomising knowledge’. Similarly, one way we could ensure greater equity in the early years is to introduce exams at KS1, rather than teacher assessments, since we have some evidence that teacher assessments at this age are biased against pupils from a low-income background – but again, if you suggest replacing teacher assessments with tests, you generally do not get a great response. So this, for me, was the difference between the two sides: we both acknowledged the flaws in the current exam system, and both had very similar aims for education, but on our side, we felt that the problems could be solved by wiser use of exams, and perhaps even more of them, whereas for the proposition, they felt that there should just be fewer exams, that they should be of less importance, and that there should be more extended project-type assessments.

In my speech itself, I had two main arguments. First, I argued that exams were accurate and less subject to bias than other methods of assessment, such as teacher assessment and coursework. This is a well-established finding in the literature, but it is curiously little-known. Teacher assessment is frequently biased against disadvantaged pupils, but people assume again and again that actually, such assessment helps these pupils. I will write more about this in the future, but if you are interested in it, then my article in the Policy Exchange collection of essays here has more on this, and Rob Coe’s video here does too. I also argued that properly designed tests really do predict things of value. For example, this fascinating study by Benbow and Lubinski tracked top scorers on the American SAT at age 13. At age 38, many of them had gone on to achieve remarkable things, and not just in the predictable areas of business and academia – many of them excelled in the creative professions too. And the American SAT is the kind of test that many would criticise for ‘atomising knowledge’, or for just being, as David Baddiel says here, a kind of ‘cognitive trick’. This is not true, and it fundamentally misunderstands how tests work.  Questions on tests do not have to look exactly like the kind of problems we face in real life in order to provide useful information about how we might do on such real-life problems.

My second argument was that testing was also a useful pedagogical technique. We know from Bjork’s research on the testing effect that straining to recall something, as we do in a test, actually helps us to recall it in future. We also know that practice testing is a very effective revision technique: much more effective than the more common approach of rereading notes and highlighting them. The frequently-heard line about how ‘weighing the pig isn’t the same as feeding it’ is false: in the case of education, weighing the pig is the same as feeding it. Testing actually helps you to learn.

We narrowly lost the debate, and afterwards I spoke to a number of pupils in the audience whose experiences of exams were similar to those of Tom’s above, and who were therefore understandably sceptical of the value of exams. It’s worrying that this is happening, and it makes reform of our exam system all the more important. This same pattern – misuse of exams leading to widespread mistrust of them – has also been seen in the US, and has been outlined brilliantly by Daniel Koretz in Measuring Up. Koretz is at pains to point out the incredibly valuable information we can get from apparently ‘narrow’ standardised tests, but he is also very critical of the No Child Left Behind Act, and the teaching to the test and gaming it has encouraged. I completely agree with both his points, but it’s a combination of views that feels very rare: often, it feels as though if you are in favour of tests, you must be in favour of teaching to them; and if you are worried about how tests are being used, you must be in favour of abolishing them. It would be nice to open up space for a more Koretz-esque view of tests in this country. A confrontational debate may not be the best way of doing this, but I did enjoy it, and I hope it did at least pique some people’s
interest!

Advertisements

11 thoughts on “Intelligence Squared debate: Don’t end the tyranny of the test

  1. fish64

    I agree with everything you say here. In addition, I would say the issue goes beyond the culture of teaching exam rubrics and encompasses the whole approach to teaching academic subjects. Reading your blog, it occurred to me that APP and its tick box approach to learning academic subjects is slowly killing off the joy of learning. At the risk of sounding like a conspiracy theorist, I do wonder whether APP is a deliberate policy to drive teachers to distraction by focusing on checklists and interventions. Once this has been done and teacher workload has gone through the roof, teachers will happily abandon teaching discrete subjects and embark on a project/skills based curriculum instead! The abolition of levels has caused dismay among proponents of APP, but in my own experience SLT are desperately trying to recreate it.

    Reply
    1. wbhs62

      As someone who played a significant part in the development of APP I can assure you that tickboxes played no part whatsoever in the picture and were anathema to us all. The central basis of APP was to acknowledge the manifestation of understanding wherever it was to be found, inside or outside the classroom, written or otherwise. We recognised, and abhorred, the tendency to create lists of microskills and demand concrete evidence for each. Any curriculum leader or adviser who requires teachers to check off a hundred tickboxes and calls the process APP is wholly mistaken.

      Reply
      1. edpodesta67

        Your comments echo something I heard Dylan Wiliam say once on a video about his approach to formative assessment. He berated himself for failing to explain to the government of the day that formative assessment was nothing to do with putting marks in a mark book, or recording grades on a spreadsheet. I wonder whether the impact of new public management policies and paradigms means that any attempted educational improvement will tend towards reduction to tick-boxes and spreadsheets?

  2. julietgreen

    I agree with what you say and have often felt rather lonely in my insistence that ‘teacher assessment’ is not the panacea it would appear to be in some assessment rhetoric. It has formed a good deal of the content of my blogs, for example:

    https://julietgreen.wordpress.com/2015/08/28/the-nonsense-of-teacher-assessment-an-analogy/

    I believe it’s the ‘cop out’ solution when real solutions have not been properly considered. I have also been critical of assumptions about rubrics and their inherent unfairness to the pupils/students.

    Has there been a cultural shift in attitudes towards exams or was I missing something when I was sitting ‘o’ and ‘a’ levels all those years ago? Our teachers could not ‘teach to the test’ except by teaching the subject properly. These were exams that I didn’t feel were imposed upon me unfairly, but, much like the driving test, a certification that I had acquired a certain degree of knowledge and competence.

    I think there is much to be gained from a reconsidering of assessment, particularly with what is possible through technology and I suspect that much of the resistance to technologically enhanced assessment is the same resistance to testing itself.

    Reply
    1. Francis Michael Farrell

      I suspect that one reason it was difficult to teach to the test in the past might be that so little was known about them. If schools were a ‘secret garden’ then the exam board were a top secret one.When I started teaching in 1980 we did not have access to mark schemes, exemplar materials or books written by the exam board (well, at least we didn’t in my school). Also, we had multiple choice questions for comprehension. To the best of my memory,they were quite rigorous. We didn’t have access to advice like this, though. http://donaldclarkplanb.blogspot.de/2014/12/how-to-outguess-multiple-choice-tests.html

      Reply
  3. Pingback: Testing – a double edged sword. | Esse Quam Videri

  4. Pingback: Tyrannosaurus test | tlamjs

  5. Robert Craigen

    I understand the critique of “teaching to the test” but it seems to me that if one teaches to a properly designed test this ought to be tantamount to teaching to the curriculum, against which I rarely hear people speak. Can tests be designed so that the most advantageous approach to preparing for them is to simply master the curriculum being tests? Not only do I think so — I believe this is the very purpose of testing. It would be lovely if tests examined one’s later preparation for life, or one’s propensity for success in adulthood. But why would we even talk about such things while evidence is on the table that many tests do not succeed in their immediate purpose? That is fixable, and we should do so.

    How can a test be designed so that “teaching to” it is tantamount teaching to the curriculum? It should simply be designed so that any element of the curriculum MAY be tested; teachers do not know in any particular instance which subset is to be tested, and that the emphases intended in the curriculum are reflected in the relative weighting of material on the exam. It’s not rocket science. Beyond that there remains the question of making a test that does these things effectively, efficiently and to the greatest degree possible, with objective fidelity. But first, one must match the test to what it is intended to test so that the most effective way to “game” it is to actually learn that material well.

    Reply
    1. julietgreen

      “It should simply be designed so that any element of the curriculum MAY be tested; teachers do not know in any particular instance which subset is to be tested, and that the emphases intended in the curriculum are reflected in the relative weighting of material on the exam”

      And this is exactly what tests were, when I took A levels, for example. So how did that change?

      Reply

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s