One of the main reasons why people say we need to keep national curriculum levels is because they provide a common language.
I am all in favour of a common language, but levels did not provide this, as I have argued before here. Since I wrote that last post, I have come across this fascinating paper by Peter Pumfrey. It was written nearly twenty years ago, when levels were first introduced. It looks at the results of pupils in the KS1 reading tests. It is summarised by Bonnie Macmillan in Why School Children Can’t Read:
An investigation comparing pupils’ standardised reading scores with their level of attainment on national curriculum tests is starkly illustrative. Children who had been assessed as having attained level 2 (the average expected for their age) on national curriculum tests were found to have reading ages, determined from standardised testing, ranging from 5.7 to 12.9 years. That is, within the group of pupils all categorised as level 2, there was an incredible 7 year range in the actual reading abilities represented. Similarly, those categorised as level 1 were found to have reading ages ranging from 5.7 to 9.6 years.
Even though I was well aware of all the problems with levels, I was still astonished to read this. Not only does the level 2 category include pupils of such differing attainment as to be practically meaningless, it also significantly overlaps with the level 1 category. That doesn’t look to me as though levels are giving us a common and shared understanding.
Although I know of no similar research which has been done more recently, a look at the distribution of levels in the KS2 tests suggests that there is something similar going on. In the KS2 tests, approximately 15% of pupils get a level 3 or below, 50% get a 4 and 35% get a 5. So the number of pupils achieving a level 4 – that is, national expectations – runs from approximately the 16th to the 65th percentile. I suspect if we did a reading age test on all of these pupils, we would find huge variations in their results. Anecdotally, I know of plenty of secondary schools who find that some of their level 4s have difficulty with reading and are placed in their catch-up reading classes. So again, how useful is it, and how much of a ‘common language’ do levels provide if the level 4 range runs from pupils who are still struggling with reading and writing up to pupils who are confident readers and writers. One of the reasons why so many secondary schools reassess their pupils on entry (CAT4, for example, is used in over 50% of UK secondaries) is because the KS2 SATs do not provide a common language or that much in the way of useful information.
These vague bands cause further problems for secondary schools because they are used as the baseline for measuring progress across secondary. Expected progress for all level 4s is a C at GCSE. 84% of pupils in the top third of that level 4 category do go on to achieve a C or above at GCSE. But only 50% of those in the bottom third of the level 4 category do. (These figures are for English; they are similar for Maths). Schools who have a lot of pupils clustered in the bottom part of the level 4 category are being held to very tough attainment targets. Schools with lots of pupils clustered at the top of that level get relatively easy attainment targets. And of course, in practice, schools will not get fair spreads of level 4 pupils. Schools in some areas will take on a disproportionate number of ‘low’ level 4s, whereas other schools will get a disproportionate number of ‘high’ level 4s.
So, in conclusion, national curriculum levels do not provide a common language and this results in many pernicious effects. As for what could provide a common language, I will return to this in my next post.
Life beyond levels? Life after levels? Life without levels? Lots of teachers, senior leaders and academics have come up with some interesting ideas for what should replace national curriculum levels. Here’s a summary of some of those ideas.
- Michael Fordham is a former history teacher and now works at Cambridge’s education department. He has written three articles which put forward a possible system for assessing history – one, two, three.
- Alison Peacock is the head of Wroxham Primary School, who moved away from levels a while ago. In this post she expresses a worry that any list of aims she writes up will become APP under another name.
- Alison Peacock was also a part of the NAHT commission who recently released a report on this.
- The NAHT report attracted quite a few comments. I’m in broad agreement with David Thomas’s post here, particularly the point he makes about how easy it to say you should assess pupils according to objective criteria, and how hard it is to actually achieve this. (See below for Paul Bambrick-Santoyo’s work on this). Gifted Phoenix also commented on it here.
- Tom Sherrington is a secondary head. I like the focus here on taking actual samples of pupil work as definitions of standards.
- Phil Stock is an English teacher who has shared his department’s plans. They involve new rubrics for assessing reading and writing, and the use of multiple choice questions.
- David Thomas is a head of maths and in this proposal he notes that there is a tension between providing teachers and students with useful feedback and providing teachers and students with a system that is easy to understand.
- Joe Kirby is an English teacher at a London secondary. A lot of the ideas in this post are ones Joe and I have discussed together. This post and this one expand on the issues.
- Michael Tidd has put forward a proposal for primary assessment here, and has made more specific proposals about a mastery approach to assessment here, with an interesting comparison to a game of Jenga.
- Alex Quigley has a draft model for assessing English here.
- Chris Waterworth has written about a possible approach for primary assessment. I’m less of a fan of this approach, as it suggests simply using the current level descriptors and APP grids, just without the levels. The problem with level descriptors is well described by …
- …Paul Bambrick Santoyo, who has written some fascinating things about the difficulty of using prose descriptions of standards as a guide for assessments. Pages 6-8 of Driven by Data explain exactly what the flaws are.
- Rob Coe has come up with a list of 47 criteria you should consider before you let a test into your classroom.
- GL Assessments have two excellent articles on their website about assessing without levels. The first one, here, explains the purpose of standardised tests and how they could feature in a world without levels. The second article, here, offers a case study of St Peter’s Collegiate School in Wolverhampton which abolished levels in 2009.
- Finally, a bit of light relief: this clip from This is Spinal Tap reminds us that whatever scale we use, it has to have some underlying meaning.
I know there are some people who are disappointed that levels are going, fearing that we will lose a common language. I am not worried at all. I’m delighted at how many people are seeing the abolition of levels as an opportunity. I am also much less worried about the loss of a common language, because I don’t think levels really did provide a common language. I have written about this before here. Since I wrote that, I came across this paper by Peter Pumfrey, which shows that a group of pupils who achieved a level 2 in the KS1 teacher reading assessments had reading ages ranging from 5-10. In these circumstances, can we really say that levels provided a common language? Rather, it seems to me that they provided the illusion of a common language, which is actually far worse than having no common language at all.
My book, Seven Myths about Education, was published this week by Routledge.
It was actually first published as an ebook by The Curriculum Centre in June 2013. There are a couple of new additions to this version. There are some slight alterations to chapter two, and, more significantly, forewords by E.D. Hirsch and Dylan Wiliam.
There’s a nice review of it here by Joe Kirby.
Before publication I wrote a short summary of each chapter. Here they are.
- Facts prevent understanding
- Teacher-led instruction is passive
- The 21st century fundamentally changes everything
- You can always just look it up
- We should teach transferable skills
- Projects and activities are the best way to learn
- Teaching knowledge is indoctrination
After publication, I also wrote three posts restating the evidence base for the book and responding to some of my critics.
Myth 7 of my book is ‘Teaching knowledge is indoctrination’. I found lots of influential educationalists who believed this, but I did also feel that it was not the most pervasive of the myths I identified. Generally, I find the problem is not that people think that teaching Romeo and Juliet is indoctrinating pupils with the cultural products of dead white European men. More, the problem is that Ofsted think that making puppets is an acceptable way of teaching Romeo and Juliet.
Most of the influential people who believe in this myth wrote their first works in the 1970s. Whilst many of them are still around today, I think their work shows signs of being a bit dated. Significantly, one of the most important promoters of this myth, Michael Young, has actually recanted. In 1971 he edited a collection of essays called Knowledge and Control which was one of the seminal works of the ‘teaching knowledge is indoctrination’ school. He’s since published a book called ‘Bringing Knowledge Back In’ which argues for the importance of teaching what he calls ‘powerful knowledge’.
So, even though some traces of this myth persist, I would have said that on the whole, the belief that knowledge is indoctrination was going the way of the mullet, Love thy Neighbour, and other unlamented aspects of the 1970s. Just as I was thinking that, however, up popped this article in the Guardian by the deputy head teacher Tait Coles.
Interestingly, Mr Coles and I seem to have very similar aims for education.
Teachers can’t ignore the contexts, culture, histories and meanings that students bring to their school. Working class students and other minority groups need an education that prepares them with the knowledge of identifying the problems and conflicts in their life and the skills to act on that knowledge so they can improve their current situations.
I agree with all this. Where Mr Coles and I would depart is the best way to achieve these aims. Mr Coles compares two different approaches to curriculum and pedagogy – that of E.D. Hirsch’s and Paulo Freire’s. For him, Paulo Freire’s methods are far superior.
In contrast, ED Hirsch’s Core Knowledge Curriculum is a ‘hegemonic vision produced for and by the white middle class to help maintain the social and economic status quo’ and ‘teaching a “core knowledge” instils a culture of conformity and an insipid, passive absorption of carefully selected knowledge among young people…Schools that adopt this method become nothing more than pipelines producing robotic citizens, perpetuating the vision of a capitalist society and consequently preventing social mobility.’
Mr Coles offers no evidence for this assertion, and it is hard to see any way in which this criticism is justified. I would genuinely like to see the evidence and logic which led him to this conclusion. I don’t want to accuse him of not having read the Core Knowledge curriculum, but it is quite hard to see how anyone could have read it and come up with this conclusion. The CK curriculum includes speeches by Martin Luther King and Sojourner Truth, units on reformers such as Lucretia Mott and Elizabeth Cady Stanton and texts such as Jacob Riis’s How the Other Half Lives, a seminal work of photojournalism which exposed the inequalities of late 19th century New York. If you commissioned someone to design a curriculum that ‘deliberately failed to consider the values and beliefs of any other particular race, class or gender’ and they came back to you with the Core Knowledge Curriculum, you’d send them away to start again. Anyone who has read it or seen it in action will know that the CK curriculum is inclusive, global and multicultural. Indeed, its global and multicultural focus has seen it become the target of criticism from the religious right in America.
Another thing you wouldn’t know from reading Mr Coles’s piece is that Hirsch and Freire both have progressive aims. Where they differ is in how they think you should achieve such aims. As Mr Coles acknowledges in his article, ‘critical pedagogy isn’t a prescriptive set of practices – it’s a continuous moral project that enables young people to develop a social awareness of freedom.’ This vagueness can make it hard to work out the practices Freire advocated. As part of my research for my book Seven Myths about Education, I read some of Freire’s works and those of Freirean practitioners and attempted to pin down exactly what he was proposing. I concluded that for Freire, the very act of transmitting knowledge is suspect and regressive. Instead, his critical pedagogy involves teachers working with the knowledge pupils already have and with the knowledge pupils are able to discover independently. The problem with this is that the knowledge pupils can discover independently is always going to be limited. Discovery learning is a wholly inefficient way of acquiring knowledge. The knowledge pupils already have is always going to be unequal, and unfortunately may also diverge along socio-economic lines. In modern Britain, it is also the case that the type of knowledge pupils will pick up from the environment will very likely come from the mass media, whose primary focus is often entertainment, not truth. As Harry Webb has argued, a pupil whose only knowledge about Winston Churchill is from the mass media would be in no position to critique Churchill’s reputation. Mr Coles himself accepts that education should ‘challenge the accepted social truths purveyed by media.’ However, a Freirean discovery-based critical pedagogy will not achieve this. It will actually just give more power to media distortions. Thus, if we are so worried about indoctrination that we teach pupils no knowledge, one result is that we actually end up outsourcing the transmission of knowledge to the mass media, which is far more likely to result in indoctrination and bias. Another consequence is that we end up entrenching and reinforcing existing class divisions. Interestingly, one of the secretaries to the Plowden Report recognised this. ‘This view of education, naturalistic, heuristic and developmental as it was, was in some unremarked conflict with the Committee’s thinking about education as a redistributive agency.’ In short, discovery learning and social justice are in conflict.
The alternative to this approach is to accept that knowledge transmission does carry with it the risk of indoctrination, but that it is also an inevitable part of teaching, and the foundation of all skill. Given these three things, then teachers and schools should take great care over the selection of that knowledge and should most certainly not leave it up to the chance of a pupil’s background or the whims of a TV producer. (There are important questions that will remain about how you choose the knowledge, who chooses the knowledge, and what knowledge you end up choosing – but these are questions that have to be answered, not questions that demolish the possibility of teaching knowledge. They are real questions, not rhetorical ones. I discuss some of the answers to them in chapter 7 of my book, and will discuss it at more length in my next blog.)
Broadly speaking, the former approach is taken by Freire, and the latter by Hirsch (and those in the early labour movement); the former approach is not backed by evidence, and the latter is. Thus, whilst Hirsch and Freire both have progressive aims, Freire’s methods simply haven’t been as effective as Hirsch’s. If we compare the empirical and theoretical evidence in favour of a Hirsch style curriculum and a Freire style curriculum, I am afraid there is no contest. The principles of the CK curriculum are based on a solid understanding of cognitive psychology and the specific curriculum has performed excellently in practice in a number of research studies, including the impressive Core Knowledge Language Arts programme which was shown to be particularly beneficial for precisely the types of disadvantaged pupils Mr Coles is worried about. There is no such evidence in favour of Freire’s pedagogy.
So, to sum up, whilst Hirsch and Freire may both be motivated by the right ideas, only Hirsch is motivated by the right methods.
From my perspective, one of the good things about Mr Coles’s article is that whilst it grossly misrepresents Hirsch, it doesn’t ignore him. Five years ago it was hard to find someone in English education who had heard of Hirsch, or Dan Willingham, or any of the evidence in favour of a content rich curriculum. As I say in my book, the real ‘hegemonic vision’ is not in Hirsch’s curriculum, but in the exclusion of Hirsch and others like him from so many teacher training curriculums. For years, the education establishment has not had to argue against people who opposed its world view because it had effectively airbrushed them out of the debate. In the last few years, things have changed. People can no longer ignore the accumulation of evidence against so many of these dominant ideologies. Instead, they misrepresent and attack this evidence. Moving from being ignored to being attacked may not seem like an improvement, but it is. For every person who reads Mr Coles’s articles and nods in agreement, I think there will be one whose interest is piqued enough to want to find out more about this Hirsch chap. The playing field is levelling. ‘Let truth and falsehood grapple; who ever knew truth put to the worse in a free and open encounter?’
In my previous two posts (here and here), I looked at the structure of my book and restated some of the evidence I’d used to make the claim that a certain set of ideas were dominant in English education. In this post, I want to restate the evidence I used to back up my second claim: that these ideas are misguided. Essentially, the evidence here is fairly straightforward and derives mostly from cognitive psychology. In summary, working memory is limited; long-term memory is powerful; and we remember what we think about. Here’s a summary of just some of the evidence about this.
Dan Willingham – Why Don’t Students Like School?
Kirschner, Sweller & Clark – Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching
Herbert Simon – Skill in Chess and Expert and novice performance in solving physics problems
John Anderson – A Simple Theory of Complex Cognition
Here are two other excellent articles which I don’t cite in my book but which deal with very similar ideas.
Greg Yates – “How Obvious”: Personal reflections on the database of educational psychology and effective teaching research
Richard E.Mayer – Should There Be a Three-Strikes Rule Against Pure Discovery Learning? The Case for Guided Methods of Instruction
These three key facts about memory have huge implications for classroom practice. It’s because working-memory is so limited that projects, authentic activities and discovery learning are so problematic. It’s because long-term memory is so powerful that we need to make sure pupils commit facts to memory, and that pupils who don’t already have background knowledge are not left to devise their own education. And it’s because we remember what we think about that we need to make sure all our classroom activities focus our pupils’ attention on what we want them to remember.
The myths I identify, and the practice they have influenced, are in direct opposition to all of this evidence. I will give just three examples of this here – one example of a lesson from Ofsted, one example of a statement from a popular educationalist, and one example of a popular non-governmental curriculum. I’m not going to repeat in detail the problems with each example – I’ve just followed it up with a quotation from the research literature.
Lesson from an Ofsted report
‘Pupils first matched each of the diverse group of party guests (baby mice through to a giant) to various balloons. Then they had to measure string of differing lengths (5cm to 2m) for tying onto the balloons for each guest. The higher level teaching assistant encouraged good debate between the pupils around whether the string should be measured and cut before tying, or tied first and then measured. She did not steer them towards the other approach when they decided to measure and tie the string first. The pupils wrestled with measuring the string after tying it to the balloons which enabled them to appreciate the difficulty of measuring accurately once the string was attached to the balloon. They also realised that some of the string was used up in tying it to the balloon. This led to good discussion around which approach should be taken. The pupils revised their strategy for the task, which they went on to complete successfully.’ (Link here.)
Evidence from research
‘Controlled experiments almost uniformly indicate that when dealing with novel information, learners should be explicitly shown what to do and how to do it.’ (Kirschner, Sweller, Clark)
‘Children who have the advantage of clear instructional cues will achieve understanding more readily than children expected to acquire knowledge via less directive teaching methods.’ (Yates)
‘Knowledge is changing so fast that we cannot give young people what they will need to know, because we do not know what it will be. Instead we should be helping them to develop supple and nimble minds, so that they will be able to learn whatever they need to.’ (Link here)
Evidence from research
‘In every domain that has been explored, considerable knowledge has been found to be an essential prerequisite to expert skill.’ (Simon, Expert and novice performance in solving physics problems)
‘Data from the last 30 years lead to a conclusion that is not scientifically challengeable: thinking well requires knowing facts, and that’s true not simply because you need something to think about. The very processes that teachers care about most — critical thinking processes like reasoning and problem solving — are intimately intertwined with factual knowledge that is in long-term memory (not just in the environment).’ (Willingham, Why Don’t Students Like School?)
RSA Opening Minds curriculum
‘Children plan their work, organise their own time and explore their own ways of learning.’ (Link here.)
‘With the primary sector’s more cross-cutting and discovery-based approach to teaching and learning we expect there to be a natural fit with how Opening Minds has evolved at Key Stage 3.’ (Link here.)
Evidence from research
‘Constructivism too often is seen in terms of student centred inquiry learning, problem-based learning and task based learning and common jargon words include “authentic”, “discovery” and “intrinsically motivated learning”. The role of the constructivist teacher is claimed to be more of facilitation to provide opportunities for individual students to acquire knowledge and construct meaning through their own activities, and through discussion, reflection and the sharing of ideas with other learners with minimal corrective intervention. These kinds of statements are almost directly opposite to the successful recipe for teaching and learning as will be developed in the following chapters.’ (Hattie, Visible Learning, p.26)
‘In all major domains, an accumulation of effective methods has occurred for teaching the accumulated knowledge and skills…teachers know how and to what degree of mastery the simpler tasks have to be acquired to serve as the building blocks of more complex skill. Unlike the beginners themselves, teachers can foresee the future demands and avoid the need for complete relearning of previously obtained skill.’ (K. Anders Ericsson, link here)
E.D. Hirsch has used the insights we have from cognitive psychology to formulate some ‘middle axioms’ for classroom practice. (I blog about this here). Middle axioms are general theoretical principles that can guide our practice. Hirsch’s seven are as follows.
• Prior knowledge as a prerequisite to effective learning.
• The right mix of generalization and example.
• Attention determines learning.
• Rehearsal (repetition) is usually necessary for retention.
• Automaticity (through rehearsal) is essential to higher skills.
• Implicit instruction of beginners is usually less effective.
I would suggest that these seven middle axioms are a better guide to educational practice than the seven myths I identify. Unfortunately, it’s the latter which currently guide much of our education system.
I have noticed that a common response to my book has been a) to deny the existence of the myths I’ve outlined and b) to claim that they are not myths after all.
This is not only rather illogical, it’s also something I anticipated prior to publication in this blog post.
Very often, I’d give a brief outline of what I thought about education and explain what that meant in practical terms – for example, teaching discrete grammar lessons. I would then get two responses, often from the same person. First, the person would say that most schools do what I am asking already, so what I am proposing isn’t anything new – eg, they would say, all schools teach grammar anyway. Second, they would say that I was backward looking and wanted to take education back to the 19th century. Self-evidently, both these criticisms cannot be true. If all good schools already do what I am asking, then I can’t be advocating a return to the 19th century. If I am advocating a return to the 19th century, then schools can’t all be doing what I am asking.
Similar criticisms would emerge whenever I tried to give an example of something I thought was wrong. So, I might say that a lesson where pupils learnt about Romeo and Juliet through making puppets was not very effective. Again, I would get two criticisms: first, the person would say that I was attacking a straw man and that nobody really taught like that. Second, they would say that making puppets to teach Romeo and Juliet was very effective. Again, you can’t really make both criticisms. If you think that I am attacking a straw man, then you are implicitly conceding that teaching Romeo and Juliet through the use of puppets is not effective. So going on to argue that such a method is effective is contradictory. Again, both criticisms are wrong. Making puppets to teach Romeo and Juliet is not an example of a straw man; it is an example which has been cited as best practice. It is also, in actual fact, ineffective practice.
Since I wrote that, I have come across a paper by the cognitive psychologist Greg Yates where he records getting exactly the same response to one of his early research findings. In his words,
At the seminar, various critics noted the findings as (a) obvious, and (b) in conflict with Piagetian theory. A strange thing for a young graduate’s findings to be seen simultaneously as obvious and at variance with one of the field’s major statements.
In order to try and forestall these types of criticism, I realised that what I needed to do was to show people not just that statement x, y and z were myths, but also that lots of people actually believed in statements x, y and z. Hence, the structure of my book. I am making two claims: one, that people believe in these myths; two, that they are indeed myths. Only once I have shown beyond doubt that people believe in a myth do I explain that it is a myth.
Gratifyingly, a number of people have said that they found this structure very useful in helping them to understand the way theory influences practice. However, despite structuring the book like this, I’ve still encountered the logical fallacy I outline at the start. First, my critics will deny that anyone believes in the myths I outline and attack the evidence I’ve used to show this. OK, fair enough – I disagree, of course, but so far so logical. But secondly, they then say that the myths I’ve outlined are not, in fact, myths.
That’s a bit like saying this. ‘Daisy, stop being stupid. No-one believes in statement x. Statement x is a straw man. You’ve created this straw man from your limited view of classroom practice. If you had more experience, you’d realise that no-one believed in statement X. Oh, and by the way, statement x is true! It must be, I read a book about it!’ On the one hand, my critics claim that I have a distorted view of the reality of classroom practice. In the next breath, they defend this exact same reality in the exact same terms as I have described it!
Here, for example, is Tom Sherrington. First, hardly anyone believes in statement x.
‘For me, the myths just don’t ring true as a general description of the state of our schooling or the issues with it.’
‘There are quite a few references to RSA’s Opening Minds and passing references to Guy Claxton. But their ideas are only used directly in a tiny sample of schools; they don’t represent the system in any way.’
Second, statement x is true.
‘I have to be open minded about the eventual outcomes [of projects].’
‘The example of Y4s talking about health and safety prior to going on a trip seemed perfectly reasonable.’
‘This Y9 History lesson, comparing bombing campaigns, sounds great to me.’
There are, obviously, some logical ways you can argue against my book, even if I think they are wrong. You can argue that my myths don’t exist – ie, that the evidence I produce to substantiate the myths isn’t strong enough, and the myths are not the problem that I claim. That is, not many people really do believe in statement x. I discuss the evidence that I use for this first claim and the criticism I’ve faced for it here.
Or, you can say that the myths are not myths after all. That is, they are sound and rational beliefs. In other words, people are right to believe in statement x because it is, after all, true.
You could also argue that I am wrong on both counts. In order to this, you’d have to say that hardly anyone believes in the myths I am outlining – which is a real shame, because they aren’t myths, they’re the best way to teach. Everyone should believe in them. In this case, the fact that you disagree with me on the first claim is actually of less importance. The disagreement over the second claim is more important.
But what you can’t do is to say that my depiction of the myths is a straw man – and then go on to say that they aren’t myths. That is, you can’t argue that statement x is a straw man AND that statement x is in fact true. You can’t claim I am attacking a straw man, before going on to show that the alleged ‘straw man’ is something you are in complete agreement with. If you do so, you are actually inadvertently giving me evidence for my first claim – that these myths really do exist – and proving that, after all, the alleged straw man is not so straw after all.
For me – and I think, in reality, for most of my critics – the really interesting claim is the second one. Even if you don’t think the myths are the systemic problem that I do, there is enough evidence to show that they are present in places. So it would be nice if we could at least agree on this, and then move on to the more interesting and educationally meaningful second claim – are the myths really myths? Are the types of lessons I criticise really deserving of criticism? That will be the subject of next week’s post.
In Seven Myths about Education, I make two claims: first, that in English education, a certain set of ideas about education are predominant; second, that these ideas are misguided. Finding the evidence to prove the second point was relatively straightforward. It is scientifically well-established that working memory is limited and that long-term memory plays a significant role in the human intellect. This has clear implications for classroom practice, implications which others have made and which I was happy to recap.
However, there was no one evidence base which could prove (or indeed, disprove) the first claim. Instead, I identified a range of different evidence bases to prove my point that in English education, the seven ideas I discuss are predominant. (In doing so, I also defined clearly what these ideas meant in theory and practice.) Here’s a recap.
• The writings of prominent theorists, and the proof that such theorists are indeed prominent (eg book sales, presence on government committees, the judgment of their peers, etc.)
• The advice given in popular teacher training textbooks, and the proof that such textbooks are popular (eg book sales, presence on reading lists)
• The National Curriculum
• Ofsted reports – I created an appendix of 228 exemplar lessons that were described in Ofsted subject reports from the last three years. These exemplar lessons were in turn drawn from the thousands of lesson observations done by Ofsted inspectors over the previous 3-5 years. See here for more information about this and a link to the appendix, which is available for free online.
• Popular non-governmental curriculums (and the evidence of such popularity, eg by number of schools applying it)
• Examples of lessons from popular resource sharing websites
I could have included a lot, lot more examples from the latter category which would have proved my point, and then some. I deliberately chose not to because it is hard to tell how popular or influential such lessons are. Had I used a lot of these, it would have been easier to accuse me of simply cherry picking the worst examples I could find on the entire internet. So I went easy on these, and only used a few examples from such websites, and then only from websites I could demonstrate were popular.
Given this range of evidence, I found it odd that some people criticised the book as being reliant on anecdotal evidence. As I say in the introduction, I do add in the odd anecdote to try and liven up the text, but only when I have clearly established that my anecdotal experience is in line with the evidence. In his review of my book, Tom Sherrington refers to my book being based on ‘personal anecdotal experience…from a very specific teaching situation’. That is absolutely not the case. It’s also particularly baffling that Tom Sherrington would say this given that the only counter-evidence he brings to bear is his own anecdotal experience. His strongest counter-argument is that ‘for me, the myths just don’t ring true’.
Interestingly, when I first started writing the book, my instinct was that my own personal anecdotal experience would bear little relation to the wider system, simply because I couldn’t believe that an entire system would have endorsed beliefs that were so completely at odds with all the available evidence. I actually set out planning to write a book that critiqued some parts and aspects of the education system. It was a genuine surprise to me to see that clear examples of bad practice were being endorsed as good practice on a system-wide level.
In a largely positive review, Michael Fordham did criticise my use of Ofsted reports as ‘a reliable and unproblematic account of pedagogy’. I think he is right to say that Ofsted reports are problematic. In the book I do in fact discuss the strengths and weaknesses of Ofsted reports, conceding that,
The one flaw with these reports is that Ofsted inspections are pre-announced. This means that teachers can, and do, put on a show for Ofsted. It has long been a complaint of many teachers that the kind of lesson Ofsted grade as outstanding is simply not possible to repeat consistently. So, when I give an example of an outstanding lesson from an Ofsted report, I do not mean to suggest that lessons exactly like this are going on all the time in every lesson. However, given the power we have seen Ofsted have, given the cottage industry of Ofsted preparation and given the fact that most schools will organise their own internal observation systems around Ofsted criteria, it is still fair to say that if Ofsted demand a certain type of lesson, that matters.
Also, I am not just using Ofsted reports as evidence for how teachers teach. I am also using them as evidence for how teachers are told to teach.
So, Ofsted’s inspection reports and subject reports are a fairly reliable guide to what actually happens in schools and a very reliable guide to what teachers are told to do.
Given that I am trying to prove what the dominant views are in the English education system, finding out how teachers are told to teach is just as important as finding out how they actually do teach.
Finding completely reliable and valid evidence of how teachers in England teach is always going to be difficult. Even a well-funded research study would run into methodological problems. Any large-scale observation programme, however low-stakes, would face the Hawthorne effect. However, whilst finding completely reliable and valid evidence is hard, I still think the Ofsted reports, backed up by the other sources listed above, offer a fairly reliable and valid picture.
Also, finding reliable and valid evidence of how teachers in England are told to teach is nothing like as methodologically difficult. Essentially, it is as straightforward as looking at what Ofsted and the government tell teachers. I would argue that even if teachers completely and utterly ignored this advice, the advice would still tell us something important about the education establishment. And in any case, there is good evidence that teachers and schools do not and can not ignore Ofsted. Since I first published the book, I think events have reconfirmed the central importance of Ofsted and their judgments in the education system.
I am always interested in discovering new sources of evidence about classroom practice. If people think there are other important sources I’ve missed, I would really like to know. In Tom Sherrington’s case, he seems to be suggesting that his own personal experience is a more valid and reliable source of evidence than the list I’ve outlined above. In Michael Fordham’s case, he doesn’t put forward any alternative. If anyone wants to suggest any others in the comments thread, I’d be glad. (Two very interesting sources which in the end I didn’t use were the PISA and TIMSS teacher, school and student surveys. These are really fascinating and have a lot of interesting data in them. Liz Truss and Laura McInerney, amongst others, have used these surveys to make interesting points. But they didn’t have the level of detail about classroom practice that I was looking for. TIMSS have done video surveys of some classrooms, but unfortunately not, so far as I know, of English classrooms.)
In any case, as I will go on to discuss in my next post, a lot of the arguments about the evidence base for my first claim are actually just a rather illogical smokescreen for people who are really interested in challenging my second claim.