Author Archives: The Wing to Heaven

My top 10 education books of 2018

In no particular order…

Learning How to Learn by Barbara Oakley, Terrence Sejnowski and Alistair McConville
This book is a fun, witty and very, very practical summary of how we learn, and how we can learn better. Oakley and Sejnewoski are the creators of the world’s most successful online learning course, and in the book you can see why – it is always engaging and interesting. This book is designed for children and teenagers, but there’s a grown-up version from a couple of years ago too. I reviewed the former in more detail for the TES earlier this year. If you know students who are worried about their summer exams, buy them a copy!

How I Wish I’d Taught Maths by Craig Barton
I’m a former English teacher, so you might not think I’d get much out of this – but actually it’s an amazing guide to how to teach anything. There’s a lot of detail here, but it’s always engaging and useful. My favourite chapter was chapter 6 on making the most of worked examples. We hear a lot about worked examples, but they are not always easy to use in practice, and this chapter really gets to the heart of what makes them work.

The Tyranny of Metrics by Jerry Muller
Not a book about education per se – but the issue of how metrics corrupt performance is so vital to education that Ofsted’s Chief Inspector name checked the book in a speech, and the TES reviewed it too. I didn’t agree with everything the book said, and it could have benefitted from more acknowledgement of the very real flaws of human judgement – but nevertheless this book is essential reading for anyone involved in dealing with white-collar accountability systems. One of its final conclusions is a useful lesson for life too: ‘recognising the limits of the possible is the beginning of wisdom’.

The Big Ideas in Physics and How to Teach Them by Ben Rogers
As with Craig’s book, this is not the most obvious choice for a former English teacher, but it’s also a great exemplar of how to teach complex material. The stories about scientific discoveries are fascinating, and the explanation of the different meanings words like ‘charge’ have in science and in everyday life illustrate the difficulty of using words to provide precision and the dangers of relying on definitions to provide a shared meaning.

The Coddling of the American Mind by Jonathan Haidt and Greg Lukianoff 
The central thesis of this book is incredibly thought-provoking and stimulating: that the rise in mental health problems amongst young people in the US and UK has its roots in distorted emotional reasoning that is encouraged by modern politics. I disagreed profoundly with the 10% of the book which dealt with school education, but thought the other 90% was excellent. The line that’s stayed with me: “[having your most cherished beliefs challenged] is not harassment or a personal attack; it is part of the process by which people do each other the favour of counteracting each other’s confirmation bias.”

The Truth about Teaching by Greg Ashman
Greg’s blog has long been a prolific resource for anyone wondering about the research basis of a particular educational strategies. This book gives a concise summary of what we do and don’t know about teaching, with the focus on how this can practically help in the classroom.

Responsive Teaching by Harry Fletcher-Wood
I love the title of this book, which is a reminder that a vital part of formative assessment is about responding in real-time in the lesson – not waiting to trawl through written work days later. The chapter I enjoyed most was the one on ‘How can I tell what students learned?’ It has an extended discussion of the difference between performance and learning, and a reminder to ‘check what students know at the end of a lesson to identify problems rapidly, not because they will remember the lesson’s content forever’.

Creating the Schools our Children Need by Dylan Wiliam
Dylan Wiliam’s work on formative assessment has been an enormous influence on me, and probably just about every teacher in the UK. This book is about improving the US school system, but its broader themes are relevant to any country. It’s incredibly well-written and pithy, and covers an enormous amount of ground. There are some of the classic Wiliam hit tunes on classroom assessment and teacher improvement (‘love the one you’re with’!), but also some new and thought-provoking takes on memory and knowledge-based curriculums.

Understanding How We Learn: A Visual Guide by Yana Weinstein and Megan Sumeracki 
A Visual Guide to learning, for visual learners…only kidding! Learning styles are just one of the fads that’s debunked in this very readable guide to the research teachers need to know about. Featuring great illustrations by Oliver Caviglioli.

Thinking Reading: What every secondary teacher needs to know about reading by Dianne and James Murphy 
Teaching pupils to read is perhaps the most important part of any school system. And yet in many countries, including the UK, a significant minority of pupils leave school functionally illiterate. And in England at least, many secondary English teachers will tell you they do not feel prepared at all to teach children how to read, as the system assumes that every pupil will have learnt to read by age 11. This book plugs an important gap.

Finally – here’s two other books I enjoyed which feature small contributions from me.

Wholesome Leadership by Tom Rees 
This book looks at the challenges of being a head, and of using research, policy and moral purpose to inform your leadership. I particularly liked Tom’s honesty about his own Ofsted inspections, the comparisons between education and architecture and medicine, and the analogy with Test match cricket: “School improvement, like test match cricket, can be a game of attrition”! Each chapter features interviews with various education people – one with me.

Natural-Born Learners by Alex Beard
This book is an entertaining tour of global education – from the ubiquitous Finland and Korea, to the trendy US west-coast, via an interview with me in an office in London!


Global Education and Skills Forum 2018

Last weekend I spoke at the Global Education and Skills Forum in Dubai. I spoke for the motion in the following debate: ‘”I can just Google it” is making us stupid.’ You can see the video here. I’ve put a transcript of my speech below, together with references.

In a letter to a friend, the ancient philosopher Seneca recounted the story of a rich Roman merchant who wanted to appear as though he was a very well-read man. This merchant decided that instead of actually reading books himself, he would instead hire a team of slaves to do it for him. “He spent an enormous amount of money on slaves: one of them to know Homer by heart, another to know Hesiod, while he assigned one apiece to each of the nine lyric poets. Then, he used these slaves to give his dinner guests nightmares: He would have these fellows at his elbow so that he could continually be turning to them for quotations from these poets which he might repeat to the company.”[1]

Of course, no one nowadays has slaves to remember things for them. But we do all feel very comfortable with the idea that we can outsource our memories to Google. In my book, Seven Myths about Education, I devoted a chapter to collecting examples of technologists and educationalists telling us that remembering things just isn’t necessary in a world with ubiquitous smartphones.[2]

These people are wrong, and they are dangerously wrong. And it is not just ancient writers like Seneca who tell us they are wrong. There is a whole body of modern scientific literature which makes the same point. Somewhat ironically, a great deal of this research derives from the work of Herbert Simon, one of the pioneers of artificial intelligence and modern computing. What we know from this research about how the brain works is that memory and attention are two vital parts of our intellectual equipment.[3] We also know that memory and attention are under siege from modern technology like never before.[4] Let us consider these two vital components in turn: why do they matter, and why are they under threat from technology?

First, memory. Our memories matter because we need facts stored in long-term memory in order to be able to think. This is because our working memory – what you might think of as consciousness – is extremely limited and can handle only about 4 – 7 new items of information. That isn’t nearly enough to do anything complex like driving a car, or reading a book. But we can cheat working memory’s limitations – not by hiring a bunch of slaves or using Google, but by committing facts to long term memory. This is why memorising times tables matters. When you solve a complex real world maths problem, you have to process a lot of information in working memory. If you also have to stop every second to type the times tables into your smartphone, your working memory will quickly be overwhelmed, and you will not be able to solve the problem. You’ll forget what the start of the problem was by the time you get to the end.[5] As one group of researchers have said, long term memory is the seat of human intellectual skill.[6] What we know influences how we see the world, how we think and how we reason. Intuition and creativity are the function of large well-memorised bodies of knowledge clashing against each other.[7] We can’t outsource this stuff.

If memory is so important, how do we make memories? The simplest answer is that we remember what we pay attention to – and that brings me to the second thing I want to talk about – attention.[8] If we pay attention to something, we are more likely to remember it. Our attention determines our memories. And nearly all of the major technology companies make their money by harvesting our attention, and selling it to advertisers.[9] These companies have invented increasingly sophisticated methods of grabbing our attention, even if it involves distorting the truth, manufacturing outrage, and exploiting loneliness.[10]  In the process, they don’t just distract our attention: they degrade its quality. Think how hard it is to concentrate on a book after spending an hour or so on social media.[11] Recent research shows that even the sight of a switched off phone makes it harder to focus.[12] Given the vital importance of attention for forming memories, a system that is built on stealing and degrading our attention cannot make us smarter.

At this point, people might typically say, but what about the good uses of technology? What about the Khan Academys, the Duolingos, the Courseras? What about Andrew’s platform Cerego, which uses the science of learning to design educational content that really will stick in long-term memory? And I agree that these kinds of websites are fantastic. They give billions of people access to quality educational content at low or even no cost, which is amazing. We on this side of the house are absolutely not opposed to educational technology. I work for an ed tech company. In my previous jobs as an English teacher I was always experimenting with different methods of online learning. What we are opposed to are misconceptions like the one in the title of this debate, that you can just Google it. Or, as one Google executive said recently, ‘I don’t know why children are learning the quadratic equation. I don’t know why they can’t just ask Google for the answer.” (See footnote 2). And in fact, the reason we are so particularly opposed to misconceptions like this one is that such misconceptions damage good education technology. They make it harder for the really powerful and effective methods of education technology to fulfil their potential, because the really effective education technology is not about outsourcing memory, but about making the process of memorisation as effective, efficient and fun as possible.

Not only that, but good forms of education technology are also being damaged by the tech companies’ insatiable appetite for attention. Online education courses have a phenomenally high drop-out rate. One study from 2014 showed that just 13% of people who enrol on an online course complete it.[13] Why is this? Plenty of reasons have been put forward, but I would like to suggest that one important reason is that because these courses are delivered online, they are therefore competing with everything else that online has to offer – the instant social updates, the flash shopping discounts, the cat videos, Donald Trump’s twitter feed. It isn’t enough to create fantastic educational content for free.[14] In order for it to make people smarter, people have to pay attention to it. And large numbers of them simply aren’t.

Of course one could imagine a world in which technology was used to make us smarter. I would happily sketch for you the outlines of a world where technology did make us smarter.[15] The point is that that is not the world we currently live in. The technology we use prioritises entertainment, outrage, distraction and convenience ahead of learning. By and large, the big money in technology is not going towards helping children to learn their times tables in the most efficient and fun way possible. It is going towards encouraging children to take another selfie, and to forget about the times tables because there’s a robot who will do it for them.

Seneca concluded his story of the Roman merchant with the following moral: “A sound mind can neither be bought nor borrowed.”[16] I would add the following modern updating. “A sound mind can neither be bought, nor borrowed, nor outsourced to the cloud.” And until we recognise that truth, Google will continue to make us stupider.

[1] Seneca: letters from a Stoic. Ed. Campbell, Robin. Penguin, 1969, Letter XXVII

[2] Christodoulou, Daisy. Seven myths about education. Routledge, 2014, chapter 4. Seven Myths was published in 2014; plenty of similar claims have been made since then, including, for example, here by Jonathan Rochelle, Google’s director of education apps: “Referring to his own children, he said: “I cannot answer for them what they are going to do with the quadratic equation. I don’t know why they are learning it.” He added, “And I don’t know why they can’t ask Google for the answer if the answer is right there.”

[3] EG, see Frantz R. “Herbert Simon. Artificial intelligence as a framework for understanding intuition.” Journal of Economic Psychology 2003; 24: 265–277. Simon also wrote explicitly about education here: Anderson J. R., Reder L.M. and Simon H.A. Applications and misapplications of cognitive psychology to mathematics education. Texas Education Review 2000; 1: 29–49. I discuss this paper in my blog post here.

[4] EG see Wu, Tim. The attention merchants: The epic scramble to get inside our heads. Vintage, 2017, also Teixeira, Thales S. “The rising cost of consumer attention: why you should care, and what you can do about it.” (2014). Simon also commented on the economics of attention here: Simon, Herbert A. “Designing organizations for an information-rich world.” (1971): 37-72. “In an information-rich world, the wealth of information means a dearth of something else: a scarcity of whatever it is that information consumes. What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.”

[5] Cowan N. “The magical number 4 in short-term memory: A reconsideration of mental storage capacity.” Behavioral and Brain Sciences 2001; 24: 87–114; Cowan N. Working Memory Capacity: Essays in Cognitive Psychology. Hove: Taylor and Francis, 2005. See also Miller G.A. “The magical number seven, plus or minus two: Some limits on our capacity for processing information.” Psychological Review 1956; 63: 81–97; More recently, Professor Daniel Willingham has written this New York Times article about this exact issue.

[6] Sweller J., van Merriënboer J.J.G. and Paas F.G.W.C. Cognitive architecture and instructional design. Educational Psychology Review 1998; 10: 251–296.

[7] Larkin, J., McDermott, J., Simon, D. P., & Simon, H. A. “Expert and novice performance in solving physics problems.” Science, 1980; 208(4450), 1335-1342, p.1335.

[8] Willingham D.T. Why Don’t Students Like School? San Francisco: Jossey-Bass, 2009, p. 53. William James also discusses attention in chapter 11 of The principles of psychology: ‘My experience is what I agree to attend to.’

[9] As Tristan Harris argues, the advertising model which underpins the modern technology economy means that companies ‘have an unbounded interest in getting more of people’s time on a screen’.

[10] See for example this article from the Guardian which investigates YouTube’s ‘Most Recommended’ algorithm and this on how Facebook uses information on users’ emotional states. See also Jean Twenge, in this article in the Atlantic and iGen: Why Today’s Super-connected Kids are Growing Up Less Rebellious, More Tolerant, Less Happy–and Completely Unprepared for Adulthood–and what that Means for the Rest of Us. Simon and Schuster, 2017. “The more time teens spend looking at screens, the more likely they are to report symptoms of depression.” See also Tromholt, Morten. “The Facebook experiment: Quitting Facebook leads to higher levels of well-being.” Cyberpsychology, Behavior, and Social Networking 19.11 (2016): 661-666.

[11] One small-scale study showed that undergraduates switch windows on their computers every 11 seconds on average. Yeykelis, Leo, James J. Cummings, and Byron Reeves. “The Fragmentation of Work, Entertainment, E-Mail, and News on a Personal Computer: Motivational Predictors of Switching Between Media Content.” Media Psychology (2017): 1-26.

[12] Ward, Adrian F., et al. “Brain drain: the mere presence of one’s own smartphone reduces available cognitive capacity.” Journal of the Association for Consumer Research 2.2 (2017): 140-154.

[13] Onah, Daniel FO, Jane Sinclair, and Russell Boyatt. “Dropout rates of massive open online courses: behavioural patterns.” EDULEARN14 proceedings (2014): 5825-5834.

[14] It should also be pointed out that whilst there is a lot of brilliant educational content on the internet, there are also a lot of educational claims made for websites, activities and games that are unlikely to lead to real learning. In his book Deep Work, Cal Newport points out the ‘absurdity of the now common idea that exposure to simplistic, consumer-facing products—especially in schools—somehow prepares people to succeed in a high-tech economy. Giving students iPads or allowing them to film homework assignments on YouTube prepares them for a high-tech economy about as much as playing with Hot Wheels would prepare them to thrive as auto mechanics.’ Newport also argues for the importance of attention, seeing uninterrupted ‘deep work’ as one of the main creators of value in the modern economy. Newport, Cal. Deep work: Rules for focused success in a distracted world. Hachette UK, 2016.

[15] For some suggestions, see the final chapter of my second book, Making Good Progress?: The future of Assessment for Learning. Oxford University Press, 2017.

[16] Seneca, ibid.

Research Ed 2017

This was the fifth national Research Ed conference, and in my mind they’ve started becoming a bit like FA Cup Finals or Christmas – recurring events that start to blur into one. “Oh, South Hampstead – was that the one where Ben Riley from Deans for Impact visited and it all kicked off about grammars?” “No, that was Capital City 2016South Hampstead 2015 was the one where Eric Kalenze visited and where James Murphy taught us the Maori word for green.” Etc. Looking back at my notes from 2013, I find that Ben Goldacre warned then against the ‘energy-zappers’ who will criticise everything you do – too true.

  • The title of my talk was: Improving assessment: the key to education reform.
  • You can download my slides here: Research Ed 2017
  • The livestream is here.
  • If you’re interested in finding out more about comparative judgement, one of the things I talked about, then there are still a few places left on our London training day later this week.

As ever, it is inspiring to meet so many people who are so committed and excited about the cause of research in education, and to be able to talk and share ideas with them. I always come away from these conferences with my mind buzzing with new ideas. Research Ed has only been around for four years, but I cannot imagine the world of education without it. Here’s to many more brilliant conferences.

Feedback and English mocks

You can also read this post on the No More Marking blog.

In the previous few posts, I’ve looked at the workload generated by traditional English mock marking, and at the low reliability, and I’ve suggested that comparative judgement can produce more reliable results and take less time. However, one question I frequently get about comparative judgement is: what about the feedback? Traditional marking may be time-consuming, but it often results in pupils getting personalised comments on their work. Surely this makes it all worthwhile? And beyond a grade, what kind of feedback can comparative judgement give you? This post is a response to those questions.

First, there’s a limit to the amount of formative feedback you can get from any summative assessment. That’s because summative assessments are not designed with formative feedback in mind: they are instead designed to give an accurate grade. So for the most useful kind of formative feedback, I think you need to set non-exam tasks. I write about this more in Making Good Progress.

Still, whilst formative feedback from summative assessments is limited, it does exist. When you read a set of exam scripts, there are obviously insights you’ll want to share back with your pupils, and similarly it’s always helpful to read examiners’ reports to get an idea of the common misconceptions all pupils make. I think we need to do fewer mock exams, because their usefulness is limited, but clearly when we do do them, we want to get whatever use we can from them.

So what is the best way for a teacher to give feedback on mock performance? The dominant method at the minute seems to be written comments at the bottom of an exam script. This is extraordinarily time-consuming, as we’ve documented here, and as other bloggers have noted here, here and here. What I want to suggest in this post is that these kinds of comments are also very unhelpful. Dylan Wiliam sums up why perfectly:

‘I remember talking to a middle school student who was looking at the feedback his teacher had given him on a science assignment. The teacher had written, “You need to be more systematic in planning your scientific inquiries.” I asked the student what that meant to him, and he said,“I don’t know. If I knew how to be more systematic, I would have been more systematic the first time.” This kind of feedback is accurate — it is describing what needs to happen — but it is not helpful because the learner does not know how to use the feedback to improve. It is rather like telling an unsuccessful comedian to be funnier — accurate, but not particularly helpful, advice.’

Wiliam, Dylan. Embedded formative assessment. Indiana: Solution Tree Press, 2002, p.120.

This might seem like a funny and slightly flippant comment, but actually it expresses a profound philosophical point put forward in the work of philosophers such as Michael Polanyi and Thomas Kuhn, which is that words are not always that good at explaining new concepts to novices. Often, part of what a novice needs to learn is what some of these words like ‘systematic’, or, to use an example from Kuhn, ‘energy’, really mean. If pupils don’t know what these words really mean, they can get stuck in a circular loop, similar to the one you might have experienced as a child when you didn’t know the meaning of a word, so you looked it up in a dictionary, only to find you didn’t know any of the words in that definition, so you looked those up, only to find that you didn’t understand the words in those definitions, and so forth…

Much more helpful than written comments are actions: things that a pupil has to do next in order to improve their performance. These do not have to be individual to every pupil, and they do not have to be laboriously written at the bottom of every script. They can be communicated verbally in the next lesson, and they can be acted on in that lesson too.

How does all this fit in with comparative judgement? One objection people have to comparative judgement is that whilst it may give an accurate grade, it doesn’t give pupils a comment at the bottom of their script. We’ve heard of a couple of schools where after judging a set of scripts, they’ve then required staff to go back and write comments on the scripts too. This is totally unnecessary and unhelpful! Instead, we’d recommend combining comparative judgement with whole-class marking. Whole-class marking is a concept I first came across on blogs by Joe Kirby and Jo Facer at Michaela Community School. Instead of writing comments on a set of books, you can jot down the feedback you want to give on a single piece of paper. You can formalise this a bit more by developing a one-page marking proforma, which gives you a structure to record your insights as you mark or judge a set of scripts, and to help you plan a lesson in response to the scripts. Here’s an example we’ve put together based on some year 7 narrative writing. The parts in red are the parts that involve teacher and/or pupil actions.

Caveat: this is written out far more neatly and coherently than is necessary — we’ve only done this to illustrate how it works. These proformas can be much more messy, as in Toby French’s example here. What’s important is the thought process they support, and the record they will provide over time of actions and improvements. In short, combining comparative judgement with one-page marking proformas will drastically reduce the time it takes to mark a set of scripts, and will give your pupils far more useful feedback than a series of written comments.

Our aim with our Progress to GCSE English project is to use tools like the one above to allow schools to replace traditional mock marking with comparative judgement. We ran our first training days in July, and will be running more in the autumn term. To find out more, sign up to our mailing list here. Our primary project, Sharing Standards, takes a similar approach, and you can read more about it here.

Workload and English mocks

You can also read this post on the No More Marking blog here.

Last weekend, I posted a question to English teachers on Twitter.

Most of the answers were in the range of 10 – 30 minutes. People also pointed out that the time it took to mark mocks varied depending on whether you wrote lengthy comments at the bottom of each script or not.

My own experience of marking the old spec GCSE English Language papers was that it took me about 15 minutes to mark each paper, which included some fairly brief comments. I also found it difficult to mark for more than about 90 minutes / 2 hours in one go, and if I did try and mark for longer than that, I would get slower and need to take more frequent breaks.

If we take 15 minutes, therefore, as a relatively conservative estimate, that means that if you teach 28 pupils, it will take you 7 hours to mark those scripts. That doesn’t include any moderation. If we assume a 90 minute moderation session for each mock, plus 90 minutes to go back and apply the insights from moderation, that means we are looking at a total of 10 hours.

That’s for one English Language Paper. There are two English Language papers, and two English Literature paper. So if you want pupils to do a complete set of English mocks, that’s a total of 40 hours of marking for the teacher.

With the old specification which included a lot of coursework, I think most English teachers spent the bulk of year 10 teaching and marking coursework essays, and didn’t get on to doing mocks until year 11. I was really pleased when coursework was abolished as I felt it would free up so much more time for teachers to plan and teach, instead of mark and administer coursework. However, it does appear as though a lot of this gained time has now been replaced with equally time-consuming mock marking, with mocks being introduced more and more in year 10. Many schools have three assessment points a year. If you were to do two mock papers three times a year in both year 10 and 11, then a teacher who taught one year 10 class and one year 11 class would spend 120 hours of the year marking GCSE mocks. That’s three normal working weeks, or nearly 10% of the contracted 1,265 annual hours of directed time.

In our first No More Marking Progress to GCSE English training days last week, we looked at how schools could use comparative judgement to reduce the amount of time it took to mark an English mock paper. The exact amount of time it takes to judge a set of scripts using comparative judgement will depend on the ratio of English teachers to pupils in your school. But we think that at worst, using comparative judgement will halve the amount of time it takes to grade a set of GCSE English papers; that is, it will take 5 hours instead of 10. The best case scenario is that we can get it down to 2 hours. That includes built-in moderation, as well as time to discuss the results with your department and prepare whole-class formative feedback. You can read more about the pilot, and how to sign up for it, here.

Of course, workload is not the only issue we should consider when looking at planning assessment calendars and marking policies. At No More Marking, we like to evaluate the effectiveness of an assessment by looking at these three things.

  • Efficiency and impact on workload
  • Reliability – is the assessment consistent?
  • Validity – does the assessment allow us to make helpful inferences about pupils, and does it help pupils and teachers to improve?

In future blog posts, we’ll consider how reliable and valid traditional mock marking is. But for now, it’s clear that on the measure of efficiency, traditional mock marking doesn’t do that well.

Life after Levels: Five years on

Exactly five years ago, the government announced that national curriculum levels would be removed – and not replaced.

Here’s a quick guide to some of my life after levels blog posts from the last five years.

It was definitely a good thing to abolish levels. As I argued here, here and here, they didn’t give us a shared language. Instead, they provided us with the illusion of a common language, which is actually very misleading. This is because they were based on prose performance descriptors, which can be interpreted in many different ways. Unfortunately, many replacements for NC levels were based around the same flawed prose descriptor model.

If prose descriptors don’t work, what does? One good idea is to define your standards really clearly as questions. EG, instead of saying ‘Pupils can compare fractions to see which is larger’, actually ask them ‘what’s bigger: 4/7 or 6/7? 2/3 or 3/4? 5/7 or 5/9?’ And don’t expect that if they get one of those questions right that they will get them all right!

This works well for maths, but what about things like essays? How do you mark those without a descriptor or a rubric? Another great idea is to use comparative judgement. I first wrote about this back in November 2015. It is basically the most exciting thing to happen to assessment ever. I am so excited about it that I am going to work for No More Marking who provide an online comparative judgement engine. If you haven’t read about it already, do! You can also watch this video of me talking about one of our pilot projects at Research Ed in 2016.

The two books I’ve found most helpful in thinking about assessment are Measuring Up by Daniel Koretz, and Principled Assessment Design by Dylan Wiliam. My review of William’s book is here. My review of Koretz’s book is in three parts: Part one is How useful are tests?, part two is Validity and reliability, and part three is Why teaching to the test is so bad.

In February 2017, Oxford University Press published my own book on assessment, Making Good Progress?: The Future of Assessment for Learning. You can read more about it here. At the Wellington Festival of Education in 2016, I gave a talk which summarised the book’s thesis – you can see the video of this here.

I think the abolition of levels has given teachers the chance to take control of assessment, and has sparked debate, discussion and innovation around assessment which has been hugely valuable. Of course, things still aren’t perfect. National primary assessment has had a number of setbacks, and there are still lots of examples of ‘new’ assessment systems which are essentially rehashed levels. But overall I am really excited, both about the work that has happened in the last five years, and the potential for even further improvements in the next few years.



Five ways you can make the primary writing moderation process less stressful

The primary interim frameworks are now in their second year, and their inconsistencies have been well-documented. Education Datalab have shown that last year there were inconsistencies between local authorities, while more recently the TES published an article revealing that many writing moderators were unable to correctly assess specimen portfolios. Here are five ways to help deal with the uncertainty.

1. Look outside your school or network
Teachers are great judges of their pupils’ work, but find it much harder to place those judgements on a national scale. So wherever possible, try to get exposure to work outside your school to get a clearer idea of where the national standard is.

2. Use what we know about results last year
The interim frameworks were used for the first time last year and, as noted, there are plenty of inconsistencies in how they were applied. However, we do now know that last year, nationally, 74% of pupils were awarded EXS+, and 15% GDS. This compares to 66% and 19% respectively in reading.

3. Check your greater depth (especially if you’re a school in a disadvantaged area)
There is particular evidence that greater depth is being applied inconsistently, and that schools with below average attainment overall are reluctant to award greater depth.

4. Remember that all achievement is on a continuum
Like all grades, ‘greater depth’ and ‘expected standard’ are just arbitrary lines. A pupil who just scrapes ‘expected standard’ actually has more in common with a pupil at the top of ‘working towards’ than they do with a pupil at the top of ‘expected standard’. Not everyone in the same grade will have exactly the same profile, and sometimes the differences between pupils getting the same grade will be greater than pupils with different grades.

5. Use the Sharing Standards results
In March, 199 schools and 8512 pupils took part in Sharing Standards: a trial using comparative judgement to assess Year 6 writing. The results are available here, together with exemplar portfolios. The results offer all four of the benefits above: they involve teacher judgement from across the country; they use information from last year’s results to set this year’s standard; this means they avoid the problem of school-level bias; and they allow you to see the distribution of scripts, not just the grade.

Some people have expressed surprise at the quality of the work at the greater depth threshold. But as we’ve seen, there is no national agreement about what greater depth is.  It is true that the comparative judgement process does not use the interim frameworks, but it does have the same intention: to support professionals in assessing writing quality. In our follow-up survey with schools, 98% of the respondents said they are planning to use their results in their moderation process as they felt the results supported their internal assessment of writing standards. The Sharing Standards results are the only nationally standardised scale of Key Stage 2 writing, so it can’t hurt to take a look and see how thousands of pupils nationally are doing.