Ed Select Committee report – improvements to come?

The Education Select Committee has published its report into the impact of the changes to primary assessment. It’s been an interesting journey from the point at which I submitted written evidence on primary assessment; I wrote a blog back in October, where I doubted there would be much response, but in fact I was wrong. Not only did they seem to draw widely from practioners, stake-holders and experts to give evidence, the report actually suggests that they might have listened quite well, and more to the point, understood the gist of what we were all trying to say. For anyone who had followed assessment research, most of this is nothing new. Similar things have been said for decades. Nevertheless, it’s gratifying to have some airing of the issues at this level.

Summative and formative assessment

The introduction to the report clarifies that the issues being tackled relate to summative assessment and not the ongoing process of formative assessment carried out be teachers. For me, this is a crucial point, since I have been trying, with some difficulty sometimes, to explain to teachers that the two purposes should not be confused. This is important because the original report on assessment without levels suggested that schools had ‘carte blanche’ to create their own systems. Whilst it also emphasised that purposes needed to be clear, many school systems were either extensions of formative assessment that failed to grasp the implications and the requirements of summative purposes, or they were clumsy attempts to create tracking systems based on data that really had not been derived from reliable assessment!

Implementation and design

The report is critical of the time-scale and the numerous mistakes made in the administration of the assessments. They were particularly critical of the STA, which was seen to be chaotic and insufficiently independent. Furthermore, they criticise Ofqual for lack of quality control, in spite of Ofqual’s own protestations that they had scrutinised the materials. The report recommends an independent panel to review the process in future.

This finding is pretty damning. This is not some tin-pot state setting up its first exams – how is incompetence becoming normal? In a climate of anti-expertise, I suppose it is to be expected. It will be very interesting to see if the recommendations have any effect in this area.

The Reading Test

The report took on board the wide-spread criticism of the 2016 Reading Test. The STA defense was that it had been properly trialled and performed as expected. Nevertheless, the good news (possibly) is that the Department has supposedly “considered how this year’s test experience could be improved for pupils”. 

Well we shall see on Monday! I really hope they manage to produce something that most pupils will at least find vaguely interesting to read. The 2016 paper was certainly the least well-received of all the practice papers we did this year.

Writing and teacher assessment

Teacher assessment of writing emerged as something that divided opinion. On the one hand there were quotes from heads who suggested that ‘teachers should be trusted’ to assess writing. My view is that they miss the point and I was very happy to be quoted alongside Tim Oats, as having deep reservations about teacher assessment. I’ve frequently argued against it for several reasons (even when moderation is involved) and I believe that those who propose it may be confusing the different purposes of assessment, or fail to see how it’s not about ‘trust’ but about fairness to all pupils and an unacceptable burden on teachers.

What is good to see, though, is how the Committee have responded to our suggested alternatives. Many of us referred to ‘Comparative Judgement’ as a possible way forward. The potential of comparative judgement as an assessment method is not new, but is gaining credibility and may offer some solutions – I’m glad to see it given space in the report. Something is certainly needed, as the way we currently assess writing is really not fit for purpose. At the very least, it seems we may return to a ‘best-fit’ model for the time being.

For more on Comparative Judgment, see:

Michael Tidd  The potential of Comparative Judgement in primary

Daisy Christodoulou Comparative judgment: 21st century assessment

No More Marking

David Didau  10 Misconceptions about Comparative Judgement

Support for schools

The report found that the changes were made without proper training or support. I think this is something of an understatement. Systems were changed radically without anything concrete to replace them. Schools were left to devise their own systems and it’s difficult to see how anyone could not have foreseen that this would be inconsistent and often  inappropriate. As I said in the enquiry, there are thousands of primary schools finding thousands of different solutions. How can that be an effective national strategy, particularly as, by their own admission, schools lacked assessment expertise? Apparently some schools adopted commercial packages which were deemed ‘low quality’. This, too, is not a surprise. I know that there are teachers and head-teachers who strongly support the notion of ‘doing their own thing’, but I disagree with this idea and have referred to it in the past as the ‘pot-luck’ approach. There will be ways of doing things that are better than others. What we need to do is to make sure that we are trying to implement the most effective methods and not leaving it to the whim of individuals. Several times, Michael Tidd has repeated that we were offered an ‘item bank’ to help teachers with ongoing assessment. The report reiterates this, but I don’t suggest we hold our collective breath.

High-stakes impact and accountability

I’m sure the members of the Assessment Reform Group, and other researchers of the 20th century, would be gratified to know that this far down the line we’re still needing to point out the counter-productive nature of high-stakes assessment for accountability! Nevertheless, it’s good to see it re-emphasised in no uncertain terms and the report is very clear about the impact on well-being and on the curriculum. I’m not sure that their recommendation that OFSTED broadens its focus (again), particularly including science as a core subject, is going to help. OFSTED has already reported on the parlous state of science in the curriculum, but the subject has continued to lose status since 2009. This is as a direct result of the assessment of the other subjects. What is assessed for accountability has status. What is not, does not. The ASE argues (and I totally understand why) that science was impoverished by the test at the end of the year. Nevertheless, science has been impoverished far more, subsequently, in spite of sporadic ‘success stories’ from some schools. This is a matter of record. (pdf). Teacher assessment of science for any kind of reliable purpose is even more fraught with difficulties than the assessment of writing. The farce, last year, was schools trying to decide if they really were going to give credence to the myth that their pupils had ‘mastered’ all 24 of the objectives or whether they were going to ‘fail’ them. Added to this is the ongoing irony that primary science is still ‘sampled’ using an old-fashioned conventional test. Our inadequacy in assessing science is an area that is generally ignored or, to my great annoyance, completely unappreciated by bright-eyed believers who offer ‘simple’ solutions. I’ve suggested that complex subjects like science can only be adequately assessed using more sophisticated technology, but Edtech has stalled in the UK and so I hold out little hope for developments in primary school!

When I think back to my comments to the enquiry, I wish I could have made myself clearer in some ways. I said that if we want assessment to enhance our pupils’ education then what we currently have is not serving that purpose. At the time, we were told that if we wished to further comment on the problem of accountability, then we could write to the Committee, which I did. The constant argument has always been ‘…but we need teachers to be accountable.’ I argued that they need to be accountable for the right things and that a single yearly sample of small populations in test conditions, did not ensure this. This was repeated by so many of those who wrote evidence for the Committee, that it was obviously hard to ignore. The following extract from their recommendations is probably the key statement from the entire process. If something changes as a result of this, there might be a positive outcome after all.

Many of the negative effects of assessment are in fact caused by the use of results
in the accountability system rather than the assessment system itself. Key Stage 2
results are used to hold schools to account at a system level, to parents, by Ofsted, and results are linked to teachers’ pay and performance. We recognise the importance of holding schools to account but this high-stakes system does not improve teaching and learning at primary school. (my bold)

Timings and Tides: the Chartered College of Teaching inaugural conference – Sheffield

I’ve followed the development of the Chartered College of Teaching with some interest and much scepticism. In this mode, I joined as a founder member and spent not an inconsiderable amount of money and time attending the inaugural conference in Sheffield. I’d have liked to attend the London conference, but they saw fit to hold it during the week when only half of us were actually on half-term and many of us could not attend.

Nevertheless, I went with an open mind. I’m aware that there are great enthusiasts out there who see this as a bright beacon of hope on our general plain of educational misery. I wanted to see if there was any basis for this. The answer is that I’m not sure; I’m still sceptical. This blog is my discussion of the conference itself and the College overall.

Why the profession needs a collective voice

I’m afraid I am unable to comment on the first two items on the agenda, as it was impossible to arrive on time, coming by train but I was in time to catch the talk by Professor Chris Husbands (Vice-Chancellor of Sheffield Hallam University). He drew well on his experience as a teacher and spoke convincingly, I thought, on notions of ownership and what matters. He sought to redirect the idea that ‘teachers matter’ towards ‘teaching matters’. I think he was making the point that we needed to focus less on individuals being the key to a successful education system and more towards a systematic improvement of the process. If so, then I would agree this is probably correct – we need to address education in this country, at a level that is more than just ‘holding teachers to account’. Nevertheless there were dissenting voices in the room, arguing (rightly) that teaching is dependent on the individuals, in terms of defending teachers’ well-being, and because teaching requires complex ‘on-the-hoof’ analysis and seems inextricably tied up with human interactions and relationships.

I question, too, the fundamental assumption of a collective voice. Whilst I hate the ‘pot-luck’ approach to education that the English seem unable (and unwilling) to challenge, I know that there are many voices and I worry that collective may turn out to be dominant. I’m unconvinced that the cult of evidence is going to prevent teachers being censured, yet again, by the opinionated but ill-informed, for doing the ‘wrong thing’. I know this is certainly the fear of the neo-trads, even though the tide seems currently in their favour and in fact it’s a cultural, not a political problem: fads and flavours of the month cut all ways, and always have. Which brings me to the next item.

Why we need evidence

Chaired by Ann Mroz, The panel was: Sinead Gaffney, Lisa Pettifer, Aimee Tinkler, John Tomsett and Professor Samantha Twiselton. John opened by proposing the need to weigh up the forces of authority and evidence, with the suggestion that we should not be afraid to swim against the tide if necessary. Well, at this point, I couldn’t agree more, although I suspected that not all tides were equal in his mind. There was much discussion about the need for an evidence-based approach, but I was prompted to tweet thus:

There was a lot said about the importance of an ‘evidence-based’ profession, the use of evidence and about teachers conducting research in their own environments; this is where I derive my  concern that we’re heading towards something more like a ‘cult of evidence’ than an informed profession that questions assumptions (I’m not alone, I imagine). To those without a scientific or a sceptical background, the use of ‘evidence’ as a holy grail, is as dangerous as it is essential. I felt that Sinead was something of a lone voice calling hard for the critical evaluation of evidence rather than the gullible application of a set of tools condoned by the EEF. Personally, I have found that most people are easily persuaded by rhetoric and quickly descend into ritual. Teachers generally are ignorant. (If you wish to rebut that, consider (my anecdotal evidence) that not a single member of the very large staff at my school had actually heard of the College of Teaching, as I was leaving on Friday).

It is very difficult to impress upon people that research evidence almost never says we should do something one way or the other. On the contrary, its power is in calling into question things for which there is very little or no evidence. Most educational research would be considered worthless by any scientific standards and much of it is contradictory. Almost none of it stands up to replication. See this timely article. Like Sinead, I have looked past the meta-analyses on the EEF toolkit and examined some of the original research. If you do the same, you’ll find much of it evaporates into thin air. Try it for ‘feedback’ and see what happens. Moreover, there should be serious doubts about encouraging widespread experimentation and research conducted by teachers. It’s difficult to obtain rigour in research, even under the best experimental conditions. Biology is notoriously tricky. If you add to that the ethical and social considerations of working with young children and then sharing those unreliable findings, we’re opening a massive can of worms (no biological pun intended). Whilst some on the panel were arguing that we needed to be able to judge whether the evidence was robust, audience members, without irony, were still calling for the application of instinct and John T reminded us that any consideration of evidence at all, was still ‘miles away’ in most institutions.

The other elephant-in-the-room is that there is probably far too great an acceptance of the way in which we measure effects in educational research. I don’t mean a statistical issue, but a logistical and a philosophical one. When we try to determine if a practice has an effect, how do we measure the effect, if the product is learning? It may be easy enough to determine if an intervention on multiplication facts has worked – simply test pupils to see if they know those facts. But what if the outcome is trickier to measure? I entered a discussion recently where it was argued that allowing pupils to  do practical science might not be important because the evidence showed that didactic methods trumped investigation! My question would be, ‘in what way?’ If the measure is filling in the answer boxes on a test paper (and believe me when I say I’m a strong advocate of that in appropriate ways), then perhaps teaching the pupils to do just that will produce a greater effect. Yet practical science is about being able to do practical science! Investigations should enable us to become better at investigations. I’m not alone in arguing for appropriate measures – yet most evidence is based on a very narrow set. It’s difficult to see a move away from this in an educational system that now expects secondary teachers to predict art grades from KS2 aggregated English and Maths scores!

Going beyond your comfort zone

Penny Mallory was extremely engaging and I was extremely discomfited by the implications of her speech. Penny overcame self-doubt and domestic adversity to become a champion rally driver. Her questions to the audience were, ‘Can anyone become “world class?”‘ and ‘What qualities does a “world class” person have?’. I was gratified to hear some of the answers along the lines of, ‘It depends what you mean’ and ‘Good genes’. I know what the motivational intention was: we limit ourselves; we need a growth mindset; we should take risks etc. I’m slightly, but not entirely, on board with the growth mindset philosophy. I believe it is true that we can play what Eric Berne’s patients used to refer to as the game of ‘Wooden Leg’ and I work hard to counter that with my pupils. However, I profoundly dislike the contemporary message of ‘social mobility’ and the new populism which the College also seems to be promoting. Winning depends on there being others who will lose. Climbing the greasy pole will require stepping on the competitors. It’s a toxic message in a ruthless climate which seeks to replace the greater aspiration of social justice. Aiming to be ‘world class’ as an individual is a very selfish pursuit which by necessity will always be limited to a few. Becoming world class as an organisation  (or as a country!) needs a different approach altogether – one that I feel we’ve departed from rapidly since the 1980s.

Why being brave is important

Tim O’Brien (Visiting Fellow in Psychology and Hyman Development -UCL Institute of Education) chaired this panel. It being after lunch, my note-taking had decreased and I had moved myself to the back of the room to exit, if need be, but it was interesting to consider ideas of bravery. Perhaps the College could be a force for good, recognising that the profession is currently driven more by fear than it should be.

I know that many pin their hopes on the College to remedy this. Tim’s an eminent psychologist who comes across as knowing his stuff. I was in one of the focus groups he led in the ‘grounded theory’ research he conducted when the College was deciding its remit, and he spoke of this, thanking those of us who were there. In the midst of all the concurrence on the need for bravery, however, I wished I could have had the opportunity to point out that there’s a reason for the fear in education; being brave comes at a cost. Do those advocating it, understand the risks they are asking teachers to take?

Networking

I networked just enough to find that most of the attendees were enthusiasts and that some of them at least, were waiting specifically for the last part of the day – ‘Improving Wellbeing in the Classroom’ – Professor Tanya Byron. She was a bona-fide TV celebrity, so to speak, and the audience seemed engaged. I left before the end – it was old territory for me.

So was it worth it? Well, I still feel I have done the right thing in joining and in attending. This is a novel development that may bring something good. At the very least, access to research is something that I’ve missed since finishing the Master’s. In the conference itself, I would have benefited enormously from a more structured approach to networking. This was left largely to us to do informally during the breaks. I knew that there were twitter contacts there I would have liked to meet, but it was not easy to discern who they were and my social ineptitude hindered me in approaching people ‘cold’, particularly if they were already talking in apparently established groups.

Ultimately, I’d make a plea to those who are sceptical, members or otherwise. Keep it up. To the enthusiasts, I’d say the same, alongside the request that you allow all manner of criticism. There was much enthusiasm evident among the attendees; this in itself can create a charismatic tide. Those who swim against it are always needed.