Timings and Tides: the Chartered College of Teaching inaugural conference – Sheffield

I’ve followed the development of the Chartered College of Teaching with some interest and much scepticism. In this mode, I joined as a founder member and spent not an inconsiderable amount of money and time attending the inaugural conference in Sheffield. I’d have liked to attend the London conference, but they saw fit to hold it during the week when only half of us were actually on half-term and many of us could not attend.

Nevertheless, I went with an open mind. I’m aware that there are great enthusiasts out there who see this as a bright beacon of hope on our general plain of educational misery. I wanted to see if there was any basis for this. The answer is that I’m not sure; I’m still sceptical. This blog is my discussion of the conference itself and the College overall.

Why the profession needs a collective voice

I’m afraid I am unable to comment on the first two items on the agenda, as it was impossible to arrive on time, coming by train but I was in time to catch the talk by Professor Chris Husbands (Vice-Chancellor of Sheffield Hallam University). He drew well on his experience as a teacher and spoke convincingly, I thought, on notions of ownership and what matters. He sought to redirect the idea that ‘teachers matter’ towards ‘teaching matters’. I think he was making the point that we needed to focus less on individuals being the key to a successful education system and more towards a systematic improvement of the process. If so, then I would agree this is probably correct – we need to address education in this country, at a level that is more than just ‘holding teachers to account’. Nevertheless there were dissenting voices in the room, arguing (rightly) that teaching is dependent on the individuals, in terms of defending teachers’ well-being, and because teaching requires complex ‘on-the-hoof’ analysis and seems inextricably tied up with human interactions and relationships.

I question, too, the fundamental assumption of a collective voice. Whilst I hate the ‘pot-luck’ approach to education that the English seem unable (and unwilling) to challenge, I know that there are many voices and I worry that collective may turn out to be dominant. I’m unconvinced that the cult of evidence is going to prevent teachers being censured, yet again, by the opinionated but ill-informed, for doing the ‘wrong thing’. I know this is certainly the fear of the neo-trads, even though the tide seems currently in their favour and in fact it’s a cultural, not a political problem: fads and flavours of the month cut all ways, and always have. Which brings me to the next item.

Why we need evidence

Chaired by Ann Mroz, The panel was: Sinead Gaffney, Lisa Pettifer, Aimee Tinkler, John Tomsett and Professor Samantha Twiselton. John opened by proposing the need to weigh up the forces of authority and evidence, with the suggestion that we should not be afraid to swim against the tide if necessary. Well, at this point, I couldn’t agree more, although I suspected that not all tides were equal in his mind. There was much discussion about the need for an evidence-based approach, but I was prompted to tweet thus:

There was a lot said about the importance of an ‘evidence-based’ profession, the use of evidence and about teachers conducting research in their own environments; this is where I derive my  concern that we’re heading towards something more like a ‘cult of evidence’ than an informed profession that questions assumptions (I’m not alone, I imagine). To those without a scientific or a sceptical background, the use of ‘evidence’ as a holy grail, is as dangerous as it is essential. I felt that Sinead was something of a lone voice calling hard for the critical evaluation of evidence rather than the gullible application of a set of tools condoned by the EEF. Personally, I have found that most people are easily persuaded by rhetoric and quickly descend into ritual. Teachers generally are ignorant. (If you wish to rebut that, consider (my anecdotal evidence) that not a single member of the very large staff at my school had actually heard of the College of Teaching, as I was leaving on Friday).

It is very difficult to impress upon people that research evidence almost never says we should do something one way or the other. On the contrary, its power is in calling into question things for which there is very little or no evidence. Most educational research would be considered worthless by any scientific standards and much of it is contradictory. Almost none of it stands up to replication. See this timely article. Like Sinead, I have looked past the meta-analyses on the EEF toolkit and examined some of the original research. If you do the same, you’ll find much of it evaporates into thin air. Try it for ‘feedback’ and see what happens. Moreover, there should be serious doubts about encouraging widespread experimentation and research conducted by teachers. It’s difficult to obtain rigour in research, even under the best experimental conditions. Biology is notoriously tricky. If you add to that the ethical and social considerations of working with young children and then sharing those unreliable findings, we’re opening a massive can of worms (no biological pun intended). Whilst some on the panel were arguing that we needed to be able to judge whether the evidence was robust, audience members, without irony, were still calling for the application of instinct and John T reminded us that any consideration of evidence at all, was still ‘miles away’ in most institutions.

The other elephant-in-the-room is that there is probably far too great an acceptance of the way in which we measure effects in educational research. I don’t mean a statistical issue, but a logistical and a philosophical one. When we try to determine if a practice has an effect, how do we measure the effect, if the product is learning? It may be easy enough to determine if an intervention on multiplication facts has worked – simply test pupils to see if they know those facts. But what if the outcome is trickier to measure? I entered a discussion recently where it was argued that allowing pupils to  do practical science might not be important because the evidence showed that didactic methods trumped investigation! My question would be, ‘in what way?’ If the measure is filling in the answer boxes on a test paper (and believe me when I say I’m a strong advocate of that in appropriate ways), then perhaps teaching the pupils to do just that will produce a greater effect. Yet practical science is about being able to do practical science! Investigations should enable us to become better at investigations. I’m not alone in arguing for appropriate measures – yet most evidence is based on a very narrow set. It’s difficult to see a move away from this in an educational system that now expects secondary teachers to predict art grades from KS2 aggregated English and Maths scores!

Going beyond your comfort zone

Penny Mallory was extremely engaging and I was extremely discomfited by the implications of her speech. Penny overcame self-doubt and domestic adversity to become a champion rally driver. Her questions to the audience were, ‘Can anyone become “world class?”‘ and ‘What qualities does a “world class” person have?’. I was gratified to hear some of the answers along the lines of, ‘It depends what you mean’ and ‘Good genes’. I know what the motivational intention was: we limit ourselves; we need a growth mindset; we should take risks etc. I’m slightly, but not entirely, on board with the growth mindset philosophy. I believe it is true that we can play what Eric Berne’s patients used to refer to as the game of ‘Wooden Leg’ and I work hard to counter that with my pupils. However, I profoundly dislike the contemporary message of ‘social mobility’ and the new populism which the College also seems to be promoting. Winning depends on there being others who will lose. Climbing the greasy pole will require stepping on the competitors. It’s a toxic message in a ruthless climate which seeks to replace the greater aspiration of social justice. Aiming to be ‘world class’ as an individual is a very selfish pursuit which by necessity will always be limited to a few. Becoming world class as an organisation  (or as a country!) needs a different approach altogether – one that I feel we’ve departed from rapidly since the 1980s.

Why being brave is important

Tim O’Brien (Visiting Fellow in Psychology and Hyman Development -UCL Institute of Education) chaired this panel. It being after lunch, my note-taking had decreased and I had moved myself to the back of the room to exit, if need be, but it was interesting to consider ideas of bravery. Perhaps the College could be a force for good, recognising that the profession is currently driven more by fear than it should be.

I know that many pin their hopes on the College to remedy this. Tim’s an eminent psychologist who comes across as knowing his stuff. I was in one of the focus groups he led in the ‘grounded theory’ research he conducted when the College was deciding its remit, and he spoke of this, thanking those of us who were there. In the midst of all the concurrence on the need for bravery, however, I wished I could have had the opportunity to point out that there’s a reason for the fear in education; being brave comes at a cost. Do those advocating it, understand the risks they are asking teachers to take?

Networking

I networked just enough to find that most of the attendees were enthusiasts and that some of them at least, were waiting specifically for the last part of the day – ‘Improving Wellbeing in the Classroom’ – Professor Tanya Byron. She was a bona-fide TV celebrity, so to speak, and the audience seemed engaged. I left before the end – it was old territory for me.

So was it worth it? Well, I still feel I have done the right thing in joining and in attending. This is a novel development that may bring something good. At the very least, access to research is something that I’ve missed since finishing the Master’s. In the conference itself, I would have benefited enormously from a more structured approach to networking. This was left largely to us to do informally during the breaks. I knew that there were twitter contacts there I would have liked to meet, but it was not easy to discern who they were and my social ineptitude hindered me in approaching people ‘cold’, particularly if they were already talking in apparently established groups.

Ultimately, I’d make a plea to those who are sceptical, members or otherwise. Keep it up. To the enthusiasts, I’d say the same, alongside the request that you allow all manner of criticism. There was much enthusiasm evident among the attendees; this in itself can create a charismatic tide. Those who swim against it are always needed.

It’s 2017 – What on Earth can we do?

Though I felt I would have preferred to be at home drinking cocoa, I played saxophone for a small, local gig on New Year’s Eve. The revelry seemed suitably subdued as the clock struck midnight and the guitarist wished me a ‘Happy New Year’, saying that there was no way 2017 could possibly be worse than 2016. I sadly disagreed and prophesied that we would look back on 2016 as the last year of Recognisable Things before we really began to notice that nothing was ever the same again.

Anyone who has read my blogs before will see that they tend not to be very upbeat, generally. Nobody would describe me as a ‘bubbly’ personality and  I’m generally inspired to write when I have something to critique. As much as I admire spirit-lifting attempts, I perceive them as fundamentally flawed and self-centred in the sense that they seem to ignore reality.

So how do I manage to work with primary school pupils? Basically, I lie by omission. I can not possibly tell them what I believe their future holds and were I to openly discuss with them what’s going on in the world, I would risk censure for ‘extremist views’.

It was in a staff room, over 20 years ago, that I said that I was pretty sure that climate change would be the biggest challenge we would face in the new millennium. The reaction then was along the lines of, ‘Oh, really? Is that because of CFCs and things? I don’t really know much about it. We can’t be doom-mongers. Well I’m not really into the environment and all that – it’s more your sort of thing.’ Over the decades, everything that I said was likely to happen, has happened and sadly, the reaction I get now is pretty much the same, in spite of the global scientific consensus and the general acceptance that it is no longer a conspiracy.

On a day-to-day basis, I engage with the business of ‘business as usual’, and in 2016 I made some efforts to push against what I felt to be detrimental to the education of our pupils. I actively responded to every government consultation and was gratified to give evidence on primary assessment to the Education Select Committee. I try to promote an agenda that a quality education is a global citizen entitlement and is not about toxic notions of ‘attainment’ and ‘social mobility’. I agree that curriculum subjects should be rigorously taught by teachers with excellent subject knowledge and I welcome the increase in attention to evidence over mythology. Perverse incentives aside, I do continue to try to do my best to develop pupils’ knowledge and understanding in the ‘core’ and ‘foundation’ subjects of our National Curriculum, as though the future will resemble the past. Deep down, I have misgivings; I probably should spend more time teaching them basic survival skills. From how things are currently panning out, the next few decades will be an escalation of the challenges we have faced this year:

It was wrong to make heroes of those who have climbed the greasy pole over their fellows, those who have risen to the top of their chosen career and gained huge amounts of wealth, and those who have dominated nations through displays of power and authority, because their big noise drowned out the voices of reason to which we should have listened and now we have to face the consequences as well as we can. Knowing that we have tipped the climate balance, it’s very difficult to see how things could improve or even stay the same but there is something to do.

If things are going to get a lot trickier, then I see that there is a need to remember that many of us are not psychopaths. We know about co-operation, consideration and compassion and we should exercise these. If we possess the trait of empathy, we know about the suffering of others and we have to be kinder – to humans and non-human animals. If we know the difference, we need to be emphatically better to each other, because there are those who will be emphatically worse. How we treat each other should be a matter of concern – in school, the supermarket, on the road, and in our (t)wittering online which appears often to deteriorate into childish insults and point-scoring. If we have the wit, let us use it to exercise consideration and circumspection in 2017.

Primary assessment is more than a fiasco – it’s completely wrong

I’ve written my submission to the Education Committee’s inquiry on primary assessment for what it’s worth. I can’t imagine that they’re interested in what we have to say, given that this government have ignored just about all the expert advice they’ve ever received or requested on nearly everything else. This country has ‘had enough of experts’ after all.

I won’t paste my submission here – there are various restrictions on publishing them elsewhere, it seems. However it’s a good time to get some thoughts off my chest. Primary assessment (and school-based assessment generally) has all gone a bit wrong. OK, a lot wrong. It’s so wrong that it’s actually very damaging. Conspiracy theorists might have good cause to think it is deliberate; my own cynicism is that it is underpinned by a string of incompetencies and a distinct failure to listen at all to any advice.

In thinking about why it has all gone wrong, I want to pose a possibly contentious question: is the attainment we are attempting to measure, a thing that should dominate all educational efforts and discourse? I’ve written before about my growing doubts about the over-emphasis on attainment and how I think it detracts from the deeper issue of education. The further we get down this line, particularly with the current nonsense about bringing back selective education, the more this crystalises for me. Just to be clear, this is not an anti-intellectual stance, nor a woolly, liberal dumbing-down view. I fully embrace the idea that we should not put a ceiling on all kinds of achievement for everybody. Having a goal and working towards it – having a way of demonstrating what you have achieved – that’s an admirable thing. What I find ridiculous is that the kind of attainment that is obsessing the nation, doesn’t actually mean very much and yet somehow we are all party to serving its ends. Put it this way – tiny fluctuations in scores in a set of very narrow domains make headlines for pupils, teachers, schools, counties etc. Every year we sweat over the %. If there’s a rise above the ‘expectation’ we breathe a sigh of relief. If, heaven forbid, we had a difficult cohort and a couple of boxes are in the ‘blue zone’ we dread the repercussions because now we’re no longer an outstanding school. But, as Jack Marwood writes here, there’s no pattern. We’ve even begun to worry about whether we’re going to be labelled a ‘coasting school’! Good should be good enough because the hysteria over these measures is sucking the life out of the most important resource – us. Of course the inspectorate needs to be on the lookout for actually bad schools. Are these really going to be so difficult to spot? Is it really the school that was well above average in 2014 and 15 but dipped in 16? Is the child who scores 99 on the scaled score so much more of a failure than the one who scored 101? Is our group of 4 pupil premium children getting well above average, in a small set of tests, an endorsement of our good teaching compared to another school’s 4 getting well below?

Attainment has become an arms race and teachers, pupils and parents are caught in the crossfire. In spite of the ‘assessment without levels’ rhetoric, all our accountability processes are driven by a focus on an attainment with one level. This is incredibly destructive in my experience. Notwithstanding those self-proclaimed paragons of good practice who claim that they’ve got the balance right etc., what I’ve mainly seen in schools are teachers at the end of their wits, wondering what on earth they can further do (what miracle of intervention they can concoct) to ‘boost’ a group of ‘under-performing’ children to get to ‘meeting’, whilst maintaining any kind of integrity with regard to the children who have never been anywhere near. I was recently told in a leadership meeting that all children should make the same amount of progress. Those ‘middle achievers’ should be able to progress at the same rate as the ‘high achievers’. It’s the opposite which is true. The high achievers are where they are exactly because they made quicker progress – but the ‘middle achievers’ (and any other category – good grief!) will also get to achieve, given time. And while all this talk of progress is on the table – let’s be honest – we’re talking about ‘attainment’ again: a measure taken from their KS2 assessments, aggregated, and compared to KS1 in a mystical algorithm.

It’s not like the issues surrounding assessment have never been considered. Just about all the pitfalls of the recent primary debacle have been written about endlessly, and frequently predicted. High-stakes testing has always been the villain of the piece: perverse incentives to teach to the test, narrowing of the curriculum, invalidity of testing domain, unreliability/bias/downright cheating etc. The problem is the issues won’t go away, because testing is the wrong villain. Testing is only the blunt tool to fashion the club of attainment with which to beat us (apologies for extended metaphor). I’m a big fan of testing. I read Roediger and Karpicke’s (pdf) research on ‘testing effect’ in the early days, long before it became a fashionable catch-phrase. I think we should test as many things in as many ways as we can: to enhance recall; to indicate understanding; to identify weaknesses; to demonstrate capacity; to achieve certification etc. I was all in favour of Nicky Morgan’s proposal to introduce an online tables test. What a great idea! Only – make it available all the time and don’t use the results against the pupil or the teacher. No – testing doesn’t cause the problem. It’s caused by the narrow, selective nature, the timing and the pressure of attaining an arbitrary ‘meeting expectations’ (one big level, post levels). The backwash on the curriculum is immense. Nothing has any status anymore: not art, not music, not D&T, not history nor geography, and certainly not science – that ‘core subject’ of yore! Some might argue that it’s because they’re not tested, and of course, I agree up to a point, but the real issue is that they’re not seen as being important in terms of attainment.

I shall add a comment here on teacher assessment, just because it continues to drag on in primary assessment like some old ghost that refuses to stop rattling its chains. If teacher assessment is finally exorcised, I will be particularly grateful. It is an iniquitous, corrupted sop to those who believe ‘teachers are best placed to make judgements about their own pupils’. Of course they are – in the day to day running of their class and in the teaching of lessons – but teacher assessment should not be used in any way to measure attainment. I am not arguing that teachers are biased, that they make mistakes or inflate or deflate their assessments. I am arguing that there is simply no common yardstick and so these cannot be considered reliable. The ‘moderated’ writing debacle of 2016 should have put that fact squarely on the table for all doubters to see. Primary assessments are used in accountability. How can we expect teachers to make judgements that could be used against them in appraisal and in pay reviews?

I’m an idealist in education. I think that it has a purpose beyond the establishment of social groups for different purposes (leadership, administrative work, manual labour). I don’t think that it is best served by a focus on a narrow set of objectives and an over-zealous accountability practice based on dubious variations in attainment. I tried to sum up my proposals for the Education Committe, and I will try to sum up my summing up:

  • Stop using small variations in flawed attainment measures for accountability
  • Give us fine-grained, useful but low-stakes testing, for all (use technology)
  • If we have to measure, get rid of teacher assessment and give us lots of common, standardised tools throughout the primary phase
  • Give us all the same technology for tracking the above (how many thousands of teacher hours have been spent on this?)
  • If you have to have end of stage tests, listen to the advice of the experts and employ some experts in test design – the 2016 tests were simply awful
  • Include science
  • Be unequivocal in the purposes of assessment and let everybody know

I didn’t say ‘get rid of the end of key stage assessments altogether and let us focus again on educating our pupils’. Maybe I should have.

 

 

 

Of Wasps and Education

A long time ago I lived with Jim, a zoologist – the sort that actually liked to know about animals. He taught me, contrary to all popular English culture, to be friendly to wasps – to sit still and observe rather than flap about, leap up, scream, etc. Actually, I was an easy pupil because I’d not had that particular education and was stunned and appalled when, as a 15-year-old, newly-arrived and attending my first English school, I witnessed a fellow pupil smash a stray wasp to death rather than simply let it out of the window as we would have done ‘back home’. Anyway, Jim used to let the wasps land on his finger and drink lemonade – a trick I subsequently performed (without being stung) in front of many a bemused audience. Since then, I’ve learned lots about these clever insects. They can recognise each other as individuals and they can recognise human faces. Allow a wasp to do its zig-zagging buzz in front of you it will learn what you look like and generally fly off, leaving you alone.

This year, I’m one of the very few to be concerned that there are practically no wasps about. Nor many other insects. I take their absence as a bad sign, where I suspect most people are just happy not to be ‘pestered’ by them. I did see one last night though, whilst waiting with my fellow band members before a gig. It took me a while to realise why they were jumping up and flapping their hands – it was a lone, wasp interested in the meat in their pork baps, so I did the trick; the wasp landed on my fingers and took the small piece I offered. I didn’t get stung, it didn’t get tangled in my hair, it didn’t land on my face or do any of the the other things that terrify people. It didn’t bother me at all and I could continue to sit on my hay bale and calmly contemplate the beautiful evening.

So how does this relate to anything? Well it’s something like this: what I have learned about wasps trumps popular culture and folk knowledge, and allows me to make both a compassionate and a superior decision. This is what I consider to be the goal of education. Yet, it’s a losing battle – education is pointless in the face of both a widespread, ignorant culture and a ruling minority that makes decisions for us, based not on evidence and expertise (badger cull, abolition of dept of energy and climate change), but for some other agenda, unnoticed by the majority and unfathomable to the rest.

 

Got the T-shirt (a moderate tale)

Given that teacher assessment is a nonsense which lacks reliability, and that moderation can not really reduce this, nor ensure that gradings are comparable, our moderation experience was about as good as it could be! It was thus:

Each of we two Y6 teachers submitted all our assessments and three children in each category (more ridiculous, inconsistent and confusable codes, here), of which one each was selected, plus another two from each category at random. So, nine children from each class. We were told who these nine were a day in advance. Had we wanted to titivate, we could have, but with our ‘system’ it really wasn’t necessary.

The ‘system’ was basically making use of the interim statements and assigning each one of them a number. Marking since April has involved annotating each piece of work with these numbers, to indicate each criterion. It was far less onerous than it sounds and was surprisingly effective in terms of formative assessment. I shall probably use something similar in the future, even if not required to present evidence.

The moderator arrived this morning and gave us time to settle our classes whilst she generally perused our books. I had been skeptical. I posted on twitter that though a moderator would have authority, I doubted they’d have more expertise. I was concerned about arguing points of grammar and assessment. I was wrong. We could hardly have asked for a better moderator. She knew her stuff. She was a y6 teacher. We had a common understanding of the grammar and the statements. She’d made it her business to sample moderation events as widely as possible and therefore had had the opportunity to see many examples of written work from a wide range of schools. She appreciated our system and the fact that all our written work from April had been done in one book.

Discussions and examination of the evidence, by and large led to an agreed assessment. One was raised from working towards; one, who I had tentatively put forward as ‘greater depth’, but only recently, was agreed to have not quite made it. The other 16 went through as previously assessed, along with all the others in the year group. Overall my colleague and I were deemed to know what we were doing! We ought to, but a) the county moderation experience unsettled us and fed my ever-ready cynicism about the whole business and b) I know that it’s easy to be lulled into a false belief that what we’ve agreed is actually the ‘truth’ about where these pupils are at. All we can say is that we roughly agreed between the three of us. The limited nature of the current criteria makes this an easier task than the old levels, (we still referred to the old levels!) but the error in the system makes it unusable for accountability or for future tracking. I’m most interested to see what the results of the writing assessment are this year – particularly in moderated v non-moderated schools. Whatever it is, it won’t be a reliable assessment but, unfortunately it will still be used (for good or ill) by senior leaders, and other agencies, to make judgements about teaching.

Nevertheless, I’m quite relieved the experience was a positive one and gratified and somewhat surprised to have spent the day with someone with sense and expertise. How was it for you?

 

 

 

 

Trialling moderation

A quick one today to cover the ‘trialling moderation’ session this afternoon.

We had to bring all the documents and some samples of pupils’ writing, as expected.

Moderators introduced themselves. They seemed to be mainly Y6 teachers who also were subject leaders for English. Some had moderated before, but obviously not for the new standards.

The ‘feel’ from the introduction to the session was that it wasn’t as big a problem as we had all been making it out to be. We were definitely using the interim statements and that ‘meeting’ was indeed equivalent to a 4b.

At my table, we expressed our distrust of this idea and our fear that very few of our pupils would meet expected standards. Work from the first pupil was shared and the criteria ticked off. We looked at about 3 pieces of work. It came out as ‘meeting’ even though I felt it was comparable to the exemplar, ‘Alex’. The second pupil from the next school was ‘nearly exceeding’. I wasn’t convinced. There were lots of extended pieces in beautiful handwriting but sentence structures were rather unsophisticated. There was arguably a lack of variety in the range and position of clauses and transitional phrases. There was no evidence of writing for any other  curriculum area, such as science.

I put forward the work from a pupil I had previously thought  to be ‘meeting’ but had then begun to doubt. I wanted clarification. Formerly, I would have put this pupil at a 4a/5c with the need to improve consistency of punctuation. Our books were the only ones on our table (and others) that had evidence of writing across the curriculum; we moved a few years ago to putting all work in a ‘theme book’ (it has its pros and cons!).

Unfortunately the session was ultimately pretty frustrating as we didn’t get to agree on the attainment of my pupil; I was told that there needed to be evidence of the teaching process that had underpinned the writing that was evident in the books. That is to say, there should be the grammar exercises where we had taught such things as ‘fronted adverbials’ etc. and then the written pieces in which that learning was then evidenced. I challenged that and asked why we couldn’t just look at the writing as we had done for the first pupil. By then the session was pretty much over. In spite of the moderator’s attempt to finish the moderation for me, we didn’t. The last part of the session was given over to the session leader coming over and asking if we felt OK about everything, and my reply that no, I didn’t. I still didn’t know which of the multiplicity of messages to listen to and I hadn’t had my pupil’s work moderated. I had seen other pieces of work, but I didn’t trust the judgements that had been made.

The response was ‘what mixed messages?’ and the suggestion that it may take time for me to ‘get my head around it’ just like I must have had to do for the previous system. She seemed quite happy that the interim statements were broadly equivalent to a 4b and suggested that the government certainly wouldn’t want to see the data showing a drop in attainment. I suggested that if people were honest, that could be the only outcome.

My colleague didn’t fare much better. She deliberately brought samples from a pupil who fails to write much but when he does, it is accurate, stylish and mature. He had a range of pieces, but most of them were short. The moderator dismissed his work as insufficient evidence but did inform my colleague that she would expect to see the whole range of text types, including poetry because otherwise how would we show ‘figurative language and metaphor’?

I’m none the wiser but slightly more demoralised than before. One of my favourite writers from last year has almost given up writing altogether because he knows his dyslexia will prevent him from ‘meeting’. Judging the writing of pupils as effectively a pass or fail is heart-breaking. I know how much effort goes into their writing. I can see writers who have such a strong grasp of audience and style, missing the mark by just a few of the criteria. This is like being faced with a wall – if you cant get over it, stop bothering.

We are likely to be doing a lot of writing over the next few weeks.

 

Final report of the Commission on Assessment without Levels – a few things.

I’ve read the report and picked out some things. This is not a detailed analysis, but more of a selection of pieces relevant to me and anyone else interested in primary education and assessment:

Our consultations and discussions highlighted the extent to which teachers are subject to conflicting pressures: trying to make appropriate use of assessment as part of the day-today task of classroom teaching, while at the same time collecting assessment data which will be used in very high stakes evaluation of individual and institutional performance. These conflicted purposes too often affect adversely the fundamental aims of the curriculum,

Many of us have been arguing that for years.

the system has been so conditioned by levels that there is considerable challenge in moving away from them. We have been concerned by evidence that some schools are trying to recreate levels based on the new national curriculum.

Some schools are hanging on to them like tin cans in the apocalypse.

levels also came to be used for in-school assessment between key stages in order to monitor whether pupils were on track to achieve expected levels at the end of key stages. This distorted the purpose of in-school assessment,

Whose fault was that?

There are three main forms of assessment: in-school formative assessment, which is used by teachers to evaluate pupils’ knowledge and understanding on a day-today basis and to tailor teaching accordingly; in-school summative assessment, which enables schools to evaluate how much a pupil has learned at the end of a teaching period; and nationally standardised summative assessment,

Try explaining that to those who believe teacher assessment through the year can be used for summative purposes at the end of the year.

many teachers found data entry and data management in their school burdensome.

I love it, when it’s my own.

There is no intrinsic value in recording formative assessment;

More than that – it degrades the formative assessment itself.

the Commission recommends schools ask themselves what uses the assessments are intended to support, what the quality of the assessment information will be,

I don’t believe our trial system using FOCUS materials and assigning a score had much quality. It was too narrow and unreliable. We basically had to resort to levels to try to achieve some sort of reliability.

Schools should not seek to devise a system that they think inspectors will want to see;

!

Data should be provided to inspectors in the format that the school would ordinarily use to monitor the progress of its pupils

‘Ordinarily’ we used levels! This is why I think we need data based on internal summative assessments. I do not think we can just base it on a summative use of formative assessment information!

The Carter Review of Initial Teacher Training (ITT) identified assessment as the area of greatest weakness in current training programmes.

We should not expect staff (e.g. subject leaders) to devise assessment systems, without having had training in assessment.

The Commission recommends the establishment of a national item bank of assessment questions to be used both for formative assessment in the classroom, to help teachers evaluate understanding of a topic or concept, and for summative assessment, by enabling teachers to create bespoke tests for assessment at the end of a topic or teaching period.

But don’t hold your breath.

The Commission decided at the outset not to prescribe any particular model for in-school assessment. In the context of curriculum freedoms and increasing autonomy for schools, it would make no sense to prescribe any one model for assessment.

Which is where it ultimately is mistaken, since we are expected to be able to make comparisons across schools!

Schools should be free to develop an approach to assessment which aligns with their curriculum and works for their pupils and staff

We have a NATIONAL CURRICULUM!

Although levels were intended to define common standards of attainment, the level descriptors were open to interpretation. Different teachers could make different judgements

Well good grief! This is true of everything they’re expecting us to do in teacher assessment all the time.

Pupils compared themselves to others and often labelled themselves according to the level they were at. This encouraged pupils to adopt a mind-set of fixed ability, which was particularly damaging where pupils saw themselves at a lower level.

This is only going to be made worse, however, by the ‘meeting’ aspects of the new system.

Without levels, schools can use their own assessment systems to support more informative and productive conversations with pupils and parents. They can ensure their approaches to assessment enable pupils to take more responsibility for their achievements by encouraging pupils to reflect on their own progress, understand what their strengths are and identify what they need to do to improve.

Actually, that’s exactly what levels did do! However…

The Commission hopes that teachers will now build their confidence in using a range of formative assessment techniques as an integral part of their teaching, without the burden of unnecessary recording and tracking.

They hope?

Whilst summative tasks can be used for formative purposes, tasks that are designed to provide summative data will often not provide the best formative information. Formative assessment does not have to be carried out with the same test used for summative assessment, and can consist of many different and varied tasks and approaches. Similarly, formative assessments do not have to be measured using the same scale that is used for summative assessments.

OK – this is a key piece of information that is misunderstood by nearly everybody working within education.

However, the Commission strongly believes that a much greater focus on high quality formative assessment as an integral part of teaching and learning will have multiple benefits:

We need to make sure this is fully understood. We must avoid formalising what we think is ‘high quality formative assessment’ because that will become another burdensome and meaningless ritual. Don’t get me started on the Black Box!

The new national curriculum is founded on the principle that teachers should ensure pupils have a secure understanding of key ideas and concepts before moving onto the next phase of learning.

And they do mean 100% of the objectives.

The word mastery is increasingly appearing in assessment systems and in discussions about assessment. Unfortunately, it is used in a number of different ways and there is a risk of confusion if it is not clear which meaning is intended

By  leading politicians too. A common understanding of terms is rather important, don’t you think?

However, Ofsted does not expect to see any specific frequency, type or volume of marking and feedback;

OK, it’s been posted before, but it’s worth reiterating. Many SL and HTs are still fixated on marking.

On the other hand, standardised tests (such as those that produce a reading age) can offer very reliable and accurate information, whereas summative teacher assessment can be subject to bias.

Oh really? Then why haven’t we been given standardised tests and why is there still so much emphasis on TA?

Some types of assessment are capable of being used for more than one purpose. However, this may distort the results, such as where an assessment is used to monitor pupil performance, but is also used as evidence for staff performance management. School leaders should be careful to ensure that the primary purpose of assessment is not distorted by using it for multiple purposes.

I made this point years ago.

Primary Science Assessment – no miracles here

In April I wrote here on the draft science assessment guidance from the TAPS group. The final version is now out in the public domain (pdf), described thus:

“The Teacher Assessment in Primary Science (TAPS) project is a 3 year project based at Bath Spa University and funded by the Primary Science Teaching Trust (PSTT), which aims to develop support for a valid, reliable and manageable system of science assessment which will have a positive impact on children’s learning.”

I was vainly hoping for a miracle: valid, reliable AND manageable! Could they pull off the impossible? Well if you read my original post, you’d know that I had already abandoned that fantasy. I’m sorry to be so disappointed – I had wished to be supportive, knowing the time, effort (money!) and best of intentions put into the project. Others may feel free to pull out the positive aspects but here I am only going to point out some of the reasons why I feel so let down.

Manageable?

At first glance we could could probably dismiss the guidance on the last of the three criteria straight away. 5 layers and 22 steps would simply not look manageable to most primary school teachers. As subject leader, I’m particularly focussed on teaching science and yet I would take one look at that pyramid and put it away for another day. Science has such low priority, regardless of the best efforts of primary science enthusiasts like myself, that any system which takes more time and effort than that given to the megaliths of English and Maths, is highly unlikely to be embraced by class teachers. If we make assessment more complicated, why should we expect anything else? Did the team actually consider the time it would take to carry out all of the assessment steps for every science objective in the New Curriculum? We do need to teach the subject, after all, even if we pretend that we can assess at every juncture.

Reliable?

In my previous post on this subject, I did include a question about the particular assessment philosophy of making formative assessment serve summative aims. I question it because I assert that it can not. It is strongly contested in the research literature and counter-indicated in my own experience. More importantly, if we do use AfL (assessment for learning/formative assessment) practices for summative data then in no way can we expect it to be reliable! Even the pupils recognise that it is unfair to make judgements about their science based on their ongoing work. Furthermore, if it is teacher assessment for high stakes or data driven purposes then it can not be considered reliable, even if the original purpose is summative. At the very least, the authors of this model should not be ignoring the research.

Valid?

Simply put, this means ‘does what it says on the tin’ – hence the impossibility of assessing science adequately. I’m frequently irritated by the suggestion that we can ‘just do this’ in science. Even at primary school (or perhaps more so) it’s a massive and complex domain. We purport to ‘assess pupils’ knowledge, skills and understanding’ but these are not simply achieved. At best we can touch on knowledge, where at least we can apply a common yardstick through testing. Skills may be observed, but there are so many variables in performance assessment that we immediately lose a good deal of reliability. Understanding can only be inferred through a combination of lengthy procedures. Technology would be able to address many of the problems of assessing science, but as I’ve complained before, England seems singularly disinterested in moving forward with this.

Still, you’d expect examples to at least demonstrate what they mean teachers to understand by the term ‘valid’. Unfortunately they include some which blatantly don’t. Of course it’s always easy to nit-pick details, but an example, from the guidance, of exactly not assessing what you think you are assessing is, ‘I can prove air exists’ (now there’s a fine can of worms!) which should result from an assessment on being able to prove something about air, not the actual assessment criterion ‘to know air exists’ (really? In Year 5?).

1. Ongoing formative assessment

This is all about pupil and peer assessment and also full of some discomforting old ideas and lingering catch phrases. I admit, I’ve never been keen on WALTs or WILFs and their ilk. I prefer to be explicit with my expectations and for the pupils to develop a genuine understanding of what they are doing rather than cultivate ritualised, knee-jerk operations. Whilst I concede that this model focusses on assessment, it’s not very evident where the actual teaching takes place. Maybe it is intended to be implied that it has already happened, but my concern is that this would not be obvious to many teachers. The guidance suggests, instead, that teachers ‘provide opportunities’, involve pupils in discussions’, ‘study products’, ‘adapt their pace’ and ‘give feedback’. I would have liked to see something along the lines of ‘pick up on misconceptions and gaps in knowledge and then teach.’

Most disheartening, is to see the persistence of ideas and rituals to do with peer assessment. Whilst peer assessment has come under some scrutiny recently for possibly not being as useful as it has been claimed, I think it does have a place, but only with some provisos. In my experience, the most useful feedback comes not when we insist that it’s reduced to a basic format (tick a box, etc.) but when pupils can genuinely offer a thoughtful contribution. As such, it has to be monitored for misinformation; the pupils have to be trained to understand that their peers might be wrong and this takes time. After fighting hard against mindless practices such as ‘two stars and a wish’, my heart sinks to find it yet again enshrined in something that is intended for primary teachers across the country.

2. Monitoring pupil progress

In this layer, we move from the daily activities which are considered part of ongoing, formative assessment, to the expectation that teachers are now to use something to monitor ‘progress’. This involves considerable sleight of hand and I would have to caution teachers and leadership to assume that they can just do the things in the boxes. Let’s see:

TEACHERS BASE THEIR SUMMATIVE JUDGEMENTS OF PUPILS’ LEARNING ON A RANGE OF TYPES OF ACTIVITY

When? To get a good range, it would have to start early in the year, particularly if it includes all the science coverage from the curriculum. In that case, summative judgements are not reliable, because the pupils should have progressed by the end of the year. If it takes place at the end of the year, do we include the work from the earlier part of the year? Do we ignore the areas covered up to February? If we don’t, do we have time to look at a range of types of activity in relation to everything they should have learned? Neither ongoing work, nor teacher observation, are reliable or fair if we need this to be used for actual comparative data.

TEACHERS TAKE PART IN MODERATION/DISCUSSION WITH EACH OTHER OF PUPILS’ WORK IN ORDER TO ALIGN JUDGEMENTS

Oh how I despise the panacea of moderation! This is supposed to reduce threats to reliability and I’m constantly calling it out in that regard. Here they state:

“Staff confidence in levelling is supported by regular moderation. The subject leader set up a series of 10 minute
science moderation slots which take place within staff meetings across the year. Each slot consists of one class
teacher bringing along some samples of work, which could be children’s writing, drawings or speech, and the staff agreeing a level for each piece. This led to lengthy discussions at first, but the process became quicker as staff developed knowledge of what to look for.”

Where to begin? Staff confidence does not mean increased reliability. All it does is reinforce group beliefs. 10 minute slots within staff meetings are unrealistic expectations, both in perceiving how long moderation takes and in the expectation that science will be given any slots at all. Whatever staff ‘agree’, it can not be considered reliable: a few samples of work are insufficient to agree anything; the staff may not have either science or assessment expertise to be qualified to make the judgement; more overtly confident members of staff may influence others and there may be collective misunderstanding of the criteria or attainment; carrying out a 10 minute moderation for one pupil in one aspect of science does not translate to all the other pupils in all the aspects of science we are expected to assess. It might also have been a good idea to vet this document for mention of levels, given that it was brought out to address their removal.

3.Summative reporting

A MANAGEABLE SYSTEM FOR RECORD-KEEPING IS IN OPERATION TO TRACK AND REPORT ON PUPILS’ LEARNING IN SCIENCE

I just want to laugh at this. I have some systems for record-keeping which in themselves are quite manageable, once we have some real data. Where we have testable information, for example, factual knowledge, they might also mean something, but as most of us will know, they quickly become a token gesture simply because they are not manageable. Very quickly, records become ‘rule of thumb’ exercises, simply because teachers do not have the time to gather sufficient evidence to back up every statement. I note that one of the examples in the guide is the use of the old APP rubric which is no longer relevant to the new curriculum. We made the best of this in our school in a way that I devised to try to be as sure of the level as was possible, but even then, we knew that our observations were best guesses. The recording system is only as good as the information which is entered, despite a widespread misconception that records and assessment are the same thing! I’m no longer surprised, although still dismayed, at the number of people that believe the statistics generated by the system.

I didn’t intend this to be a balanced analysis – I’d welcome other perspectives – and I apologise to all involved for my negativity, but we’re clearly still a long way from a satisfactory system of assessing primary science. The model can not work unless we don’t care about reliability, validity or manageability. But in that case, we need no model. If we want a fair assessment of primary science, with data on pupils’ attainment and progress that we feel is dependable, then we need something else. In my view, it only begins to be attainable if we make creative use of technology. Otherwise, perhaps we have been led on a wild goose chase, pursuing something that may be neither desirable, nor achievable. Some aspects of science are amenable to testing, as they were in the SATs. I conceded to the arguments that these were inadequate in assessing the whole of science, particularly the important parts of enquiry and practical skills, but I don’t believe anything we’ve been presented with has been adequate either. Additionally, the loss of science status was not a reasonable pay-off. To be workable, assessment systems have to be as simple and sustainable as possible. Until we can address that, if we have to have tracking data (and that’s highly questionable), perhaps we should consider returning to testing to assess science knowledge and forget trying to obtain reliable data on performance and skills – descriptive reporting on these aspects may have to be sufficient for now.

Awaiting some ministerial decisions

What a joke! This from them today:

Changes to 2016 tests and assessments We are aware that schools are waiting for additional information about changes to the national curriculum tests and assessments to be introduced for the next academic year. We are still awaiting some ministerial decisions, in particular in relation to teacher assessment. We will let you know in September, as more information becomes available

Only they’re not kidding. Mike Tidd comments on the same here, but I was unrealistically (and uncharacteristically) optimistic that something would come out before we had to have everything in place in September. Should we laugh or tear our hair out that they are ‘awaiting ministerial decisions’? What – the ministers haven’t been able to decide after 2 years? I won’t hold my breath for anything sensible then. Of course, ‘teacher assessment’ should be a matter for serious consideration, but I doubt that their decisions are delayed for the types of reservations I have on the matter. Whilst it seems to have become the global panacea for all assessments that are too complex to manage, I keep banging on about how inappropriate and unreliable it is. If we are to expect pupil attainment to be a criterion for teacher appraisal and progression, then how can we possibly expect teachers to carry out that assessment themselves? That would be wrong, even if we had extremely reliable tools with which to do it, but we don’t. We have nothing of the sort and we never will have, as long as we assess by descriptive objectives.

So what do I really want? Well, to be honest, although I believe in the essential role of testing within learning, I really want to stop assessing attainment in the way it has become embedded within English culture.  It’s a red herring and has nothing to do with education. I never thought I’d say that – I always had highly ‘successful’ results within the old levels system – but I’m very much questioning the whole notion of pupil attainment as currently understood. It’s based on a narrow set of values which, in spite of all the rhetoric of ‘closing the gap’ are never going to be brilliantly addressed by all pupils. That’s an inescapable statistical fact. And why should they be? Attainment is not the same as education in the same way that climbing the ladder is not the same as being equipped to make informed decisions.

But if we must, then give us all the same tools – the same yardstick. At the end of Year 6, all pupils will be assessed by written tests for maths, reading, spelling and grammar. Their results will then be effectively norm referenced (after a fashion). Do that for all the year groups. I’d prefer it if we moved into the latter half of the 20th century in terms of the effective use of technology, but even an old Victorian style paper is better than the vague nonsense we are currently working with.

So, anyway, as it stands, are we justified in Autumn 2015, when we are visited by OFSTED, in having an assessment system in disarray or are we supposed to have sorted it all out, even though they haven’t?

Too much stirring is spoiling the pudding

The world of education seems to me to be currently in a state of frenzy, particularly in England, but probably fuelled in good part by US ideology. Teachers, like myself, who actually read the bulletins, follow the research, go on facebook, watch the news etc., (maybe there are some that do none of these) are assailed from all directions, with the underlying message that something must be done. For example – in random order:

  • RI/Good/Outstanding
  • Ofsted’s new directives
  • Just Ofsted!
  • Coasting schools
  • Failing teachers
  • Failing heads
  • Teachers want to leave
  • Workload
  • The New Curriculum is good/bad/indifferent
  • Mastery
  • Levels were a bad idea
  • Assessment for Learning is a wonderful thing
  • Progressivism was a terrible idea
  • Trojan horses
  • Text books are great/not great
  • Phonics is good (as is grammar!)
  • Academies will save us/damn us all
  • Parents have the right to choose
  • State schools should be more like private schools
  • Close the gap
  • Practice should be based on research – take your pick which piece
  • Marking is essential feedback/not essential/done badly
  • Independent learning
  • Individualised trajectories
  • Whole class teaching
  • Age related expectations
  • Progressive targets
  • Accountability measures
  • Observations are important/detrimental
  • New technologies are going to save us/damn us all
  • SMCS
  • British Values

I could go on, and readers of this blog could probably add hundreds more items to the list. When I read articles, blogs and research online, everyone has an opinion. Sometimes there is ‘evidence’, although not the kind of evidence that would be accepted within the ‘hard’ sciences. If we teachers were to try to take on board everything that they tell us, so that we are not ‘failing teachers’, we’d become useless. And what are all these methods, tools, strategies, for, exactly? An improvement in attainment of 3 months? Really? Is that anything? I meet successful former pupils – I can not begin to think how I can relate their success to something as nebulous as a 3 month difference in attainment in primary school, even if I could believe that such things can be measured. (In fact, give us that measuring tool – it would help us all a lot!). And then, what is the measure of their success? Are they making a useful contribution to the economy? Is that what it’s about? I really don’t think it is or that it should be. I would like it to stop, now. Nothing can operate well within a climate of such unremitting, frequent and conflicting input and I don’t believe it’s as complicated as all that. There have been successful educators in the past – we have to admit that teachers must have managed it before we had so many directives and all this ‘evidence based practice’. Some of my own teachers were brilliant, but that’s not even the issue. The responsibility for learning, lies with the learner, not the teacher! If we continue to believe we can ‘fix’ things by directing our remedies at the teachers, we’ll fail. The main issue with the teachers is not what they do but what they (don’t) know, and a focus on teaching distracts us from that issue. English teachers are themselves the product of the system and the result of a culture that has removed the responsibility of learning from the learner. I’ve seen this myself, where, if a teacher lacks subject knowledge (for example in the new computing requirements), they do nothing until the CPD is provided for them, yet we live in a technologically advanced world where access to information has never been easier. If we really want to remedy the ills of the English education system, we should:

  • stop making up new responsibilities for teachers
  • stop endlessly tweaking the system
  • recognise that we can’t ‘close the gap’*
  • require excellent subject knowledge
  • recognise that the learner is responsible for their learning

*’closing the gap’ is a phrase for another tirade. Try closing the gap between my sprinting time and that of Usain Bolt!