Trialling moderation

A quick one today to cover the ‘trialling moderation’ session this afternoon.

We had to bring all the documents and some samples of pupils’ writing, as expected.

Moderators introduced themselves. They seemed to be mainly Y6 teachers who also were subject leaders for English. Some had moderated before, but obviously not for the new standards.

The ‘feel’ from the introduction to the session was that it wasn’t as big a problem as we had all been making it out to be. We were definitely using the interim statements and that ‘meeting’ was indeed equivalent to a 4b.

At my table, we expressed our distrust of this idea and our fear that very few of our pupils would meet expected standards. Work from the first pupil was shared and the criteria ticked off. We looked at about 3 pieces of work. It came out as ‘meeting’ even though I felt it was comparable to the exemplar, ‘Alex’. The second pupil from the next school was ‘nearly exceeding’. I wasn’t convinced. There were lots of extended pieces in beautiful handwriting but sentence structures were rather unsophisticated. There was arguably a lack of variety in the range and position of clauses and transitional phrases. There was no evidence of writing for any other  curriculum area, such as science.

I put forward the work from a pupil I had previously thought  to be ‘meeting’ but had then begun to doubt. I wanted clarification. Formerly, I would have put this pupil at a 4a/5c with the need to improve consistency of punctuation. Our books were the only ones on our table (and others) that had evidence of writing across the curriculum; we moved a few years ago to putting all work in a ‘theme book’ (it has its pros and cons!).

Unfortunately the session was ultimately pretty frustrating as we didn’t get to agree on the attainment of my pupil; I was told that there needed to be evidence of the teaching process that had underpinned the writing that was evident in the books. That is to say, there should be the grammar exercises where we had taught such things as ‘fronted adverbials’ etc. and then the written pieces in which that learning was then evidenced. I challenged that and asked why we couldn’t just look at the writing as we had done for the first pupil. By then the session was pretty much over. In spite of the moderator’s attempt to finish the moderation for me, we didn’t. The last part of the session was given over to the session leader coming over and asking if we felt OK about everything, and my reply that no, I didn’t. I still didn’t know which of the multiplicity of messages to listen to and I hadn’t had my pupil’s work moderated. I had seen other pieces of work, but I didn’t trust the judgements that had been made.

The response was ‘what mixed messages?’ and the suggestion that it may take time for me to ‘get my head around it’ just like I must have had to do for the previous system. She seemed quite happy that the interim statements were broadly equivalent to a 4b and suggested that the government certainly wouldn’t want to see the data showing a drop in attainment. I suggested that if people were honest, that could be the only outcome.

My colleague didn’t fare much better. She deliberately brought samples from a pupil who fails to write much but when he does, it is accurate, stylish and mature. He had a range of pieces, but most of them were short. The moderator dismissed his work as insufficient evidence but did inform my colleague that she would expect to see the whole range of text types, including poetry because otherwise how would we show ‘figurative language and metaphor’?

I’m none the wiser but slightly more demoralised than before. One of my favourite writers from last year has almost given up writing altogether because he knows his dyslexia will prevent him from ‘meeting’. Judging the writing of pupils as effectively a pass or fail is heart-breaking. I know how much effort goes into their writing. I can see writers who have such a strong grasp of audience and style, missing the mark by just a few of the criteria. This is like being faced with a wall – if you cant get over it, stop bothering.

We are likely to be doing a lot of writing over the next few weeks.

 

Advertisements

An upbeat end to a long, hard week

This is an uncharacteristic post for me. I started this blog as a vehicle for sharing critical views on aspects of education and I suppose a lot of them are approaching what some might term ‘rants’. In any case, it’s never been a blog for sharing good practice or teacher resources (although some of those are actually available elsewhere!) and yes, much of it is somewhat negative.

This, however, is simply a feelgood story after a week I expected to be hard-going. The latter because maintaining concentration following 6 hours of parents’ evenings after school is always quite a slog. That was all good, however, in spite of the very depressing and alarming expectations thrust at us by the DfE on Monday.

Today, however, was a delight. We had set aside the whole day to cover some of the art curriculum which has taken such a back seat recently and the pupils apparently had an incredibly good time. It wasn’t complicated. We were developing the theme I had introduced earlier and extending the creative bit through printing and 3D mobile sculpture. It was almost a guilty pleasure to not be shoehorning them into passive sentences and modal verbs for a change.

So that was fun – but to cap it, I run the band after school on Fridays. While we were waiting for the keyboard players to sort out their parts, I picked up the bass, since my bass player was away ill, and started plucking out the line to Zawinul’s ‘Mercy, Mercy, Mercy’, at which point both guitars and the drummer started playing along like it was a bona fide jam session. Bear in mind that the age range is 6 to 11yrs. We followed that by a full play through of ‘Stairway to Heaven’ and it appeared that the keyboard players had actually learned their (synth flute) parts at last. Several players took solos on ‘Watermelon Man’ and then, as we were packing up, we got into conversation about other music they would all like to play. The list included: Smells Like Teen Spirit; Paranoid or Iron Man; anything by ACDC; ‘something by Steppenwolf’ and possibly Oye Como Va, Santana style. I couldn’t be more chuffed.

Coming to a stadium near you soon!

 

If I were the school leader…

I have a student teacher on placement in my class at the moment. It’s interesting to remind myself of the long list of criteria in the teachers’ standards, that we have to consider in observations. As a teacher giving advice, I know which of these are important and which I’d give a lot less weight to when making any kind of value judgement.

I’ve never been a fan of classroom observations – for all the reasons that are now part of general discussion – particularly those that attempt to grade the teacher based on a snapshot of 20-40 minutes. It’s not how I’d do it. But the job of a school leader is a tough one, I believe, and nowhere tougher than in securing quality of teaching among the staff. If it were me, what would I look for?

When teachers are worrying about trying to tick the increasing number of boxes put forward to us, actual performance deteriorates. We’re focussed on what we think will be the assessment of what we should be doing, not on what we are actually doing. Humans can’t multi-task. By thinking of the process, the process itself suffers. This is a well-documented tactic used by those who would seek to remove unwanted personnel: increase the level of scrutiny and nit-pick every move so that eventually the subject can hardly function. This is a hard-nosed game that often ends in resignation, mental breakdown and sometimes suicide.

It’s not really the way I would go, and if micro-management is not the best way to ensure the pupils are getting a good education, then perhaps it boils down to a much reduced but more important key set of desirable features/skills. I think my list would be brief. I’d be looking specifically for evidence that the teacher:

  • Knows the subject(s) (and the curriculum) well
  • Knows what the pupils have learned and what to teach next
  • Manages behaviour so that pupils can focus
  • Teaches clearly so that pupils can understand
  • Picks up on issues and remedies them
  • Is compassionate

Everything else, surely, is either part of the craft or derived from opinion?

Challenge to this is welcome.

 

Education, not ‘attainment’.

I clearly remember two separate, but similar introductions to the new year from teachers at my junior school, though the identity of the teachers has now faded from my memory. On both occasions, something was written in chalk, on the board and obviously with enough dramatic impact for me to recall for 40 years. One of these was the literal translation of the word ‘educere’ as ‘to lead/draw out’ and the other was the old catechism ‘wissen ist macht’ translated as ‘knowledge is power’ but conveyed to us as our knowledge would give us power (to think for ourselves, to make informed decisions, to resist negative pressure). I’m reminded of these when I am witness to educational debates and particularly those that focus on the purpose of education. Of course, like so many of us, I must be strongly influenced by my early experience, which in this case leads me to question some of the current assumptions about teaching and learning and to wonder what happened to the idea that education was about education as opposed to being about attainment.

What is wrong with attainment?

I think the issue is less with the notion that we can ‘strive to attain’, but rather with what it leaves out, or shoves aside, and the perverse incentives which it generates. I’m concerned, currently, that there has been a recent surge to embrace, in England, the ‘freedom’ afforded by the removal of levels alongside a new, unquestioning fandom for ‘mastery’ and the idea that all primary pupils in a certain year, learn a certain body of knowledge in order to be ‘meeting’ the requirements of that year. I would worry about the assumptions behind what that body of knowledge should be, whether it was appropriate for all pupils within a year, regardless of their age and experience, even without the high-stakes ranking tests at the end of KS2. But this should be a matter for concern. Attainment has moved from being a step along a continuum and has returned, very much to a comparison of a pupil’s worth against other pupils of the same year, within school and within the country. You can only attain highly, if you are well above the average score. Teachers can only be very successful, if they achieve the impossible and somehow push their students to the right of the bell-curve. This, like the drive to achieve ever higher levels in the old system, is high stakes indeed. Unfortunately, if we focus so highly on attainment, we increase the incentive to cheat – examples from education and sport abound. The focus on doing it well has been replaced by the focus on doing it better than everyone else. I’m suggesting that is a corruption of the purposes of education.

But I’m concerned not only with the focus, but also our failure to question the presumption of what ‘doing well’ means. I think it would be true to say that we don’t all agree on this. The polarisation of the debate about skills v knowledge is only one example of difference in views and of the pendulum swing of ideology in English (US?) education. This matters, because it affects what is valued and what is taught.

Primary science is a classic victim of ideological meddling having an impact on what goes on in schools. It goes briefly like this: science is seen as having too low a status. It is given ‘core’ status and tested at the end of KS2. The tests are seen to be inadequate in assessing ‘real science’ and are having a negative ‘backwash’ on the teaching of science. The tests are scrapped. Teaching is supposed to focus on science skills, but without the tests, science loses its comparative status. Subject knowledge resurfaces as crucial. Teachers are given dozens of knowledge-based objectives against which to assess pupils, but as of yet, no effective means to do this. (I would predict the return of some form of testing). International comparisons (PISA) historically place England above average, with little change from 2006, in terms of science (pdf), but the impact (or lack of) of scrapping the SATs in 2009 has yet to be measured.

But what of PISA, anyway?

I confess, I love data. I want to see trends, graphical representations of percentages, comparisons, etc. and I crave evidential bases for actions. However I’m not alone in questioning the validity and reliability of the use of data in the day to day business of education, especially in the form of international comparisons which have a major impact on what we do in English schools: ‘PISA says xxxx and therefore…’ My issues are this:

  • How do we measure attainment?

As yet, we are still dependent on conventional testing as the reliable method of measuring attainment. PISA write at length of the limitations of this, but ultimately have no alternative. Reliable, however does not mean valid, hence the arguments in favour of alternatives such as teacher assessment, which I argue strongly against as a means of reliable measurement.

  • What are we measuring, exactly?

Although I’m a massive fan of tests and exams for their impact on personal learning and as an opportunity to demonstrate knowledge, we have to accept that they are selective and limited in what they test. PISA represents a view of what should be tested, but what is left out, is necessarily vastly more than what is included. What exactly does a high score in PISA represent? It represents something but is it the thing that we most want out of our education system or is there something that we haven’t measured? We necessarily have to cherry pick our criteria but these relate to attainment in very specific curricula and not, we could argue, to the overall education of a population. Extrapolation is also a problem. If we look at high attaining countries, do we really know what it is that makes the difference and can we really just extract that and apply it to our own, even if we think this measure of attainment is valuable?

  • Who is it for?

This is an important question and a political one. PISA represents an OECD idea of what to measure for attainment. We might agree with much of this and we might recognise (as I do!) that national curricula map quite closely with each other. However, there is room for doubt, particularly when one’s own government can make a political decision, for example, to leave out any mention of climate change in its new curriculum! PISA was set up, apparently, with a view to improving education policies and outcomes, which seems laudable, but which outcomes and for whom? Are we talking about a future of a reasonably competent population, capable of contributing towards the ‘economic success’ of their country, or of a highly informed, analytical, thinking population that has the wherewithal to solve problems and question policies? Or, is there something else, too?

So what then?

I will continue to argue for something rather more idealistic than an attainment model of education. Although competition can be some kind of incentive, and I have used it myself in the class, education, in my view, is not about being better, higher up the tree, earning more than the other or, dare I say it, consuming faster and causing more destruction. Education is really big. It’s too big to consign to the ‘mastery’ of a handful of objectives and it’s not measured by the attainment of a high score in a set of tests at the end of a Key Stage. Really good teaching is useful but insufficient and, I have to say, not essential (had it been, I would never have passed ‘O’ level Chemistry). Any single source, such as one teacher, is not enough. Without learners taking responsibility, it doesn’t happen. Maybe ‘character’ is what that drives people to the top, but ‘the top’ can be a very destructive place and it’s not everybody’s idea of fulfilment. Education doesn’t require ‘character’ it requires a kind of craving, not for attainment but to know and to be able to do and to understand more than before. By all means, learners should be given opportunities to challenge themselves in chosen areas, marked by assessments of various types, but these shouldn’t be hurdles; they should be available when wanted.

Some views of attainment

Here are some models of pupils within the system in relation to attainment of objectives. The question is, who is doing better? Who is attaining higher? Who is likely to get a lot less teacher attention and why? (apologies for the blurriness – I’ll get back to that but wanted to get this posted while I have time)

p a

pb

pc

My curriculum that isn’t in the curriculum

I thought I’d just end with some examples (we could all come up with these and please feel free to add yours) of some of the ‘important’ things I’ve taught so far this term:

  • Talking politely gets a better result
  • The large spider is called Tegenaria and he is a male – look at his clubbed pedipalps
  • The environment won’t change to suit your needs
  • Using a back loop when you’re sewing means the thread stays in
  • Slugs have a right to live, too (and by the way are related to to the octopus)
  • You’ve just used an example of the ‘pathetic fallacy’, without knowing it
  • Excel needs numerical data
  • What you learn in class time is not enough
  • Poems work better if the strong rhyme comes in the second line

Final report of the Commission on Assessment without Levels – a few things.

I’ve read the report and picked out some things. This is not a detailed analysis, but more of a selection of pieces relevant to me and anyone else interested in primary education and assessment:

Our consultations and discussions highlighted the extent to which teachers are subject to conflicting pressures: trying to make appropriate use of assessment as part of the day-today task of classroom teaching, while at the same time collecting assessment data which will be used in very high stakes evaluation of individual and institutional performance. These conflicted purposes too often affect adversely the fundamental aims of the curriculum,

Many of us have been arguing that for years.

the system has been so conditioned by levels that there is considerable challenge in moving away from them. We have been concerned by evidence that some schools are trying to recreate levels based on the new national curriculum.

Some schools are hanging on to them like tin cans in the apocalypse.

levels also came to be used for in-school assessment between key stages in order to monitor whether pupils were on track to achieve expected levels at the end of key stages. This distorted the purpose of in-school assessment,

Whose fault was that?

There are three main forms of assessment: in-school formative assessment, which is used by teachers to evaluate pupils’ knowledge and understanding on a day-today basis and to tailor teaching accordingly; in-school summative assessment, which enables schools to evaluate how much a pupil has learned at the end of a teaching period; and nationally standardised summative assessment,

Try explaining that to those who believe teacher assessment through the year can be used for summative purposes at the end of the year.

many teachers found data entry and data management in their school burdensome.

I love it, when it’s my own.

There is no intrinsic value in recording formative assessment;

More than that – it degrades the formative assessment itself.

the Commission recommends schools ask themselves what uses the assessments are intended to support, what the quality of the assessment information will be,

I don’t believe our trial system using FOCUS materials and assigning a score had much quality. It was too narrow and unreliable. We basically had to resort to levels to try to achieve some sort of reliability.

Schools should not seek to devise a system that they think inspectors will want to see;

!

Data should be provided to inspectors in the format that the school would ordinarily use to monitor the progress of its pupils

‘Ordinarily’ we used levels! This is why I think we need data based on internal summative assessments. I do not think we can just base it on a summative use of formative assessment information!

The Carter Review of Initial Teacher Training (ITT) identified assessment as the area of greatest weakness in current training programmes.

We should not expect staff (e.g. subject leaders) to devise assessment systems, without having had training in assessment.

The Commission recommends the establishment of a national item bank of assessment questions to be used both for formative assessment in the classroom, to help teachers evaluate understanding of a topic or concept, and for summative assessment, by enabling teachers to create bespoke tests for assessment at the end of a topic or teaching period.

But don’t hold your breath.

The Commission decided at the outset not to prescribe any particular model for in-school assessment. In the context of curriculum freedoms and increasing autonomy for schools, it would make no sense to prescribe any one model for assessment.

Which is where it ultimately is mistaken, since we are expected to be able to make comparisons across schools!

Schools should be free to develop an approach to assessment which aligns with their curriculum and works for their pupils and staff

We have a NATIONAL CURRICULUM!

Although levels were intended to define common standards of attainment, the level descriptors were open to interpretation. Different teachers could make different judgements

Well good grief! This is true of everything they’re expecting us to do in teacher assessment all the time.

Pupils compared themselves to others and often labelled themselves according to the level they were at. This encouraged pupils to adopt a mind-set of fixed ability, which was particularly damaging where pupils saw themselves at a lower level.

This is only going to be made worse, however, by the ‘meeting’ aspects of the new system.

Without levels, schools can use their own assessment systems to support more informative and productive conversations with pupils and parents. They can ensure their approaches to assessment enable pupils to take more responsibility for their achievements by encouraging pupils to reflect on their own progress, understand what their strengths are and identify what they need to do to improve.

Actually, that’s exactly what levels did do! However…

The Commission hopes that teachers will now build their confidence in using a range of formative assessment techniques as an integral part of their teaching, without the burden of unnecessary recording and tracking.

They hope?

Whilst summative tasks can be used for formative purposes, tasks that are designed to provide summative data will often not provide the best formative information. Formative assessment does not have to be carried out with the same test used for summative assessment, and can consist of many different and varied tasks and approaches. Similarly, formative assessments do not have to be measured using the same scale that is used for summative assessments.

OK – this is a key piece of information that is misunderstood by nearly everybody working within education.

However, the Commission strongly believes that a much greater focus on high quality formative assessment as an integral part of teaching and learning will have multiple benefits:

We need to make sure this is fully understood. We must avoid formalising what we think is ‘high quality formative assessment’ because that will become another burdensome and meaningless ritual. Don’t get me started on the Black Box!

The new national curriculum is founded on the principle that teachers should ensure pupils have a secure understanding of key ideas and concepts before moving onto the next phase of learning.

And they do mean 100% of the objectives.

The word mastery is increasingly appearing in assessment systems and in discussions about assessment. Unfortunately, it is used in a number of different ways and there is a risk of confusion if it is not clear which meaning is intended

By  leading politicians too. A common understanding of terms is rather important, don’t you think?

However, Ofsted does not expect to see any specific frequency, type or volume of marking and feedback;

OK, it’s been posted before, but it’s worth reiterating. Many SL and HTs are still fixated on marking.

On the other hand, standardised tests (such as those that produce a reading age) can offer very reliable and accurate information, whereas summative teacher assessment can be subject to bias.

Oh really? Then why haven’t we been given standardised tests and why is there still so much emphasis on TA?

Some types of assessment are capable of being used for more than one purpose. However, this may distort the results, such as where an assessment is used to monitor pupil performance, but is also used as evidence for staff performance management. School leaders should be careful to ensure that the primary purpose of assessment is not distorted by using it for multiple purposes.

I made this point years ago.

Why my hatred of Harvest Festival is more than sour grapes

I never liked having to do Harvest Festival, but at least when I was running morning singing assemblies, I could inject it with my own brand of culture, inviting musicians to volunteer to play to the songs we’d chosen and scoring simple accompaniments. That was almost fun. When singing assemblies were scrapped in favour of ‘visitors’, I was somewhat peeved, but also glad I could take a step back and let someone else do it. The festival has been something of a limp affair ever since, but I remain as aloof as I can. That’s the sour grapes bit.

The school’s celebration of it is historical. In spite of being a non-denominational state school, they have traditionally traipsed all the children from KS2 up to the local church once a year for the event and I can’t fathom why. We’re not affiliated with that church or any other. Most of the pupils I have taught have referred to themselves as having ‘no religion’, although there may well be something else written on their data sheets by their parents. I find the ritual that we are required to carry out, irksome if not downright disturbing. It’s narrow-minded, selfish, arrogant, patronising and hypocritical to sing songs of praise to a deity who has apparently created everything and given us this ‘bounty’, and in the same breath, ‘pray for those in need’. Are we really saying, ‘Thanks for all this stuff, Lord. Shame about those who starved to death this year.’ and all the while smugly thinking that our measly donation of a tin of spaghetti is somehow the meaning of it all? I do despise the atmosphere of benign indifference, the unquestioning faith in one’s own entitlement, the incredible Victorian confidence that things are the way they should be, without a hint of alarm at the suffering caused by our harvesting of the world’s resources and the outrageously widening gulf between the 1% who own more than half of it and the other 99%. Surely I can’t be the only one who finds this celebration to be both ridiculous and insulting.

Unpicking just one tiny part of interim teacher assessment

We’ve been waiting, but not, I may say, with bated breath. There was no doubt in my mind that the descriptors would be less useful for measuring attainment than a freshly-caught eel. Let’s just look at Reading for KS2 and see how easy it would be to make judgements that would be fair across pupils, classes and schools.

The pupil can:
• read age-appropriate books with confidence and fluency (including whole novels)

  • which books are deemed age-appropriate?
  • define confidence
  • define fluency
  • novels?
  • compare it to the KS1 statement: read words accurately and fluently without overt sounding and blending, e.g. at over
    90 words per minute

• read aloud with intonation that shows understanding

  • intonation does not imply understanding. My best orator from last year, had no understanding of what he was reading so beautifully.

• work out the meaning of words from the context

  • how is this an end of ks2 requirement? This is what we do from the moment we start to read.

• explain and discuss their understanding of what they have read, drawing inference and justifying these with evidence

  • again – how do we extract the end of KS2 requirement from this? It could apply to year1 or PhD level.
  • compare the KS1 requirement: make inferences on the basis of what is said and done

• predict what might happen from details stated and implied

  • again – end of KS2 requirement?
  • compare to KS1 working at greater depth: predict what might happen on the basis of what has been read so far

• retrieve information from non-fiction

  • again – end of KS2 requirement? To what extent? What level of non-fiction? What type of information? In what way? If a child can not retrieve information from non-fiction, they are operating at a very much lower level than the end of the key stage.

• summarise main ideas, identifying key details and using quotations for illustration

  • to what extent? Again, this is also a degree level requirement

• evaluate how authors use language, including figurative language, considering the impact on the reader

  • to what extent?

• make comparisons within and across books.

  • what comparisons? ‘This book has animals and this book has machines.’
  • KS1 greater depth: make links between the book they are reading and other books they have read

I’ve felt like I’ve been arguing for many years, against the strength of mythological belief in the wonders of teacher assessment. Fortunately, it looks like, at long last, there is some recognition in this report that it can not be used where reliability is an issue, e.g.

Some types of assessment are capable of being used for more than one purpose. However, this may distort the results, such as where an assessment is used to monitor pupil performance, but is also used as evidence for staff performance management. School leaders should be careful to ensure that the primary purpose of assessment is not distorted by using it for multiple purposes. (p 24)

and the attempt to create assessment statements from the national curriculum objectives is just one clear reason why that is true. Mike Tidd suggests that we are heading towards the demise of statutory teacher assessment used in this way. Good, because it’s been a nightmare we should be happy to wake from!

The nonsense of ‘teacher assessment’ – an analogy

As we approach the start of the new school year, some of us will be continuing to try to make a silk purse out of the sow’s ear of  the new assessment requirements, ‘formally’ introduced last year. Whatever system individual schools decide to use to approach this farce, teachers will be expected to make judgements based on ‘teacher assessment’. Almost everywhere, this will be accepted without question, so I’m going to try to outline in simple terms just how I think it does not make sense.

I’m using a high-stakes analogy in which human judgement of performance needs to be seen to be as reliable as possible – the ‘execution’ score for competitive gymnastics as follows:

  • 6 independent, highly skilled, judges
  • 1 individual is judged on 1 performance at a time (and within a limited time)
  • Each performance has a small number of clearly defined criteria
  • There is no conferring (or moderating!)
  • The maximum score is 10 and points are dropped for errors

These are pretty good conditions for a high degree of reliability and yet the judges still arrive at different scores. Because of that, the top and bottom scores are dropped and the remaining 4 are averaged. Even so, the resulting scores are often ‘disputed’, although queries and official objections are not allowed. The judges are not the coaches and will not be held to account for the performance of the gymnasts.

Now let’s compare that with teacher assessment in an English primary school:

  • 1 class teacher, most of whom are not experts, neither in the subject, the curriculum nor in assessment
  • 32 individuals are judged on multiple performances in multiple subjects throughout the year
  • There are hundreds of criteria (somewhere along the lines of 130 for the core subjects in year 5)
  • Reliability is expected to be improved by moderation and discussion (conferring!)
  • There is no way to eliminate outlying judgements
  • There is no transparent way to score or translate observations of performance into grades

In most schools, there will be some kind of tracking system whereby teachers will be asked to make termly entries along the lines of ‘developing, meeting, exceeding’ and degrees thereof, for tracking purposes, culminating in a final decision which will indicate pupil attainment (readiness to move to the next stage) and teacher effectiveness for that year. In many cases, in spite of union objections, these judgements will form part of appraisal, promotion and performance-related pay. Is there any way, under those circumstances, that teacher assessment can reliable enough to be used for the high-stakes purposes expected in English primary schools?

Primary Science Assessment – no miracles here

In April I wrote here on the draft science assessment guidance from the TAPS group. The final version is now out in the public domain (pdf), described thus:

“The Teacher Assessment in Primary Science (TAPS) project is a 3 year project based at Bath Spa University and funded by the Primary Science Teaching Trust (PSTT), which aims to develop support for a valid, reliable and manageable system of science assessment which will have a positive impact on children’s learning.”

I was vainly hoping for a miracle: valid, reliable AND manageable! Could they pull off the impossible? Well if you read my original post, you’d know that I had already abandoned that fantasy. I’m sorry to be so disappointed – I had wished to be supportive, knowing the time, effort (money!) and best of intentions put into the project. Others may feel free to pull out the positive aspects but here I am only going to point out some of the reasons why I feel so let down.

Manageable?

At first glance we could could probably dismiss the guidance on the last of the three criteria straight away. 5 layers and 22 steps would simply not look manageable to most primary school teachers. As subject leader, I’m particularly focussed on teaching science and yet I would take one look at that pyramid and put it away for another day. Science has such low priority, regardless of the best efforts of primary science enthusiasts like myself, that any system which takes more time and effort than that given to the megaliths of English and Maths, is highly unlikely to be embraced by class teachers. If we make assessment more complicated, why should we expect anything else? Did the team actually consider the time it would take to carry out all of the assessment steps for every science objective in the New Curriculum? We do need to teach the subject, after all, even if we pretend that we can assess at every juncture.

Reliable?

In my previous post on this subject, I did include a question about the particular assessment philosophy of making formative assessment serve summative aims. I question it because I assert that it can not. It is strongly contested in the research literature and counter-indicated in my own experience. More importantly, if we do use AfL (assessment for learning/formative assessment) practices for summative data then in no way can we expect it to be reliable! Even the pupils recognise that it is unfair to make judgements about their science based on their ongoing work. Furthermore, if it is teacher assessment for high stakes or data driven purposes then it can not be considered reliable, even if the original purpose is summative. At the very least, the authors of this model should not be ignoring the research.

Valid?

Simply put, this means ‘does what it says on the tin’ – hence the impossibility of assessing science adequately. I’m frequently irritated by the suggestion that we can ‘just do this’ in science. Even at primary school (or perhaps more so) it’s a massive and complex domain. We purport to ‘assess pupils’ knowledge, skills and understanding’ but these are not simply achieved. At best we can touch on knowledge, where at least we can apply a common yardstick through testing. Skills may be observed, but there are so many variables in performance assessment that we immediately lose a good deal of reliability. Understanding can only be inferred through a combination of lengthy procedures. Technology would be able to address many of the problems of assessing science, but as I’ve complained before, England seems singularly disinterested in moving forward with this.

Still, you’d expect examples to at least demonstrate what they mean teachers to understand by the term ‘valid’. Unfortunately they include some which blatantly don’t. Of course it’s always easy to nit-pick details, but an example, from the guidance, of exactly not assessing what you think you are assessing is, ‘I can prove air exists’ (now there’s a fine can of worms!) which should result from an assessment on being able to prove something about air, not the actual assessment criterion ‘to know air exists’ (really? In Year 5?).

1. Ongoing formative assessment

This is all about pupil and peer assessment and also full of some discomforting old ideas and lingering catch phrases. I admit, I’ve never been keen on WALTs or WILFs and their ilk. I prefer to be explicit with my expectations and for the pupils to develop a genuine understanding of what they are doing rather than cultivate ritualised, knee-jerk operations. Whilst I concede that this model focusses on assessment, it’s not very evident where the actual teaching takes place. Maybe it is intended to be implied that it has already happened, but my concern is that this would not be obvious to many teachers. The guidance suggests, instead, that teachers ‘provide opportunities’, involve pupils in discussions’, ‘study products’, ‘adapt their pace’ and ‘give feedback’. I would have liked to see something along the lines of ‘pick up on misconceptions and gaps in knowledge and then teach.’

Most disheartening, is to see the persistence of ideas and rituals to do with peer assessment. Whilst peer assessment has come under some scrutiny recently for possibly not being as useful as it has been claimed, I think it does have a place, but only with some provisos. In my experience, the most useful feedback comes not when we insist that it’s reduced to a basic format (tick a box, etc.) but when pupils can genuinely offer a thoughtful contribution. As such, it has to be monitored for misinformation; the pupils have to be trained to understand that their peers might be wrong and this takes time. After fighting hard against mindless practices such as ‘two stars and a wish’, my heart sinks to find it yet again enshrined in something that is intended for primary teachers across the country.

2. Monitoring pupil progress

In this layer, we move from the daily activities which are considered part of ongoing, formative assessment, to the expectation that teachers are now to use something to monitor ‘progress’. This involves considerable sleight of hand and I would have to caution teachers and leadership to assume that they can just do the things in the boxes. Let’s see:

TEACHERS BASE THEIR SUMMATIVE JUDGEMENTS OF PUPILS’ LEARNING ON A RANGE OF TYPES OF ACTIVITY

When? To get a good range, it would have to start early in the year, particularly if it includes all the science coverage from the curriculum. In that case, summative judgements are not reliable, because the pupils should have progressed by the end of the year. If it takes place at the end of the year, do we include the work from the earlier part of the year? Do we ignore the areas covered up to February? If we don’t, do we have time to look at a range of types of activity in relation to everything they should have learned? Neither ongoing work, nor teacher observation, are reliable or fair if we need this to be used for actual comparative data.

TEACHERS TAKE PART IN MODERATION/DISCUSSION WITH EACH OTHER OF PUPILS’ WORK IN ORDER TO ALIGN JUDGEMENTS

Oh how I despise the panacea of moderation! This is supposed to reduce threats to reliability and I’m constantly calling it out in that regard. Here they state:

“Staff confidence in levelling is supported by regular moderation. The subject leader set up a series of 10 minute
science moderation slots which take place within staff meetings across the year. Each slot consists of one class
teacher bringing along some samples of work, which could be children’s writing, drawings or speech, and the staff agreeing a level for each piece. This led to lengthy discussions at first, but the process became quicker as staff developed knowledge of what to look for.”

Where to begin? Staff confidence does not mean increased reliability. All it does is reinforce group beliefs. 10 minute slots within staff meetings are unrealistic expectations, both in perceiving how long moderation takes and in the expectation that science will be given any slots at all. Whatever staff ‘agree’, it can not be considered reliable: a few samples of work are insufficient to agree anything; the staff may not have either science or assessment expertise to be qualified to make the judgement; more overtly confident members of staff may influence others and there may be collective misunderstanding of the criteria or attainment; carrying out a 10 minute moderation for one pupil in one aspect of science does not translate to all the other pupils in all the aspects of science we are expected to assess. It might also have been a good idea to vet this document for mention of levels, given that it was brought out to address their removal.

3.Summative reporting

A MANAGEABLE SYSTEM FOR RECORD-KEEPING IS IN OPERATION TO TRACK AND REPORT ON PUPILS’ LEARNING IN SCIENCE

I just want to laugh at this. I have some systems for record-keeping which in themselves are quite manageable, once we have some real data. Where we have testable information, for example, factual knowledge, they might also mean something, but as most of us will know, they quickly become a token gesture simply because they are not manageable. Very quickly, records become ‘rule of thumb’ exercises, simply because teachers do not have the time to gather sufficient evidence to back up every statement. I note that one of the examples in the guide is the use of the old APP rubric which is no longer relevant to the new curriculum. We made the best of this in our school in a way that I devised to try to be as sure of the level as was possible, but even then, we knew that our observations were best guesses. The recording system is only as good as the information which is entered, despite a widespread misconception that records and assessment are the same thing! I’m no longer surprised, although still dismayed, at the number of people that believe the statistics generated by the system.

I didn’t intend this to be a balanced analysis – I’d welcome other perspectives – and I apologise to all involved for my negativity, but we’re clearly still a long way from a satisfactory system of assessing primary science. The model can not work unless we don’t care about reliability, validity or manageability. But in that case, we need no model. If we want a fair assessment of primary science, with data on pupils’ attainment and progress that we feel is dependable, then we need something else. In my view, it only begins to be attainable if we make creative use of technology. Otherwise, perhaps we have been led on a wild goose chase, pursuing something that may be neither desirable, nor achievable. Some aspects of science are amenable to testing, as they were in the SATs. I conceded to the arguments that these were inadequate in assessing the whole of science, particularly the important parts of enquiry and practical skills, but I don’t believe anything we’ve been presented with has been adequate either. Additionally, the loss of science status was not a reasonable pay-off. To be workable, assessment systems have to be as simple and sustainable as possible. Until we can address that, if we have to have tracking data (and that’s highly questionable), perhaps we should consider returning to testing to assess science knowledge and forget trying to obtain reliable data on performance and skills – descriptive reporting on these aspects may have to be sufficient for now.

Awaiting some ministerial decisions

What a joke! This from them today:

Changes to 2016 tests and assessments We are aware that schools are waiting for additional information about changes to the national curriculum tests and assessments to be introduced for the next academic year. We are still awaiting some ministerial decisions, in particular in relation to teacher assessment. We will let you know in September, as more information becomes available

Only they’re not kidding. Mike Tidd comments on the same here, but I was unrealistically (and uncharacteristically) optimistic that something would come out before we had to have everything in place in September. Should we laugh or tear our hair out that they are ‘awaiting ministerial decisions’? What – the ministers haven’t been able to decide after 2 years? I won’t hold my breath for anything sensible then. Of course, ‘teacher assessment’ should be a matter for serious consideration, but I doubt that their decisions are delayed for the types of reservations I have on the matter. Whilst it seems to have become the global panacea for all assessments that are too complex to manage, I keep banging on about how inappropriate and unreliable it is. If we are to expect pupil attainment to be a criterion for teacher appraisal and progression, then how can we possibly expect teachers to carry out that assessment themselves? That would be wrong, even if we had extremely reliable tools with which to do it, but we don’t. We have nothing of the sort and we never will have, as long as we assess by descriptive objectives.

So what do I really want? Well, to be honest, although I believe in the essential role of testing within learning, I really want to stop assessing attainment in the way it has become embedded within English culture.  It’s a red herring and has nothing to do with education. I never thought I’d say that – I always had highly ‘successful’ results within the old levels system – but I’m very much questioning the whole notion of pupil attainment as currently understood. It’s based on a narrow set of values which, in spite of all the rhetoric of ‘closing the gap’ are never going to be brilliantly addressed by all pupils. That’s an inescapable statistical fact. And why should they be? Attainment is not the same as education in the same way that climbing the ladder is not the same as being equipped to make informed decisions.

But if we must, then give us all the same tools – the same yardstick. At the end of Year 6, all pupils will be assessed by written tests for maths, reading, spelling and grammar. Their results will then be effectively norm referenced (after a fashion). Do that for all the year groups. I’d prefer it if we moved into the latter half of the 20th century in terms of the effective use of technology, but even an old Victorian style paper is better than the vague nonsense we are currently working with.

So, anyway, as it stands, are we justified in Autumn 2015, when we are visited by OFSTED, in having an assessment system in disarray or are we supposed to have sorted it all out, even though they haven’t?