I have been a teacher for 26 years, a Headteacher for 11 years and, at the age of 50, this much I know about what we are up against when all we want is our students’ examination papers marked accurately.
Our students deserve the very best; no one I have ever worked with would disagree with that claim. And when it comes to public examinations nothing but the very best service is acceptable for our students. When they have prepared for the examinations and their scripts have been dispatched for marking, we must all have faith that the marking will be fair, accurate and expert. If we cannot believe that is the case, then the system becomes unfit for purpose.
How can mathematics marking be wildly inaccurate? This year we had one A2 student’s mathematics paper remarked; it was returned having been awarded 8 more raw marks – that’s 11% of the total marks available – and his final mathematics A2 grade consequently rose from a B to an A. My own son’s English Language & Literature AS grade has risen from a B to an A following a remark. Five years ago Anthony Seldon said, “Examiners are overworked and overstretched and the calibre of some is just not good enough.” It seems he was right and the situation has worsened since.
My students’ AS Economics results were a bit disappointing this summer. We recently received their recalled scripts. This morning I sat down for twenty minutes with my co-teacher and perused the papers. As you might expect, there were some mistakes our students could have avoided; yet there were numerous instances where, at best, the marking was justifiable but truly harsh and at worst plainly inept. I have chosen one example which typifies my concerns about the quality of marking of my students’ scripts and helps explain why I have written a blog post which is uncharacteristically whingey in tone!
NB: Before you read on, bear in mind that there are just five raw marks between grades in this AS examination, so dropping two marks is 40% of one grade at AS level.
Here’s OCR GCE Economics 2014, Unit F581: Markets in Action, question 1 b):
And here’s the mark scheme:
And here’s Candidate A’s answer which gained a full two marks:
And here’s Candidate B’s answer which gained zero marks, because, as the electronic notation makes clear, the same examiner who marked Candidate A’s script thought Candidate B’s answer was “TV” or “Too Vague”:
Sometimes you need someone else to confirm that you’re not going completely mad. When I showed Kate, my PA, both answers without her knowing what exactly they were and why I was showing them to her, she exclaimed, “Well I reckon that student has copied that one.”
It’s almost enough to drive one to write a letter of complaint to Mark Dawe, Chief Executive of OCR…
I do feel your pain! Here is a similar blog post I wrote last week about AS politics. http://heatherfblog.wordpress.com/2014/09/24/atq-very-unwise/
My reference point for zero marks vs one mark is usually “how much worse than the current answer is just leaving the question blank?” (which certainly gets zero). I don’t teach this subject, but for my money if candidate B’s response gets zero then something has gone terribly wrong!
More than 10 GCSE and GCE remarks have gone from a B to an A at my school. One went from C to A. When schools in the UK live and breath by exam results it really does throw the whole system into question…
Reblogged this on The Echo Chamber.
This is a tragedy for the student, whose grades might be affected later at A2 and may not get into their choice of university, but it’s also a tragedy for the ordinary teacher who has worked all year to get their students to the best possible place in readiness for the exam. As we face our Heads at this time of year looking over our results and Appraisal targets, and the Head says that insufficient progress is being made, if I could account for that purely from poor or erratic marking at the exam boards, I think I’d be considering suing them for loss of earnings.
We must be able to trust the system or it will fall down.
Ofqual have a stated preference for untiered examinations so that every student has access to every grade. One of the consequences, at GCSE, is that you have to ‘fit’ 9 grade boundaries into each exam.
Assuming that we don’t want the lowest grade to be awarded for an exam score which is barely better than getting lucky with a few random thoughts then, effectively, we have to fit 9 grades into scores of 20% and 100%.
Assuming the quality of marking remains constant subjects that are currently tiered at GCSE will see greater proportion of remarks resulting in grade changes.
Each reader of a given paper sees a text differently, even when fully alert and not pressured by time or boredom. This is well known and is a good reason for not relying on (or expecting) exam marking to be “exact”. For what it’s worth I thought that Candidate B matched the mark scheme rather better that Candidate A. Candidate B seemed to clarify the need to choose between different models, rather than just over the number of models in total. Both are imprecise and I think I might have given each 1.
However, I can also get a flashback to meetings in exam board offices where six of us are arguing about exactly this kind of (inevitable) inexactitude. Eventually a chair will say something like “well, I am saying this is 2 marks, so can we all calibrate on that?
My point, I suppose, is that as long as the difference between 2 marks and 0 marks on one question can change a pupil’s career, then we are all guilty of collaborating in an unreasonable system that looks for clear distinctions that do not exist in real life.
I can’t believe what I have just seen! Can you query or argue against this mark? Is it a conspiracy or just plain stupidity on the part of the marker?
I have done the same in the past and found two candidates getting different marks for the same answer. I had recalled papers in the January so it was the students themselves who had to see the errors and face the frustration.
Perhaps if remarks weren’t quite so expensive exam boards would invest more in getting it right the first time. I know if they were cheaper I would have had hundreds remarked just this year.
I don’t disagree with this sentiment but this illustrates the scale and importance of the problem. If our response is to submit more and more appeals, the system is broken. As well as the principal that we need to be able to trust the quality of marking, the need to appeal more results benefits wealthier schools/parents. Just one more reason why this problem needs to be solved.
My over-riding fear is that no-one in a position to solve this problem really cares. Are there enough headlines in this to make it a focus?
Our appeals have tripled in the last 3 years (almost all are parent/student initiated and paid for). The reason is simple – all have brothers, sisters, neighbours or friends who have had successful appeals.
The scary thing isn’t those students who were 1 mark below a boundary, put in a speculative appeal and ended up going up a grade (there will inevitably be subtly different interpretations of markschemes on the margins) but the number of times we are seeing huge increases (over 10% of the total available marks for the paper in some cases) which can only point to ineptitude of the marker.
Of course this leaves us in the position that the wealthy parents/students have access to such remarks whilst the less wealthy don’t. Perhaps schools should blow their Pupil Premium budget on speculative remarks for all eligible students in an effort to level the playing field?
I suspect that exam boards would take the matter more seriously if there were some form of financial penalty for erronous marking – perhaps they repay the exam fees for all such students?
If the variability and unreliability of GCSE marking was replicated in the classroom by teachers, Ofsted would grade a school RI at best, possibly Inadequate. My daughter went from 178/200 total aggregated mark to 196/200 – from Grade A to A* – on a remark of two of the four module papers for her GCSE Religious Studies this summer. This is a 8% change: actually 16% for the two modules concerned, which together account for 100 of the total aggregated available marks. It’s destroyed her faith in the fairness of the system, which is tragic. Nothing more demotivating than to believe, based on irrefutable evidence, that your qualifications will depend more on random lottery and the tiredness of an overworked marker, than on the sustained effort and commitment, and hundreds of hours of study, over two years.
I totally agree with this but sadly don’t think it is a new situation (although I can accept that it might be getting even worse).
I wonder if having a single exam board might help to resolve the problem. I understand the argument that having competition allows us to choose a better board/spec but does this choice actually benefit students in reality? It seems to me that there is a possibility that competition drives a race to lower standards. At best, there doesn’t seem to be any positive effect of competition.
To make things even more concerning, there will be fewer opportunities to correct the effects of poor marking by retaking exams.
This much I know – OCR marking is rubbish and has been getting worse for a number of years – to the point where we abandoned them and moved to another board.
I am surprised by your experience but perhaps this is more about the process and quality control of the awarding organisation. OCR marking basically scans the paper and then sends these images to examiners to mark and has been gaining more and more positive responses, not least because of the ability to maintain a very high quality control over examiners consistency with the use of seeded scripts (previous marked scripts sent unmarked to examiners).
But if it didn’t work the students have to come first.
Exams give an illusion of objectivity but are actually prey to markers’ errors and idiosyncrasies that cannot all be checked. Manu other countries manage without national exams, most obviously the USA. Moderated teacher assessment would save a huge amount of money and time.
I didn’t mark the sample question shown (OCR GCE Economics) but as an experienced examiner and writer of mark schemes I’d say the marks shown are valid with reference to the mark scheme. The mark scheme shows that the ‘opportunity’ must be clearly related to production, or kits, or models, (first mark) and then the second mark related to influence.
Although both candidates answers contained ‘limited number of kits’ I believe this on its own wasn’t enough for the first mark – looking at candidate one’s answer I think it was the extension (‘producing model aircraft’ which got the mark. Yes, the tick was shown next to ‘kits’ but it is usual in marking to tick at the end of the (part) sentence to which the mark applies.
I agree the rest of candidate two’s answer is too vague – remember this is a GCE A-level answer, does ‘to make choices’ sound like a level 3 answer? Not to me. Mid-grade GCSE perhaps but not A-level.
This is poor marking but one of the risks of human marked free text answers. Would it be acceptable in a different format such as a multiple choice question?
The example you have given it so true.. I agree with you to a huge extent. I am a AS student who is about to do its economics 2015 paper… I really hope they won’t be as harsh as they were to the 2014 paper… This really worries me.. But thank you for sharing this since it shows the unfairness of the examiners