Asking questions to assess attainment in computing

Miles is speaking at the Northern Education Technology Conference at Leeds Beckett University on the 15th May 2017. Click to find out more.

By Miles Berry

Principal lecturer and the subject leader for Computing Education at the University of Roehampton. ​

When Ofsted reported on the state of ICT education in 2011, it observed that assessment was 'no better than satisfactory' in nearly 60 per cent of the schools that were visited. It's far from certain that the situation will have improved significantly since then, particularly with the removal of levels and the move from ICT to computing. It's possible, in fact, that even fewer teachers now know what to look for, or how to give helpful feedback to their pupils. Miles Berry, principal lecturer in computing education at the University of Roehampton, will be at The Education Show to tell us more, but here is an insight into his session.

The challenge with any assessment is that we can only ever use proxy measures to work out if teaching has been successful: we can watch what a pupil does, we can review the work they produce, or we can ask questions. Despite the prevalence of assessing a piece of work against agreed criteria, there's a growing body of opinion that believes asking pupils questions about their knowledge and understanding is more efficient, more reliable and more valid.
​
The McIntosh report on 'Assessment without Levels' recommended the establishment of a 'national item bank' of formative assessment questions. Daisy Christodoulou, head of assessment at Ark Schools, affirms the role of questions in assessing learning: "Instead of having teachers making a judgment about whether a pupil has met each criterion, have pupils answer questions instead." Similarly, the educationalist Dylan Wiliam argues for the use of 'hinge' questions as a way to check on a whole class’ understanding at the crucial hinge points of a lesson, such as after the initial introduction of concepts.

In computing, we're taking this advice seriously. Computing at School, the subject association for computer science, is working with Cambridge Assessment, Durham's Centre for Evaluation and Measurement, and Diagnostic Questions to crowd source a bank of multiple choice assessment questions for teachers to use however they want, to assess their pupils' 

Diagnostic Questions Screenshot

progress in computing. We call this Project Quantum: http://bit.ly/projectquantum, and our focus is on supporting the teaching of computing through guiding content. ​

​The national curriculum for computing is admirably brief, but that can leave teachers at a bit of a loss as to how it should be interpreted. Through measuring progress, the Quantum questions allow teachers to check what pupils know and what they don't, both before and after teaching a topic, and through identifying misconceptions. As more pupils use the Quantum questions, we get a more accurate view of which bits of computing they struggle with, such as understanding how variables are assigned values and just what does make a good password.

The project has been running for less than a year, but we've already got over 2,500 computing questions online (see www.diagnosticquestions.com), covering all three strands of the computing curriculum: foundations, applications and implications.

Whilst we've got a good number of questions from the likes of exam boards, Bebras and code.org, lots of the questions have been written by teachers, initially for use in their own schools, then generously shared through the project. We're starting to curate the questions into quizzes and collections; it's all free, and questions can be exported for use on other platforms too.
​
Quantum will really come into its own when we've got a critical mass of pupils using it on a regular basis. At that point, we'll be able to use statistical analysis to figure out which questions do a good job at really checking understanding, as well as spotting where the common difficulties in computing lie – and perhaps even figure out what we can do to address these.

Miles is speaking at the Northern Education Technology Conference at Leeds Beckett University on Monday, 15th May 2017

Find out more