Evidence-Centred Assessment

Most tests are written backwards.

Somebody sits down to write a quiz or an exam or a workplace assessment, and the first question they ask themselves is some version of "what sort of question should I put on this thing." Multiple choice or short answer. Five options or four. Two marks or ten. All of these are reasonable things to decide eventually, but they are the wrong starting point, because they are decisions about the format before anyone has worked out what the assessment is actually supposed to tell you. The result is a test that looks professional and measures something, but the something is often not quite the thing anyone cared about. The assessment produces a number, the number gets attached to a person, and a decision gets made on the strength of a measurement that nobody ever quite pinned down.

A more careful approach starts from the opposite end. You begin with what you want to claim about the learner. That they can do a particular thing, that they understand a particular relationship, that they can apply a skill in a new context. Then you ask what you would have to see them do to believe that claim. Only then do you design the task that gives them a chance to show you. The format is the last decision, not the first, and a tool made this way ends up measuring what it said it would measure. Most tests never go through this, which is why so many grades feel unreliable to the people receiving them. When was the last time you took a test that seemed to actually measure what it claimed to measure?

Last week we said that good explanation has more to do with restraint than with budget. This week we look at why most tests are written backwards and what writing them forwards would look like.