Non-Compensatory Decisions

Some mistakes cannot be cancelled out by successes elsewhere.

Imagine an AI system deciding whether a student is ready to move on from a topic, based on dozens of different signals. Accuracy on the last quiz. Time spent on the material. Engagement with the practice exercises. Participation in discussions. Performance on a concept map. Some of these signals point yes, some point maybe, some point no. The obvious thing to do is to average them somehow, perhaps weight the ones you trust most, and come out with a single score. If the score is above the threshold, green light. This is how most adaptive systems work, and it is how most people assume a careful evaluator ought to work too. Lots of small signals, combined into one composite answer.

But some signals are not the kind that can be outweighed. If the student has not actually demonstrated the core skill, no amount of strong engagement compensates. If a safety check fails, no amount of speed makes it all right. These are called non-compensatory variables, and they are the signals that have to clear a hurdle before anything else counts. Treating them as just another input, to be averaged with the rest, looks rigorous and is actually dangerous, because it allows high performance on easy things to mask failures on hard ones. A decision framework that knows the difference has a hierarchy. Some things are prerequisites. Others are refinements. The student who is engaged but has not grasped the fundamentals is a different situation from the student who has grasped the fundamentals but is less engaged. Most systems treat them the same. When you think about how your own performance is evaluated, are there things that should clear a hurdle first, rather than being averaged in?

Last week we said evaluation has to start before the building does. This week we look at the kind of decisions where one missing piece should never be cancelled out by ten good ones.