Algorithmic Fairness

Nobody can agree on what it means for a computer to be fair.

This is not a philosophical shrug. It is a mathematical fact. Researchers have now identified more than twenty different definitions of fairness that you could build into an algorithm, and many of them directly contradict each other. You can design a system that treats every group equally on average, or one that is equally accurate for every group, or one that makes the same kinds of mistakes at the same rate for everyone, but you cannot have all three at once. Pick any two and you have mathematically guaranteed that the third will be unfair by somebody's definition. This is not a bug that clever engineers will eventually fix. It is built into the logic of the problem itself, and it means that every time a company says its AI is fair, it is quietly answering a question it has not told you it was asked.

The people making those choices are usually the engineers, and the people living with the consequences are usually somebody else entirely. A fairness decision about a loan algorithm changes who gets a mortgage. A fairness decision about a hiring algorithm changes who gets an interview. A fairness decision about a school placement algorithm changes which children end up where. These are not technical decisions even though they get implemented in code. They are value decisions about who should bear the cost when a trade-off is unavoidable. If you are told an algorithm is fair, the first question to ask is not whether that is true but whose definition is being used. Who do you think should be in the room when those choices are made?

Last week we said many AI explanations are stories about a different machine. This week we look at why nobody can agree on what fairness in an algorithm actually means.