Source Diversity

An AI tutor that answers every question from one textbook is an AI tutor with one professor's opinions.

This sounds obvious when you state it. It almost never gets considered when these systems are built. A retrieval-based tutor is given a knowledge base to draw from, and very often that knowledge base is whatever happened to be available, a single course, a single institution's lecture notes, a single publisher's materials. The system retrieves passages confidently. The passages get stitched into explanations. The learner receives what appears to be the considered wisdom of the field, when in reality they are receiving the particular framing, the particular examples, the particular blind spots of one source, presented without any signal that other sources exist or would have emphasised different things.

In most fields this is at least partly a problem. In contested fields, which education is full of, it is a serious one. Historians disagree. Psychologists disagree. Even parts of the sciences are more contested than their teaching materials let on. A single-source tutor flattens all of this into a single voice speaking with the authority of the machine. A better approach deliberately draws from multiple sources, and flags when they disagree, and helps the learner see that knowledge itself is something produced by people arguing with each other, rather than a fixed stock of truths to be memorised. This is harder to build. It is also more honest, and more useful, especially at the higher levels of education where the whole point is learning to think. When a source you relied on for years turned out to have had one particular angle all along, how did you first notice?

Last week we looked at the danger of one shiny study. This week we look at what happens when an AI tutor draws all its answers from a single point of view.