Learning Analytics

Watching how students use a platform is not the same as understanding how they are learning.

This is a distinction that gets lost constantly in education technology, and every so often it leads to a scandal nobody sees coming. A dashboard shows the teacher how many minutes each student spent on the platform this week. How many questions they attempted. What percentage they got right. Whether they logged in over the weekend. The dashboard looks authoritative, and the teacher, who is tired and has thirty other students to think about, uses it to decide who needs attention. A student who was sick and offline for three days shows up red. A student who was playing with the screen while their mind was on something else shows up green. The numbers were honest. They measured behaviour, accurately. They did not measure learning at all.

Learning analytics is a real and useful field, but its most honest practitioners will tell you the whole thing rests on an uncomfortable foundation. We can measure, cheaply and automatically, almost anything a student does on a device. We can measure, at great cost and with great difficulty and mostly imperfectly, what is actually happening inside their understanding. The temptation, in a tight budget, is to let the easy measurements stand in for the hard ones. Time on task becomes a proxy for effort. Click patterns become a proxy for thinking. These proxies are sometimes helpful and sometimes wildly misleading, and a system that forgets the difference starts making decisions that look data-driven while being quietly nonsensical. The numbers on the dashboard are real. The thing the numbers are standing in for is the learning, and the two are not the same. When you last had your performance measured at work, how closely did the measurement match what you were actually doing?

Last week we asked how personal personalised learning actually is. This week we look at what gets measured when a teacher opens a dashboard, and at the gap between behaviour and learning.