Federated Learning

Keeping your data on your device is not the same as keeping your data safe.

In response to growing alarm about how much personal information big companies collect, a newer approach has become fashionable. Instead of sending all your data to a central server, the model is trained on your device, and only updates to the model get sent back. Your photos stay on your phone. Your messages stay on your phone. The company still gets something useful out of you, but they claim they never actually see the raw data itself. This is genuinely better than the old way of doing things, and it is being sold, reasonably, as a privacy solution. It is a privacy architecture, which is not quite the same thing, because every architecture has failure modes and this one has several.

The updates your device sends back are not random. They carry traces of the data they were computed from. Clever enough analysis can sometimes work out whether a particular person's data was used, or even reconstruct fragments of it. The technique is newer than the problem, and the risks are still being mapped. In education, where schools might one day train shared models without pooling student records, this matters enormously. The promise is good. The guarantee is not absolute, and calling it absolute has become a marketing habit that obscures a genuine trade-off. Where in your life are you trusting that something is safe simply because you were told the data never left your device?

Last week we said the version of you a system met first is often not the version still using it. This week we look at one of the most popular new privacy approaches and at where its limits actually sit.