Federated Learning

Borgs and Federated Learning

Federated Learning

We are the Borg! Resistance is futile

Federated Learning is a machine-learning model with data coming from a large number of participants, each with a small part of information. All the data is aggregated to be computed, evaluated and classified. The machine improves its “understanding” by assimilation data and gradually summarizing all the data accumulated. As all the information is analyzed, a new global model is created under the coordination of a central core or server. Then, the data gathered is sent back and shared with other updates. As a result of the process of this cooperation between participating clients, (known as federation) a new optimized model of the machine behavior is developed.

Federated learning doesn’t need to store data coming from the clients, thus reduces privacy risks. Besides it allows for bigger efficiency of the system. Indeed, the machine-learning model remembers which information the user searches for, what they click on and in what order. All of this is used by the machine in order to improve suggestions.

Today, Google is testing Federated Learning in Gboard, a keyboard designed for the Android handsets, as a continuous way to improve its interaction with millions of users. The individual updates coming from every user is evaluated in the Google cloud. The process is complex, Google is confident that this approach will allow to refine the interaction between the user and the machine. As to the users’ privacy, the decryption starts and is averaged only when thousands of updates have been accounted for. The data is never stored in the Google cloud.

Federated Learning is supposed not only to provide an update to the shared model, allowing for improvement of its reaction; but the improved model can be used immediately, powering experiences personalized by the way the users acts.

The Federated Learning approach can and supposedly will improve the machine-learning model.

Indeed, millions of participating devices have access to a wealth of data, and the process will greatly improve analysis of text entries, speech recognition or selection, and promotion of really convenient photos for the user.

However, Google’s approach leaves the training data on the participating devices. The process of learning is based on decentralized and locally-computed training data.

Briefly, the Federated Learning works this way: the user’s device downloads the current model of the AI model. It improves its behavior as this model learns from data on the user’s phone. After the model assimilates the changes, it summarizes them as a focused update. This insignificant update is sent to the Google cloud, using encrypted communication. Once in the cloud, the update is averaged with other user updates. As a result, the shared model changes its approach. However, all the training data remains on the user’s device, and no personal data is stored in the Google cloud.

According to Google’s engineers, communication costs are the principal constraint in Google’s activities, and a reduction in required communication will be substantial.

Everyone agrees that Google has built one of the most secure cloud infrastructures in the world for processing all the data it receives. And the approach by Federated Learning as a new model for training the system from user interaction with mobile devices is really promising. This method goes beyond the use of local models that can make predictions on mobile devices, such as the Mobile Vision API and On-Device Smart Reply. In fact, it brings model training to the device as well.

Thought the Borg civilization was thousands of light-years away? Wrong answer! They are coming! The final battle begins and it will be lost. Resistance is futile.

PS – Apple also announced that it would use a deep learning technology called long short-term memory (LSTM) to help its Quicktype keyboard offer more intelligent options during conversations. But that’s another story!