0
$\begingroup$

Currently, to train a model, you need to collect a huge blob of data.

Are there feasible concepts of decentralized machine learning? Like, feed the model somehow from isolated data sources, or merge smaller models?

$\endgroup$
1
$\begingroup$

Using federated learning the model training does not require the whole data to be present at a centralized server instead the model training is decentralized such that the model gets trained collectively on individual nodes and then summarizes the changes as a small focused update. Only this update to the model is sent to the cloud, using encrypted communication, where it is immediately averaged with other user updates to improve the shared model. All the training data remains on your device, and no individual updates are stored in the cloud.

References- https://ai.googleblog.com/2017/04/federated-learning-collaborative.html

PySyft is a library hooked into Pytorch that provides federated learning capabilities.

PySyft github repository link- https://github.com/OpenMined/PySyft

Demo code for federated learning using PySyft library- https://colab.research.google.com/drive/1F3ALlA3ogfeeVXuwQwVoX4PimzTDJhPy#scrollTo=PTCvX6H9JDCt

$\endgroup$
0
$\begingroup$

It sounds like you are asking about machine learning applied to Edge computing and IoT systems.

In industrial settings, on-board computing capability has been A fact and requirement for, well, at least over a decade already. Companies who specialize in designing and implementing value adding IoT systems to such settings (e.g networked flow meters in oil and gas fields, networked sensors in chemical manufacturing facilities, etc) have been taking data processed through logic within the on-board computer and transmitting that data (usually intermittently) to the cloud for processing, dashboards and similar. From what I understand though, handling machine learning on-board is a much newer concept but is being done now. As the amount of data being handled by each system and the speed required to ensure synchronized operation of IoT systems increases rapidly, this sort of "Machine learning computing on the edge" capability is getting a lot of attention.

Here is an example of a solution released by Google in 2018:

https://cloud.google.com/edge-tpu/

I am aware that this doesn't fully meet the requirement of being trained at the edge, it does seem to be an example of the integrated approach you are hinting. Also, IoT technology already feeds data to cloud based Machine learning training efforts. It seems that IoT technology has been compiling and uploading database entries on the cloud for use in training models for some time now.

$\endgroup$
  • $\begingroup$ hi and welcome - thanks for your reply; from what I can read here, Edge TPU is an ML accelerator i.e. to infer = operate the ML model; my question is about the actual learning process i.e. which results in a new either updated model. $\endgroup$ – J. Doe Jan 23 at 16:23
  • $\begingroup$ I see. I found this organization which seems to be a specific example, of course anything involving an ICO tends to make me think twice. I suppose I misread the specific nature of your question. I have heard the concept discussed a lot lately, though. - medium.com/decentralized-machine-learning/… $\endgroup$ – MBach Jan 23 at 16:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.