Currently, to train a model, you need to collect a huge blob of data.
Are there feasible concepts of decentralized machine learning? Like, feed the model somehow from isolated data sources, or merge smaller models?
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.Sign up to join this community
Using federated learning the model training does not require the whole data to be present at a centralized server instead the model training is decentralized such that the model gets trained collectively on individual nodes and then summarizes the changes as a small focused update. Only this update to the model is sent to the cloud, using encrypted communication, where it is immediately averaged with other user updates to improve the shared model. All the training data remains on your device, and no individual updates are stored in the cloud.
PySyft is a library hooked into Pytorch that provides federated learning capabilities.
PySyft github repository link- https://github.com/OpenMined/PySyft
Demo code for federated learning using PySyft library- https://colab.research.google.com/drive/1F3ALlA3ogfeeVXuwQwVoX4PimzTDJhPy#scrollTo=PTCvX6H9JDCt
It sounds like you are asking about machine learning applied to Edge computing and IoT systems.
In industrial settings, on-board computing capability has been A fact and requirement for, well, at least over a decade already. Companies who specialize in designing and implementing value adding IoT systems to such settings (e.g networked flow meters in oil and gas fields, networked sensors in chemical manufacturing facilities, etc) have been taking data processed through logic within the on-board computer and transmitting that data (usually intermittently) to the cloud for processing, dashboards and similar. From what I understand though, handling machine learning on-board is a much newer concept but is being done now. As the amount of data being handled by each system and the speed required to ensure synchronized operation of IoT systems increases rapidly, this sort of "Machine learning computing on the edge" capability is getting a lot of attention.
Here is an example of a solution released by Google in 2018:
I am aware that this doesn't fully meet the requirement of being trained at the edge, it does seem to be an example of the integrated approach you are hinting. Also, IoT technology already feeds data to cloud based Machine learning training efforts. It seems that IoT technology has been compiling and uploading database entries on the cloud for use in training models for some time now.