I am trying to create a library for sparse training. So it would need fast read/write of not only the machine learning model weights, but their optimizer momentum values as well. At the moment I would only like to experiment with values in the billions.
Each parameter would be a float32 value.
I think I can just use a single column, but there may be some instances where multiple columns would be helpful. For example, each column would represent a different layer in the machine learning architecture.
Each update step in the training would be looking up and writing values do the data store, so I am looking for the fastest database for my situation. Having a database at all would already significantly reduce ram memory, so that that is not a concern for me.