I have a large number of pictures that I would like to use LDA on. However, it requires too much memory, so I was wondering if it would be possible to make the learning incremental, using a sklearn implementation, or maybe another library?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
The scikit-learn library does not support incremental learning. The scikit-multiflow library does support incremental learning. However, scikit-multiflow does not have a Linear Discriminant Analysis (LDA) implementation.
answered Aug 28, 2020 at 11:59
$\begingroup$
$\endgroup$
For images the best would likely be to implement DeepLDA, from the paper Deep Linear Discriminant Analysis by Matthias Dorfer et.al (2015). Several implementations in PyTorch can be found on Github, for example liyu10000/DeepLDA. In PyTorch you would implement the learning loop yourself, so it should be possible to do this with mini-batches.
Another alternative would be to use a pretrained neural network for images to extract feature-vectors. Usually, it is possible to reduce output dimensionality to 128 dimensions (or lower). This greatly reduces memory usage. Depending on the number of images, it might become realistic to hold all of them in memory, and use a standard LDA implementation such as the one in sciki-learn.
Subsampling the data might also be useful, and one can analyze the performance sensitivity wrt number of samples (learning curve) to see whether one might be leaving any performance on the table.
