SKLearn has this broad claim in its FAQs:

Outside of neural networks, GPUs don’t play a large role in machine learning today, and much larger gains in speed can often be achieved by a careful choice of algorithms.

Anyone care to add color for or against this claim?

  • $\begingroup$ Could you clarify which algorithm do you think would benefit from the GPU? The linear models ones? $\endgroup$ – Ricardo Cruz Jan 3 at 20:01
  • $\begingroup$ They are excluding anything with neural networks in the comment, so I am looking for an inventory of machine learning algorithms that are not neural networks and are uniquely resistant to being parallelized on a GPU. $\endgroup$ – Lars Ericson Jan 4 at 4:21
  • $\begingroup$ Ah, okay, I thought you had models in mind. Other than neural networks, I don't know what else would benefit from vectorization. The linear models would benefit a little bit, but very little. Maybe Euclidean distances computed by kNN and k-means would benefit a little too. $\endgroup$ – Ricardo Cruz Jan 4 at 23:12

GPU doesn't inherently fit naturally into all machine learning algorithms. A natural contender is one that inherently takes a myriad of matrix multiplication. This makes sense since graphic processors inherently were design for this. However, for an algorithm like a Random Forest this may not be so important. Also there exist a cost to transfer information to a GPU. Any algorithm that is O(n) should not be computed on a GPU because it takes O(n) to communicate the data. There's a few other issues that GPU present dealing with RAM and Threading, each of which often render making a GPU variant of a project more of a hassle than it's worth.

Furthermore, adding GPU to the sklearn framework inherently adds a hardware dependency and complexity that seems needless for shallow algorithms. Odds are, if you're needing access to your GPU, you are dealing with a neural network, which has it's own unique architecture challenges. I think it makes far more sense to separate deep learning into it's own module (look at how huge tensorflow/pytorch/etc) project are) than force Sklearn to add hardware dependencies for marginal computational gains.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.