Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
SKLearn has this broad claim in its FAQs:
Outside of neural networks, GPUs don’t play a large role in machine learning today, and much larger gains in speed can often be achieved by a careful choice of algorithms.
Anyone care to add color for or against this claim?
GPU doesn't inherently fit naturally into all machine learning algorithms. A natural contender is one that inherently takes a myriad of matrix multiplication. This makes sense since graphic processors inherently were design for this. However, for an algorithm like a Random Forest this may not be so important. Also there exist a cost to transfer information to a GPU. Any algorithm that is O(n) should not be computed on a GPU because it takes O(n) to communicate the data. There's a few other issues that GPU present dealing with RAM and Threading, each of which often render making a GPU variant of a project more of a hassle than it's worth.
Furthermore, adding GPU to the sklearn framework inherently adds a hardware dependency and complexity that seems needless for shallow algorithms. Odds are, if you're needing access to your GPU, you are dealing with a neural network, which has it's own unique architecture challenges. I think it makes far more sense to separate deep learning into it's own module (look at how huge tensorflow/pytorch/etc) project are) than force Sklearn to add hardware dependencies for marginal computational gains.
