1
$\begingroup$

To train a machine learning model, the computer often needs more processing power. In this case, a powerful CPU is needed, since it is a large data set, it needs more memory, so rather than a CPU, GPU is the solution.

Do we need to decide which processor to use before we proceed? For example, will a 30000 sample data set need this much processing power?

Thanks in advance.

If any part of this question is not clear, please comment it.

$\endgroup$
2
$\begingroup$

Dataset (number of samples, number of features) is one variable. Algo/model complexity is another.

For example, linear regression will be much faster as compared to 4 layer neural network (and will require much lesser compute power).

So, before deciding need for compute powers, you can :

  1. Try few models with hardware (or AWS instances) you already have
  2. Estimate need for better hardware (CPU / GPU) based on the performance and what is the bottleneck

For very large data sets (say 10 TB+), I/O can become the bottleneck.

$\endgroup$
1
  • $\begingroup$ Got it, thank u. @ShamitVerma $\endgroup$ – PL_Pathum Mar 24 '19 at 7:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.