1
$\begingroup$

Since these libraries can turn CPU arrays into GPU tensors, could you parallelize (and therefore accelerate) the calculations for a decision tree? I am considering making a decision tree class written in Tensorflow/Pytorch for a school project, but I want to be certain that it makes sense.

$\endgroup$

1 Answer 1

1
$\begingroup$

Not directly, mostly because the structure of a decision tree doesn't lend itself to GPU parallelisation, and is better suited to CPUs. Even the most established decision tree algorithms use CPUs.

  • GPUs can only perform a small subset of operations at high performance, the most important being matrix multiplication, which is the fundamental building block of a neural network.
  • Decision trees don't train by gradient descent, but an iterative process of partitioning the dataset. As such, they need access to the entire dataset in memory (rather than batches).
  • Decision trees are not easily differentiable without amendment.

The Deep Neural Decision Forest paper does address these issues in a simple and clever way:

https://www.microsoft.com/en-us/research/wp-content/uploads/2016/06/ICCV15_DeepNDF_main.pdf

This is interesting as it does allow trees to be learnt with gradient descent, but it's unlikely to be faster than existing CPU bound tree algorithms.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.