2
$\begingroup$

I recently came accross these products (Nvidia Jetson) and they are all tagged as "edge", so i think they are designed only for machine learning inference and not model training. They are quite interesting for their low power consumpion and price (eg: Jetson Nano) so i hope they are suitable also for model trainig. So i would ask if someone may clarify this aspect about the focus of the product.

$\endgroup$
3
$\begingroup$

Training a neural network involves a lot more computation than inferencing from a pre-trained one. During training, a neural network must go through both forward propagation and back propagation steps. The back propagation step is computationally expensive, since it requires many gradient computations and updates to weight variables.

During inference, the only computation needed is forward propagation, where the input is multiplied to the weights to produce a prediction. Boards like the NVIDIA Jetson are optimized for fast matrix multiplication, hence why they are effective during inference but not as much for training.

Another reason for the Jetson's popularity is its small form factor. Architects choose it because it can fit within the size and price requirements of the product while also offering decent performance. If you had unlimited budget and can fit a GTX Titan in your product for inferencing, you would do that, right? I worked in the autonomous vehicle sector for a while and that there is no way we could fit a machine with any full-size GPU in a car. There are no size limitations for training models, so naturally you would go after the machines with more powerful and efficient GPUs.

$\endgroup$
1
  • $\begingroup$ thanks for clarification, counting only cuda core made this Jetson look like an affordable way, for an hobbist like me,to get a reasonable fraction of a Titan computational power at a good price. Strange (also because no one had used in that way), so i suppose that this "edge" word mean more than i have understood. $\endgroup$
    – Skary
    Jul 31 '19 at 22:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.