4
$\begingroup$

I'm looking at implementing some neural networks from research papers. However, I'm concerned about which framework I should use, because I'm uncertain if the networks form static computation graphs (making them suitable for TensorFlow/Keras) or if they are dynamic (making them more suitable for PyTorch).

What are some rules of thumb to determine if a network requires a dynamic computation graph?

$\endgroup$
  • $\begingroup$ Just use pyTorch if you have the option. Debugging Tensorflow's static graphs is no fun. $\endgroup$ – Emre Jun 13 '17 at 17:56
  • $\begingroup$ Unfortunately, a framework that I want to interact with is built on TensorFlow and I'm not sure if it's possible to interface PyTorch with TensorFlow. $\endgroup$ – Seanny123 Jun 14 '17 at 4:29
  • $\begingroup$ I think for almost all tasks you can either use static graph or dynamic graph, but if you can harness multiple machines it is better to employ the static graph framework. $\endgroup$ – Lerner Zhang Oct 19 '17 at 1:50
1
$\begingroup$

I believe current implementations of the best known libraries will allow you to implement any network you come across. So it boils down to the aspects of taste and practicality.

As far as deciding on a framework based on the network you want to create; if the network has branches or separate control flows (data transfer or logical) that are used conditionally during training, PyTorch is likely more useful. Otherwise, the best starting point is likely to use Keras - a high-level API over Tensorflow.

The (very) general rule of thumb is:

Use Pytorch for experimentation, and Tensorflow for production

Now the caveats to the above sweeping statement:

  1. Both libraries are making massive efforts to cover "the other" library's strengths:

The latest release of Pytorch (v1.0) offers several methods of making production ready code - so ways of compiling it to improve performance as well as the use of 16-bit floating point operations (as opposed to the standard 32-bit). They have also started rolling out a C++ API. These are all steps to make full-scale products for deployment. Tensorflow has introduced its eager API, for eager execution, meaning you can essentially build dynamic graphs, just as in PyTorch. The enables faster and more interactive prototyping.

  1. The experience you have while developing might be more important:

If you are a pure Python developer, PyTorch will feel more natural. If you have been a general software engineer in other statically types languages (such as C++ and Java), Tensorflow may feel more logical.

Summary

Overall, Tensorflow along with (the now built-in) Keras wrapper, is a more mature library with a larger community and longer history, meaning more support online. PyTorch has a higher level of excitemnet around it, as far as I can tell. It is used very actively in research and newer open-source projects/courses, which indicates it may become a dominant force.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.