Both PyTorch and Tensorflow Fold are deep learning frameworks meant to deal with situations where the input data has non-uniform length or dimensions (that is, situations where dynamic graphs are useful or needed).

I would like to know how they compare, in the sense of paradigms they rely on (e.g. dynamic batching) and their implications, things that can/cannot be implemented in each one, weaknesses/strengths, etc.

I intend to use this info to choose one of them to start exploring dynamic computation graphs, but I have no specific task in mind.

Note 1: other dynamic computation graph frameworks like DyNet or Chainer are also welcome in the comparison, but I'd like to focus on PyTorch and Tensorflow Fold because I think they are/will be the most used ones.

Note 2: I have found this hackernews thread on PyTorch with some sparse info, but not much.

Note 3: Another relevant hackernews thread, about Tensorflow Fold, that contains some info about how they compare.

Note 4: relevant Reddit thread.

Note 5: relevant bug in Tensorflow Fold's github that identifies an important limitation: impossibility to do conditional branching during evaluation.

Note 6: discussion on pytorch forum about variable length inputs in relation to the algorithms used (e.g. dynamic batching).

up vote 9 down vote accepted

There are a couple of good threads on Reddit right now (here and here).

I haven't used either of these frameworks, but from reading around and talking to users I gather that support for dynamic graphs in PyTorch is a 'top down design principle', whereas TensorFlow Fold is bolted on to the original Tensorflow framework, so if you're doing anything reasonably complicated with Tensorflow Fold you're probably going to end up doing a lot more hacking around than if you're using PyTorch.

  • 3
    That was pretty much my experience with trying to use Fold with convolutions over trees for the past several months. It's not mature enough to handle things like that yet. They suggest "work arounds" if you look at the closed issues on their repo. Switching to PyTorch because Fold is so inflexible - pun intended. – Soubriquet Jun 4 '17 at 14:28

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.