Why choose TensorFlow?

Question

I have noticed that most of the deep learning developers use TensorFlow. So why choose TensorFlow? What is the advantage of TensorFlow over Theano and CNTK?

Answer 1

I will outline some points about the libraries and point you to some good comparisons that I have read. The GitHub star counts are just another reference point to help compare popularity. While I don't condone simply following the masses, those stars do help you see what a lot of other people think about the frameworks.

Tensorflow

• is very well documented
• scales up into production, being able to use many GPUs or Google TPUs
• allows flexible creation of DL architectures, using basic building blocks
• is backed by Google
• has a very large following - just look at how many stars it has received over on GitHub!

For reference, the awesome NumPy only has 7059 stars!

CNTK

• is gaining popularity (they started a bit later than Google)
• has very efficient implementations for specific use cases, such as the usage of CNNs and LSTMs in the text domain
• is nicely linked to other Micosoft products, like their Azure based toolkits
• scales well in production
• is backed by Microsoft
• has a good following on Github:

Theano

While still open source (and so able to be further developed by the community), the team behind Theano announced that they will no longer actively develop it. This means it will now likely fall even further behind other leading frameworks, and new functionality coming from ongoing research are not likely to make it into the library.

We can see that many people admire Theano, but given that it is basically the oldest DL framework, the star count tells a tale:

Additional points

Have a look for some overviews and comparisons of the deep learning frameworks in general to get a better understanding of when you might use one over another. Here are some I found useful:

1. Comparing top deep learning frameworks
2. Battle of the deep learning frameworks
3. Choosing a machine learning framework - has a good conclusion

I would suggest reading about the difference between static and dynamic computational graphs. Tensorflow e.g. builds and compiles a static graph of your model and then pushes data through. PyTorch on the other hand allows you to create your model dynamically, giving more freedom during the development of new architectures.

A common workflow would be to develop and do research with PyTorch, then try to write the final production code in Tensorflow for deployment.

One last point - you should be aware of the library Keras. This is a wrapper around the base libraries, such as Tensorflow, Theano and CNTK - maybe more in the future). It is highger-level and easier to use that the others, but behind the curtains it is really just using one of the libraries you are asking about. It is just change on flag in a config file to swap between them!

Just for completeness, the Keras GitHub star banner:

[All gitHub star counts as of April 2018]

---

I realise some of these points may become outdated and untrue over time. In which case, feel free to leave a comment and I can revise my post.

Answer 2

Here is some, probably exaggerated criticism of TensorFlow:

Tensorflow sucks by Nico Jimenez, 8 Oct 2017

The article was discussed on Hacker News. I highlight some quotes that I find interesting to consider. First, about its technical content.

Despite its shortcomings, I share the same vision as this article. Here are my reasons:

• Tensorflow has a way too large API surface area: parsing command lines arguments handling, unit test runners, logging, help formatting strings... most of those are not as good as available counterparts in python.
• The C++ and Go versions are radically different from the Python version. Limited code reuse, different APIs, not maintained or documented with the same attention.
• The technical debt in the source code is huge. For instance, There are 3 redundant implementations in the source code of a safe division (_safe_div), with slightly different interfaces (sometimes with default params, sometimes not). It's technical debt.

In every way, it reminds me of Angular.io project. A failed promise to be true multi-language, failing to use the expressiveness of python, with a super large API that tries to do things we didn't ask it to do and a lack of a general sounding architecture. (by batmansmk)

Some speculation about the philosophy and politics behind TensorFlow:

I think the author raises a good point about Google envy. TensorFlow is not the most intuitive or flexible library out there, and it is very over-engineered if you're not doing large-scale distributed training. The main reason why everyone talks it up so much is because Google heavily marketed it from the outset, and everyone automatically assumes Google == Virtuoso Software Design because they couldn't make it through the interview. Really it's just modern enterprise software which has five different ways to implement batch norm that they push on the community so they don't have to train new hires on how to use it. (by zo7)

Or maybe it is built by a company that is doing large-scale distributed training, and they open sourced it not to cater to every need, but to help others trying to do the same thing they are. Companies are under no obligation to make sure their open source is well suited for others use cases. (by dsl)

That was kinda my point, it's not the be-all deep learning library because they made it for their own use case, but its towering popularity (as in 10x the number of stars of other popular libraries) is not genuine.

Also I highly doubt that the main reason Google open sourced it was to be charitable. (by zo7)

Answer 3

Reasons to chose Tensorflow (IMHO):

• Large community of users: for every problem that you run into there are 100s of solutions and sample codes.
• Keras: it makes it easy to create some advanced models on top of Tensorflow.
• Support: everyone knows that Tesnorflow is backed by Google and they regularly update that. That is true for some other frameworks too, except Theano; Mila (the lab that created Theano) stopped this project.
• Tensorboard: it allows you to easily visualize the metrics of your model and is straight forward to work with.

Given all that, I really like PyTorch too. It is easy and natural to work with, it is backed by some big names, and most importantly, if you have worked with Matlab or numpy, you feel more comfortable using it, compared to Tensorflow for example.

Answer 4

Tensorflow has much more flexibility to do the things that you want. If you just want to train a standard CNN, then perhaps Theano will do the Job. If you want to do more funny stuff, like creating a new architecture, using a new learning rate schedule or test time augmentation, Theano is too much restrictive. I have also heard that Tensorflow is a bit more performant so it is better to use in production.