As TF 2.0 looms and with it the certainty of having to rewrite or throw away most of my scripts, I am considering switching to pytorch. I initially liked TF for its low-level API — I think it is becoming a complete mess by trying to become everything at once. Also, the library is a hell to compile on Windows. Using a trained model from C++ on Windows has become virtually impossible because of that.
I have looked a bit at pytorch, I like that its API is simple and consistent, and Windows users are greeted with C++ librairies straight from the website. Frankly I feel they are flirting with me. I thought pytorch was a python wrapper around a Lua library but I have clearly been mistaken.
One of the strength (maybe the selling point to me) of TF is its input pipeline. Whether from the old
tf.queue days or the more recent
tf.data framework, TF always tried to propose an input pipeline that is able to preprocess and prefetch input data to device memory concurrently to training. It is all pretty well explained on this page.
This efficient input pipeline is important in practice to achieve a high utilization of GPUs, esp. when I/O operations and data augmentation is a bit heavy. Is there an equivalent in pytorch? I have not seen such an example in the tutorials. I saw a
Dataset class in the "Data Loading and Processing" tutorial but it looks like standard python, so I am assuming it cannot run concurrently to training.
If anyone with experience with both could weigh in on that topic, and also perhaps on other topics such as using a trained model from C++, I'd be glad to hear it.
I know I would be flamed on stackoverflow for asking this kind of broad question, I hope it's okay here.