Yes, different toolkits are suited for different purposes given they contain different algorithms. This is due to the lack of a general AI (an AI that is intelligent in all respects). Modern AI has different algorithms which are better suited for different tasks. For example a CNN framework is much better for images than a NN would be, and much better than encoders. However, a LSTM framework would be better suited to capture temporal dependencies in the data.
However, if two frameworks are using the same algorithms they should both be relatively the same. Differences will arise due to the underlining codes being different as can be seen between Keras and TensorFlow. Although, this should be limited.
Worry more about the algorithm you are choosing for your task than the framework. Coding up most machine learning algorithms is not that hard if you want to customize it.