What is the difference between adding more LSTM layers and just increasing the units of existing layers? Which one is preferred and in which situation?
When you add layers you are increasing the depth of the neural network. If you add more units to the existing layers, you are increasing the width.
In terms of hyperparameters selection, the best I can recommend is to try both and see which one gives you the best performance. Take into account considerations like over fitting, which may happen specially when you increase the complexity of a model.
You can relate this intuition for networks with LSTMs:
If you add more units, so intuitively you are adding more nodes into the hidden layer. This will allow the model to add "wide" varieties of implicit relationships ( maybe more than necessary) among the inputs it is getting, as derived info. This might help or not help for a model to improve accuracy.
If you add more layers. Then the model will not only hold relationships among inputs , but also among the derived information. Thus increasing the "depth" of relationships and combinations for the model.
To answer which approach is better? It depends on your data set and your modelling approach. Hyper-Parameter tuning is a way to push your accuracy limit and it might help you here. Also try a grid search to find if adding layers help you or adding nodes.
Hope you get me! Cheers!