As already mentioned, data leakage and having some of the same data in both the test and training sets can be problematic.
Other things that can go wrong:
the statistical properties of the target variable, which the model is trying to predict, change over time in unforeseen ways
This can happen even if you do everything right when training and is especially relevant if you're training on old data.
"Seeing the future"
Say you have some sort of time series, where current data and patterns may inform future data (like stock prices or customer behaviour). If you trained on data that is in the future relative to the test set, this doesn't reflect the real world (as you can't see the future in the real world) and the model test performance may be inaccurate.
The examples from the test set need to be after every example from the training set.
This is related to concept drift (and may or may not classify as that).
Optimising on the test set
If you're doing hyperparameter tuning, you should use a separate set (a cross-validation set) for that.
Optimising your hyperparameters based on your test set means you're basically training on your test set and then you can't really trust the performance of your model for the same reason you can't trust a model you tested on your training set.
I assume this may not be relevant based on the way the question was phrased, but it's important to mention nonetheless.
If you didn't take appropriate care when selecting your evaluation metric, you may have a model that looks good on paper, but would have quite a bad impact on the business if it actually got those same results in production.
There are many, many ways you can pick the wrong metric, but the general idea to picking the right metric is that you need to consider the real-world effects of various ways in which the model can be inaccurate (false positives and false negatives for binary classification) and what the side effects of this may be (e.g. rejecting a valid customer costs means they're much less likely to come back and may convince others to go elsewhere instead).
Don't just use some popular metric out of the box without fully understanding the implications of using that metric.
How can you mitigate the risks?
This is a large topic all by itself, but I can briefly address it here.
Apart from what I already mentioned above, you can do:
For example, you can start running the model in a few cities rather than rolling it out worldwide right away.
This limits the risk in case the model turns out to not perform so well in the real world.
This allows you to accurately evaluate a model in a way you may not otherwise be able to (e.g. if you reject a transaction flagged as fraud, you may not ever know whether it actually was fraud).
Appropriate monitoring of model performance
Keep an eye on how the model is performing in production using various metrics and make sure there isn't any unforeseen side effects.
Regular retraining of the model
This is most relevant for concept drift. Generally you would expect the model performance to decrease over time.