There are a few multiclass variants of Platt scaling. The easiest approach is as you have described; simply perform one Platt scaling on each class.
However, there are more sophisticated options--a very simple one to implement is training a standard logistic regression on the logits (the values before the softmax activation is applied). This has called matrix scaling and can overfit pretty easily, so only use this if you have a large calibration set. Alternatively, a fewer-parameter version called vector scaling is relatively simple to implement, where the weights matrix inside the logistic regression is restricted to be a diagonal matrix. Finally, a very simple option that has been shown to work well for neural networks is temperature scaling, where all logits are simply scaled by a single scalar parameter.
You can read more about these and their application to neural networks in Section 4.2 of "On Calibration of Modern Neural Networks" (2017) - available here