What is the significance of the square root in root-mean-square-error? Essentially, my question is: what is the difference between (rms error) and (rms error)$^2$?
It depends on what you are using the RMSE for. If you are merely trying to compare two models/estimators, then there is no significance to the square root. However, if you are trying to plot the error in terms of the same units as you made the measurements/estimates, then you need to take the square root to transform the squared units to the original units (much like variance vs standard deviation)
The square in RMSE is used because it always gives a positive value for error, so avoiding errors cancelling each other out, and affords greater weight to values further from the target function, so emphasising points for which the estimator is poor.
The square root is used to remove the effects of the squaring.
You could look at using the Mean Absolute Error ( MAE ) which does not have the distance weighting effect of the RMSE and just takes the average of the absolute value of the errors.
If the set that you are using the RMSE on is a linear space, a good reason to use the square root is that you turn the set into a metric space. The square root ensures the right scaling property. Essentially, the RMSE is equivalent to the Euclidean norm. As a benefit, it is possible to use results of the general theory of metric spaces.