Linear Algebra tells us that N linearly independent vectors spans all of N dimension space. In a regression setting this translates into the fact that if you have N observations and N features per observation then your regression model has a fair chance of achieving a 100 % accuracy on the NxN training data. And your chances are even better if the NxN feature set is mostly composed of noise because N randomly generated N-dimension vectors are linearly independent.
The high accuracy on the train set is because the model has overfit on the random noise in the data. Such models almost never generalize well on an out of sample test set.
So, what's happening in polynomial regression is that as you add newer features, they may add more noise (potentially useless info) but very capable of being used by the model to explain the variance in the train set but never on the test set.
That's why it is the ideally choice because it allows you to easily add newer features from existing ones and demonstrate overfitting on a train set.