Think of it in terms of navigating a landscape. The land you move across is created by your error function (that is, the realtion between your model and your data), and the way you move across the landscape is your training function. Especially in Neural Networks, falling in 'local minima' is a big problem. In fact, Neural Networks have been theoretially able to approximate any function for... 20 years? See Universal Approximation Theorem. But the problem has always been learning how to train a given network.
More specifically, in your case comaring MAE and MSE, the difference is in the 'square' part. The steepness of the 'fitness landscape' will be much steeper where error is greater with MSE compared to MAE. 1*1 = 1. i.e., squaring does nothing at unit-error. 10 * 10 = 100. Absolute error of 10 will become a squared error of 100. That is looking at one sample, and has huge implications around falling in local minima where the error is great.
Looking across the whole dataset, another huge implication is how the NN balances the error it sees. With MSE, samples with high error become samples with very high error. Thus, a NN trained with MSE will 'care more' about learning samples that are very wrong at the expense of small improvements to 'easier' samples.
