I'm studying the PCA algorithm and the theory behind it. I think I understood how does it work and the idea of dimension reduction of the data in order to find a new feature (component) that maximizes the variance of the data and minimizes the error.
My question is : in this algorithm , are the maximum variance and the minimum error reached at the same moment ?
In this example the magenta/black line is the solution of my PCA. So I find the 1 dimension vector that reduces my 2 dimensions dataset. I found this vector because the error (length of the red lines) is minimized and the variance (distance among the red projected points) is maximized.
So if I need to apply this algorithm , if I use either only the avg error minimization , will I reach the same result of the case in which I use the variance maximization principle ?