My professor said that the "holy grail of regression" is the function E(Y|X=x) i.e. the conditional expectation of Y on X. In practice, you'd take a small window of X and take the average value of Y for all observations that lie in the window.
The professor said that this is basically the best prediction you can make, but we don't usually do it because the curse of dimensionality reduces its effectiveness in when # of predictors is large. So it seems that local averaging (KNN regression is a type of this) is good with few predictors. However, in most articles and stats classes, I always see linear regression being used even in low dimensions. Why isn't local averaging used more often?