Linear Algebra tells us that N linearly independent vectors spans all of N dimension space. In a regression setting this translates into the fact that if you have N observations and N features per observation then your regression model has a fair chance of achieving a 100 % accuracy on the NxN training data. And your chances are even better if the NxN ...
Another example you could use is the separation boundaries in decision trees classification problem. In the picture below, you can see that the training error continues to go down (lower the better) when max_depth increases, whereas the testing error is not as good. This is because the model has carved out specific pink areas (for 'x') as the separation ...
KNeighborsTransformer only gives you the indices of the nearest neighbors and the distances. You need to do more work to retrieve the points to fit your linear regression.
Here's a draft that appears to be working:
from sklearn.neighbors import NearestNeighbors
from sklearn.base import RegressorMixin, BaseEstimator, clone
from sklearn.linear_model import ...
Make sure you transformed back your predictions and actual values before calculating MAPE.
You can check which observations contributed the most to high MAPE.
MAPE is very sensitive to prediction errors at small actual values. Most likely worst performing observations ("from MAPE perspective") are those with small actual values.
Depending on the ...
One option would be to transform the y / target variable to be distributed more like a Gaussian, the most common transformations are log and quantile transformation. Gaussian transformation often increases the model fit statistics.
Taking the difference (ie speed1-speed2) as the target variable effectively dismisses any low-frequency variablitiy and targets only high-frequency variability, even noise.
One approach would be to bin the (highly-variable) target variable into fixed range bins and take the mid point (or any other fixed point) of each bin as the new target (stabilised) ...
I will give you an intuitive example to try out yourself - that will showcase the significance of interaction variables.
Let us say you had to predict a person's height by age and Gender:
X = Age, Gender
y = height
You have two modelling options:
Option 1 - y_hat = b0 + (b1 x age) + (b2 x gender)
So, you have age and height as independent predictors for ...
Not quite sure if this is what your question actually is. However, when you have a data generating process such as $y = 3 + 2x$ (with some zero-mean "gaussian noise"), your model could look like:
$$ y = \beta_0 + \beta_1 x + u,$$
and you will find the $\beta_0 = 3$ and $\beta_1 = 2$.
When you have a data generating process such as $y=3+2x+3x^2$ you ...