# Linear regression incorrect prediction using Matlab

In the plot below the red crossed line is the actual curve and the crossed blue line is the predicted curve. I am using least squares for linear prediction. I have used 1:79 examples in training and the remaining for testing. The test data points are never seen during training. What is my mistake? Why am I getting such a weird prediction? I want to see the sine curve as the predicted output which should be very close to the original data.

%generate some data
x=linspace(0,2*pi,100)';
y=sin(x); %response

X=x;
y=y;
% Convert matrix values to double
X = double(X(1:79));
y = double(y(1:79));

% Plot data
plot(X, y, 'rx', 'MarkerSize', 10);

m = length(y);
% Add ones column
X = [ones(m, 1) X];

% Gradient Descent with Normal Equation
theta = (pinv(X'*X))*X'*y

% Predict  from 80 till last sample
test_samples = x(80:end);
test_samples_val = [ones(length(test_samples),1) test_samples];

% Calculate predicted value
pred_value = test_samples_val * theta;

X = vertcat(X, test_samples_val);
regressionline = X*theta;

% Plot predicted value with blue cross
plot(test_samples, pred_value, 'bx', 'MarkerSize', 10);


You are calculating a linear regression in a nonlinear environment, which means you will need nonlinear exogenous variables.

You are calculating $$Y=\beta_0+\beta_1x$$ (a linear equation) when you need at least a polynominal equation. The following code calculated this equation: $$Y=\beta_0+\beta_1x+\beta_2x^2+\beta_3x^3$$ (maybe not the best equation but a working one)

%generate some data
x=linspace(0,2*pi,100)';
y=sin(x); %response

hold on
X=x;
y=y;
% Convert matrix values to double
X = double(X(1:79));
y = double(y(1:79));

% Plot data
plot(X, y, 'rx', 'MarkerSize', 10);

m = length(y);
% Add ones column
X = [ones(m, 1) X X.^2 X.^3];

% Gradient Descent with Normal Equation
theta = (pinv(X'*X))*X'*y;

% Predict  from 80 till last sample
test_samples = x(80:end);
test_samples_val = [ones(length(test_samples),1) test_samples test_samples.^2 ...
test_samples.^3];

% Calculate predicted value
pred_value = test_samples_val * theta;

X = vertcat(X, test_samples_val);
regressionline = X*theta;

% Plot predicted value with blue cross
plot(test_samples, pred_value, 'bx', 'MarkerSize', 10);


You want to model a non-linear function by a line that is really not suppose to work.

## Exception

Well, for small ranges you can indeed approximate a sine wave by a line, for example we know that

$$sin(\theta) \approx \theta$$ if $$\theta \approx 0$$

This linerization is common when modeling physical systems.

## Comments of the experiment

You model was trainned using the range $$[0,4.85]$$ (approximately), using linearly spaced values, but tested in the range $$[4.85,6.14]$$.

If you evaluate the derivative in those ranges you will see that there are 2 sign flips, most of the interval happens with negative derivative so is very likely for a regression algorithm to fit it with a negative-signed set of weights.

I would advise you use a different model for your problem or use a mode suitable interval.

## Suggestion

1 - To overcome your interval problem, you could reduce the range to $$[1.5,4.8]$$ radians or any interval without zero-crossings in the derivative of your function.

2 - You can shuffle your data or create random points within a (realistic) interval for this application