0
$\begingroup$

In the plot below the red crossed line is the actual curve and the crossed blue line is the predicted curve. I am using least squares for linear prediction. I have used 1:79 examples in training and the remaining for testing. The test data points are never seen during training. What is my mistake? Why am I getting such a weird prediction? I want to see the sine curve as the predicted output which should be very close to the original data. im

%generate some data
x=linspace(0,2*pi,100)';
y=sin(x); %response

X=x;
y=y;
% Convert matrix values to double
X = double(X(1:79));
y = double(y(1:79));

% Plot data
plot(X, y, 'rx', 'MarkerSize', 10);

m = length(y);
% Add ones column
X = [ones(m, 1) X];

% Gradient Descent with Normal Equation
theta = (pinv(X'*X))*X'*y

% Predict  from 80 till last sample
test_samples = x(80:end);
test_samples_val = [ones(length(test_samples),1) test_samples];

% Calculate predicted value
pred_value = test_samples_val * theta;

X = vertcat(X, test_samples_val);
regressionline = X*theta;


% Plot predicted value with blue cross
plot(test_samples, pred_value, 'bx', 'MarkerSize', 10);
$\endgroup$
2
$\begingroup$

You are calculating a linear regression in a nonlinear environment, which means you will need nonlinear exogenous variables.

You are calculating $Y=\beta_0+\beta_1x$ (a linear equation) when you need at least a polynominal equation. The following code calculated this equation: $Y=\beta_0+\beta_1x+\beta_2x^2+\beta_3x^3$ (maybe not the best equation but a working one)

%generate some data
x=linspace(0,2*pi,100)';
y=sin(x); %response

hold on
X=x;
y=y;
% Convert matrix values to double
X = double(X(1:79));
y = double(y(1:79));

% Plot data
plot(X, y, 'rx', 'MarkerSize', 10);

m = length(y);
% Add ones column
X = [ones(m, 1) X X.^2 X.^3];

% Gradient Descent with Normal Equation
theta = (pinv(X'*X))*X'*y;

% Predict  from 80 till last sample
test_samples = x(80:end);
test_samples_val = [ones(length(test_samples),1) test_samples test_samples.^2 ...
test_samples.^3];

% Calculate predicted value
pred_value = test_samples_val * theta;

X = vertcat(X, test_samples_val);
regressionline = X*theta;


% Plot predicted value with blue cross
plot(test_samples, pred_value, 'bx', 'MarkerSize', 10);
$\endgroup$
1
$\begingroup$

Short answer

You want to model a non-linear function by a line that is really not suppose to work.


Exception

Well, for small ranges you can indeed approximate a sine wave by a line, for example we know that

$$ sin(\theta) \approx \theta$$ if $\theta \approx 0$

This linerization is common when modeling physical systems.


Comments of the experiment

You model was trainned using the range $[0,4.85]$ (approximately), using linearly spaced values, but tested in the range $[4.85,6.14]$.

If you evaluate the derivative in those ranges you will see that there are 2 sign flips, most of the interval happens with negative derivative so is very likely for a regression algorithm to fit it with a negative-signed set of weights.

I would advise you use a different model for your problem or use a mode suitable interval.


Suggestion

1 - To overcome your interval problem, you could reduce the range to $[1.5,4.8]$ radians or any interval without zero-crossings in the derivative of your function.

2 - You can shuffle your data or create random points within a (realistic) interval for this application

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.