0
$\begingroup$

Am trying to implement this equation from the book Doing Data Science Straight Talk from the frontline, In chapter 6, page 161, equation below: Equation of beta

From what i can tell it is pretty much an enchanced version of Ridge Regression, i believe i have implemented it below but am not sure if this is correct:

load 'Y.mat';   % Loading input Y data, 100x1 vector

lambda = 0.001; % Tuning parameter lambda
Mu = 0.001;     % Hyperparameter 3
N_lags = 15;
percent = 80;   % Percentage for Y train

Lags = 1:N_lags;            % Lag interval
X = lagmatrix(Y, Lags);     % X value is just lagged values of our input Y
X(isnan(X)) = 0;            % Convert NaN to zeros

%% Getting beta 1
gram_matrix = X' * X;
moment_vector = X' * Y;
I = eye(N_lags);
offset_one = limit * lambda^2 * I;

beta_one = (gram_matrix + offset_one) \ moment_vector;

%% Getting beta 2
M_lags = 1:N_lags;
M = lagmatrix(ones(N_lags, 1), M_lags);  % Shift operator M
M(isnan(M)) = 0;    % Convert NaN to zeros

ddo = I - M;  % Discrete derivative operator
symmetric_matrix = ddo' * ddo;
offset_two = (limit * lambda^2 * I) + (limit * Mu^2 * symmetric_matrix);

beta_two = (gram_matrix + offset_two) \ moment_vector;

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%% Training
N_ones = round(limit * percent / 100);  % Number of elements to use for training
train_matrix = zeros(limit, 1);
train_matrix(1:N_ones, 1) = 1;

Y_train = train_matrix(randperm(limit));
Y_test = ~Y_train;

% train_percent = percent / 100;  % training percentage
% test_percent = 1 - (percent / 100);
% 
% rng('default'); % To avoid getting random classification accuracy every run
% cv = cvpartition(N_lags, 'HoldOut', test_percent);
% idx = cv.test;
% 
% Y_train = beta_two(~idx, :);
% Y_test = beta_two(idx, :);

%% Plotting coffefficients Beta 1 & 2 histogram
figure(1);

subplot(1, 2, 1);
histogram(beta_one);
ylabel({'Magnitude, N'});
xlabel({'Coefficients, \beta1'});

subplot(1, 2, 2);
histogram(beta_two);
ylabel({'Magnitude, N'});
xlabel({'Coefficients, \beta2'});

%% Picking hyperparameters

%% Evaluating predictions on holdout sample (Y_test)
Y_hat_one = X(Y_test, :) * beta_one;
Y_hat_two = X(Y_test, :) * beta_two;

%% Assessing prediction error
MSE_one = mean((Y(Y_test) - Y_hat_one).^2);
MSE_two = mean((Y(Y_test) - Y_hat_two).^2);

fprintf('--- Mean Squared Error in holdout sample (Y_test) ---\n');
fprintf('MSE of Beta 1: %f\n', MSE_one);
fprintf('MSE of Beta 2: %f\n', MSE_two);

the beta_two in my script represent the B2 in the equation from the picture and the beta_one is the equation from the book as well below: enter image description here

above the divider i made is where my questions below are, the rest is my attempt at validating and training.

My Questions

  1. Is my implementation of Beta 1 and 2 correct?
  2. If you notice the first image states that the symmetric matrix should produce 1's on the sub and supper diagonal, while 2's on the diagonal however that is not the case of what am getting, the symmetric matrix is just uses I the identity matrix and M which is as quoted from the book where M is the matrix that contains zeros everywhere except on the lower off-diagonals, where it contains 1’s., if you evaluate this equation with any length of M for testing you will see that it is not possible to get the result mentioned, am i doing something wrong here?
  3. The final piece to the puzzle is picking right values for Mu and lambda as they are the hyper-parameters of the model. is there a good way to go about this?

Thank you very much in advance for reading and helping!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.