I've recently enrolled in the Coursera machine learning, and am working my way through making my own classifier for the Iris dataset problem using matlab. I'm training a classifier for each species (the one-vs-all method). The code runs smoothly without any errors, but the results are not quite what I was expecting. Here's my matlab code:

clear, clc

% loading the data
data = csvread('data.csv');

% extracting the features
X = data(:, (1:end-3));
X = [ones(size(X, 1), 1) X];

% computing the number of training examples (m) and features (n)
[m, n] = size(X);

% extracting the labels
y_setosa = data(:, end-2);
y_versicolor = data(:, end-1);
y_virginica = data(:, end);

% initializing the options for optimization algorithm
options = optimset('GradObj', 'on', 'MaxIter', 10);

% creating the initial theta vector
iniTheta = zeros(n, 1);

% tarining a classifier for each label
[theta_setosa, min_setosa] = fminunc(@(theta) costFunction(theta, m, X, y_setosa), iniTheta, options);
[theta_versicolor, min_versicolor] = fminunc(@(theta) costFunction(theta, m, X, y_versicolor), iniTheta, options);
[theta_virginica, min_virginica] = fminunc(@(theta) costFunction(theta, m, X, y_virginica), iniTheta, options);

Here's the code for my costFunction:

function [jValue, gradient] = costFunction(theta, m, X, Y)
    V = X*theta;
    jValue = (-1/m)*sum(Y.*log(sigmoid(V)) + (1-Y).*log(1-sigmoid(V)));
    gradient = (-1/m)*X'*(sigmoid(V)-Y);

And finally my sigmoid function:

function sigX = sigmoid(X)
    sigX = arrayfun(@(n) 1/(1+exp(-n)), X);

The resulting theta_setosa and theta_versicolor vectors are all zeros, and the costFunction has the same minumum for the both of them. the virginica classifier seems to be working fine however. Although for each fminunc I get the following message displayed

Local minimum possible.
fminunc stopped because it cannot decrease the objective function
along the current search direction.
<stopping criteria details>

1 Answer 1


The error stemmed apparently from the formula for the gradient: there should be no minus sign (-) at the start of it. The correct code for this would be:

function [jValue, gradient] = costFunction(theta, m, X, Y)
    V = X*theta;
    jValue = (-1/m)*sum(Y.*log(sigmoid(V)) + (1-Y).*log(1-sigmoid(V)));
    gradient = (1/m)*X'*(sigmoid(V)-Y);

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.