# Different runs of feature selection algorithm giving different set of selected feature. How to choose the best set among them?

I am using the forward feature selection algorithm from MATLAB. The code is as follows:

 X=combine_6_non;
y=target;
c = cvpartition(y,'k',10);
opts = statset('display','iter');
[fs,history] = sequentialfs(@fun,X,y,'cv',c,'options',opts)


The function fun is as follows:

function err = fun(XT,yT,Xt,yt)
model = svmtrain(XT,yT, 'Kernel_Function', 'rbf', 'boxconstraint', 1);
err = sum(svmclassify(model, Xt) ~= yt);
end


Now for different runs of the selection algorithm I am getting different feature sets. How should I zero down to the best feature set?

• Is it possible to set a seed or random_state? If yes, then that should solve the issue :)
– Dawny33
Aug 5 '16 at 8:18
• @Dawny33 what are these things? Aug 5 '16 at 8:23
• Some algorithms are (partly) stochastic, thus randomly choosing value to explore the space of solutions. However, computers cannot generate random numbers but pseudo-random numbers that are simulated based on an initial seed. If you fix it, you should get the same results each time. There is no randomness in the SVM, thus I would guess it is in the splitting part Aug 5 '16 at 8:38
• @Dawny33 so the partitioning is leading to this right? Aug 5 '16 at 8:40
• @Rishika Yes. So, pl read up on how to set a random_state or a seed in Matlab, similar to Python :)
– Dawny33
Aug 5 '16 at 8:47

You can set a seed or a random state for the splitting process. This helps in generating a fixed random number everytime, which helps you get the same data everytime you do the CV split.
This can be done by set_seed(...) in R, and adding the random_state = ... in relevant functions parameters in Python. So, please add the relevant parameter for Matlab, and your CV would do good.