Neural networks with non-negative weights [closed]

Question

Could you tell me, are there any techniques for building neural networks with non-negative weights?

Answer 1

One possible method to approach neural networks with non-negative weights is using Feedforward neural network. We can construct the neural network using optimization techniques with the constraint of non-negative weights, as opposed to the normal back propagation method.

A modified Matlab example taken for here is as follows:

load iris_dataset

% Number of neurons
n = 4;

% Number of attributes and number of classifications
[n_attr, ~]  = size(irisInputs);
[n_class, ~] = size(irisTargets);

% Initialize neural network
net = feedforwardnet(n);

% Configure the neural network for this dataset
net = configure(net, irisInputs, irisTargets); %view(net);

fun = @(w) mse_test(w, net, irisInputs, irisTargets);

% Add 'Display' option to display result of iterations
ps_opts = psoptimset ( 'CompletePoll', 'off', 'Display', 'iter', 'MaxIter', 100); %, 'TimeLimit', 120 );

% There is n_attr attributes in dataset, and there are n neurons so there 
% are total of n_attr*n input weights (uniform weight)
initial_il_weights = ones(1, n_attr*n)/(n_attr*n);
% There are n bias values, one for each neuron (random)
initial_il_bias    = rand(1, n);
% There is n_class output, so there are total of n_class*n output weights 
% (uniform weight)
initial_ol_weights = ones(1, n_class*n)/(n_class*n);
% There are n_class bias values, one for each output neuron (random)
initial_ol_bias    = rand(1, n_class);
% starting values
starting_values = [initial_il_weights, initial_il_bias, ...
                   initial_ol_weights, initial_ol_bias];

% alter the patternsearch function with the appropriate constraints, in your case we would change it so that the lower bounds of the weights are zero 
[x, fval, flag, output] = patternsearch(fun, starting_values, [], [],[],[], zeros(size(starting_values)), 1e10, ps_opts); 

where the mse_test.m function is as follows:

function mse_calc = mse_test(x, net, inputs, targets)
net = setwb(net, x');
y = net(inputs);
[row col] = size(y);
mse_calc = sum(sum((y - targets).^2))/(row * col);
end

Answer 2

You can easily take an existing Multilayer Perceptron implementation and modify the backpropagation algorithm to prevent weights from becoming negative. Of course, you would also need to make sure you initialize weights with only non-negative values.

But you need to be aware that you are limiting the decision surfaces that the neural network can learn. So you should consider whether a neural network with only non-negative weights can provide an acceptable solution for your given data set.