Newly discovered learning rule

Does anyone know how this algorithm performs the learning process for neural networks?

I've stumbled over this solution. It works, but I don't know how and why.

It's neuron-local and works without error or backpropagation.

void NeuralCluster::applyLearning(float learningRate){

//Correct each neuron random independently
vector<bool> alreadyDone;
for(int i = 0; i < weightsActive.size(); i++)alreadyDone.push_back(false);
for(int m = 0; m < weightsActive.size()-1; m++){

//Select a random neuron which is not already corrected
int i = -1;
bool done = false;
while(!done){
i = rand()%(weightsActive.size()-1);
if(alreadyDone[i] == false){
alreadyDone[i] = true;
done = true;
}
}

//Calculate the negative value of the in and output signal
float meanOutput = 0.0;
float meanInput = 0.0;
for(int j = 0; j < weightsActive.size()-1; j++){
float activationI = (EnergyFlowReal[i]);
float activationJ = (EnergyFlowReal[j]);

meanOutput += -(lastReal[i])*(weightsActive[j][i]);
meanInput +=  -(lastReal[j])*(weightsActive[i][j]);
}

//Do the correction on the weights accourding to the current activation on it
for(int j = 0; j < weightsActive.size()-1; j++){
float activationI = (EnergyFlowReal[i]);
float activationJ = (EnergyFlowReal[j]);
weightsActive[j][i] += (activationJ)*(((meanOutput))/(weightsActive.size()))*learningRate;
weightsActive[i][j] += (activationI)*(((meanInput))/(weightsActive.size()))*learningRate;
}

float activationI = (EnergyFlowReal[i]);
weightsActive[i][weightsActive.size()-1] += activationI*(((meanInput))/(weightsActive.size()))*0.01;
}

//Normalize the inputs and outputs of each neuron independently by random
for(int m = 0; m < weightsActive.size(); m++){

//Select a random neuron which is not already corrected
int i = -1;
bool done = false;
while(!done){
i = rand()%(weightsActive.size());
if(alreadyDone[i] == false){
alreadyDone[i] = true;
done = true;
}
}

//Calculate it's absolute weights at input and output
float absWeightsOut = 0.0;
float absWeightsIn = 0.0;
for(int j = 0; j < weightsActive.size(); j++){
float activationI = (EnergyFlowReal[i]);
float activationJ = (EnergyFlowReal[j]);
absWeightsOut += abs(weightsActive[j][i]);
absWeightsIn += abs(weightsActive[i][j]);
}

//Normalize the inputs and outputs of each neuron so their absoulte sum is one
for(int j = 0; j < weightsActive.size(); j++){
weightsActive[j][i] = ((weightsActive[j][i])/absWeightsOut)*weightsActive.size();
weightsActive[i][j] = ((weightsActive[i][j])/absWeightsIn)*weightsActive.size();

//Switch of some weights which are not nescessary
if((i >= 0)&& (j >= 0) && (i < numInputs)&& (j < weightsActive.size())){ weightsActive[i][j] = 0.0; }
}
}
}


Orginal source training in applyLearning() line 515

• Is there a paper in Arxiv or any other source? It is difficult to judge only with uncommented code and a summary. Jun 28, 2022 at 14:24
• Added the source with comments. Jun 28, 2022 at 15:44

2 Answers

I don't know how this algorithm performs, even with comments, because many functions are connected and use frequent loops, and they include also the output values.

I have to track every step, and it is difficult to imagine how the values are changing with time. It would be great to have a sketch or the mathematical logic behind this reasoning.

I recommend monitoring every step precisely with visualization tools and testing the algorithm with various scenarios in order to get a good understanding of what is going on.

Since you seem to be the author: is the term "CONDITIONAL NEURAL NETWORK" in the readme coined by you? Can you add some Jupiter notebooks where you show that you can learn standard 1) classification 2) regression tasks?. Also compare to https://stackoverflow.com/a/74966322/12229416 and see if you can find similarities

Based on your description it might be a variant of difference target propagation https://ar5iv.labs.arxiv.org/html/1412.7525