Newly discovered learning rule

Question

Does anyone know how this algorithm performs the learning process for neural networks?

I've stumbled over this solution. It works, but I don't know how and why.

It's neuron-local and works without error or backpropagation.

void NeuralCluster::applyLearning(float learningRate){

    //Correct each neuron random independently
    vector<bool> alreadyDone;
    for(int i = 0; i < weightsActive.size(); i++)alreadyDone.push_back(false);
    for(int m = 0; m < weightsActive.size()-1; m++){

        //Select a random neuron which is not already corrected
        int i = -1;
        bool done = false;
        while(!done){
                i = rand()%(weightsActive.size()-1);
                if(alreadyDone[i] == false){
                    alreadyDone[i] = true;
                    done = true;
                }
        }

        //Calculate the negative value of the in and output signal
        float meanOutput = 0.0;
        float meanInput = 0.0;
        for(int j = 0; j < weightsActive.size()-1; j++){
            float activationI = (EnergyFlowReal[i]);
            float activationJ = (EnergyFlowReal[j]);

            meanOutput += -(lastReal[i])*(weightsActive[j][i]);
            meanInput +=  -(lastReal[j])*(weightsActive[i][j]);
        }


        //Do the correction on the weights accourding to the current activation on it
        for(int j = 0; j < weightsActive.size()-1; j++){
            float activationI = (EnergyFlowReal[i]);
            float activationJ = (EnergyFlowReal[j]);
            weightsActive[j][i] += (activationJ)*(((meanOutput))/(weightsActive.size()))*learningRate;
            weightsActive[i][j] += (activationI)*(((meanInput))/(weightsActive.size()))*learningRate;
        }

        float activationI = (EnergyFlowReal[i]);
        weightsActive[i][weightsActive.size()-1] += activationI*(((meanInput))/(weightsActive.size()))*0.01;
    }

    //Normalize the inputs and outputs of each neuron independently by random
    for(int m = 0; m < weightsActive.size(); m++){

        //Select a random neuron which is not already corrected
        int i = -1;
        bool done = false;
        while(!done){
                i = rand()%(weightsActive.size());
                if(alreadyDone[i] == false){
                    alreadyDone[i] = true;
                    done = true;
                }
        }

        //Calculate it's absolute weights at input and output
        float absWeightsOut = 0.0;
        float absWeightsIn = 0.0;
        for(int j = 0; j < weightsActive.size(); j++){
            float activationI = (EnergyFlowReal[i]);
            float activationJ = (EnergyFlowReal[j]);
            absWeightsOut += abs(weightsActive[j][i]);
            absWeightsIn += abs(weightsActive[i][j]);
        }


        //Normalize the inputs and outputs of each neuron so their absoulte sum is one
        for(int j = 0; j < weightsActive.size(); j++){
            weightsActive[j][i] = ((weightsActive[j][i])/absWeightsOut)*weightsActive.size();
            weightsActive[i][j] = ((weightsActive[i][j])/absWeightsIn)*weightsActive.size();
            
            //Switch of some weights which are not nescessary
            if((i >= 0)&& (j >= 0) && (i < numInputs)&& (j < weightsActive.size())){ weightsActive[i][j] = 0.0; }
        }
    }
}

Orginal source training in applyLearning() line 515

Answer 1

I don't know how this algorithm performs, even with comments, because many functions are connected and use frequent loops, and they include also the output values.

I have to track every step, and it is difficult to imagine how the values are changing with time. It would be great to have a sketch or the mathematical logic behind this reasoning.

I recommend monitoring every step precisely with visualization tools and testing the algorithm with various scenarios in order to get a good understanding of what is going on.

Answer 2

Since you seem to be the author: is the term "CONDITIONAL NEURAL NETWORK" in the readme coined by you? Can you add some Jupiter notebooks where you show that you can learn standard 1) classification 2) regression tasks?. Also compare to https://stackoverflow.com/a/74966322/12229416 and see if you can find similarities

Based on your description it might be a variant of difference target propagation https://ar5iv.labs.arxiv.org/html/1412.7525