0
$\begingroup$

I am trying to implement a simple neural network with Python which will classify the Iris dataset flowers. I would like to keep the code as simple as possible and not use any ready-made libraries. I read up on it quite a bit but what I struggle to understand is how to properly map the activation function to the flower type and how to adjust the neuron weights. My understanding is that for three neurons (each for a different Iris type), each with 4 weights we have to multiply each input by each weight, so we get three results per each neuron. Then we apply the activation function and choose the greatest result, which will correspond to the Neuron's flower type. Next we update the actual results array and calculate the error by subtracting the obtained type from the actual expected type. The weights should then be updated by adding this error value and the corresponding input value. However I can't make it work. I only get values of one type and the weights become absurdly large. I will be grateful for any suggestions.

import numpy as np
import pandas as pd

# get the values from CSV as attributes, flower type (last column) presented below as 2d array for programming convenience
trainingDataSet = pd.read_csv('iris_l.csv')
trainingData = trainingDataSet.iloc[:, :-1].values
expectedTrainingOutputData = np.array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,0, 0, 0, 0, 0, 0,
 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2,
 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])

actualTrainingOutputData = np.array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])

class NeuralNetwork(object):
  def __init__(self):
 # initialize weights from input to learning layer, and from learning layer to output layer - random to start with
   self.setosaw1 = -0.5
   self.setosaw2 = 0.2
   self.setosaw3 = -0.3
   self.setosaw4 = 0.6

   self.versicolorw1 = -1.1
   self.versicolorw2 = 0.3
   self.versicolorw3 = -0.7
   self.versicolorw4 = 0.5

   self.virginicaw1 = -0.4
   self.virginicaw2 = 0.9
   self.virginicaw3 = -1
   self.virginicaw4 = 0.3


  def applySigmoidActivationFunc(self, arg):
    return 1/(1 + np.exp(-arg))

  def doLearn(self):
     loopCounter = 0
     for i in range(500):
         print('iteration: ' +str(i))
         for entry in trainingData:
             print(entry)
             resultSetosa = entry[0] * self.setosaw1 + entry[1] * self.setosaw2 + entry[2] * self.setosaw2 + entry[3] * self.setosaw1
             resultVersiColor = entry[0] * self.versicolorw1 + entry[1] * self.versicolorw2 + entry[2] * self.versicolorw3 + entry[3] * self.versicolorw4
             resultVirginica = entry[0] * self.virginicaw1 + entry[1] * self.virginicaw2 + entry[2] * self.virginicaw3 + entry[3] * self.virginicaw4

             resultSetosa = self.applySigmoidActivationFunc(resultSetosa)
             resultVersiColor = self.applySigmoidActivationFunc(resultVersiColor)
             resultVirginica = self.applySigmoidActivationFunc(resultVirginica)

             winner = 0
             if (resultSetosa > resultVersiColor and resultSetosa > resultVirginica):
                 winner = 0
             elif (resultVersiColor > resultSetosa and resultVersiColor > resultVirginica):
                 winner = 1
             elif (resultVirginica > resultSetosa and resultVirginica > resultVersiColor):
                 winner = 2

             actualTrainingOutputData[loopCounter] = winner

             localError = (expectedTrainingOutputData[loopCounter] - actualTrainingOutputData[loopCounter])
             self.setosaw1 += localError * entry[0]
             self.setosaw2 += localError * entry[1]
             self.setosaw3 += localError * entry[2]
             self.setosaw4 += localError * entry[3]

             self.versicolorw1 += localError * entry[0]
             self.versicolorw2 += localError * entry[1]
             self.versicolorw3 += localError * entry[2]
             self.versicolorw4 += localError * entry[3]

             self.virginicaw1 += localError * entry[0]
             self.virginicaw2 += localError * entry[1]
             self.virginicaw3 += localError * entry[2]
             self.virginicaw4 += localError * entry[3]

             print('winner: ' + str(winner))
             loopCounter += 1
             if loopCounter == len(trainingData):
                 loopCounter = 0
         print('internal loop ended')


if __name__ == '__main__':
    NN = NeuralNetwork()
    NN.doLearn()
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.