0
$\begingroup$

I am trying to apply a PSO algorithm to train a neural network applied to "User Identification From Walking Activity Data Set" problem. Can be found here.

So i extrated the dataset, and calculated average value and standard deviation between aceleration x,y and z at every 5 seconds for each person that would be the input of my neural network. After this i try to optimize the weights and bias of the neural network with Pyswarms functions, but i get an error which i dont know where it is coming from.

The Code :

import numpy as np
import statistics as st
import pyswarms as ps
import matplotlib.pyplot as plt
from sklearn.neural_network import MLPClassifier

#EXTRACTING THE DATASET AND CREATING INPUTS AND TARGETS/LABELS
def load_input():
  data = np.empty((0,3),float)
  for x in range(1,23):
        fich = np.loadtxt("%d.csv" %x,delimiter=",");
        aux = []
        value = 5.0;#5 seconds for each person
        for y in range(0,len(fich)):
            if fich[y][0] <= value :
                aux.append(fich[y,1])
                aux.append(fich[y,2])
                aux.append(fich[y,3])
            elif fich[y,0] > value or y == len(fich): 
                data = np.append(data,np.asfarray([[np.mean(aux),np.std(aux),x-1]]), axis =0);
                y = y-1;
                aux[:] = []
                value = value + 5;
  return data


#OPTIMIZING WEIGHTS AND BIAS
def forward_prop(params):
 n_inputs = 2
 n_hidden = 20
 n_classes = 22

 W1 = params[0:40].reshape((n_inputs,n_hidden))
 b1 = params[40:60].reshape((n_hidden,))
 W2 = params[60:500].reshape((n_hidden,n_classes))
 b2 = params[500:522].reshape((n_classes,))

 z1 = X.dot(W1) + b1 # Pre-activation in Layer 1
 a1 = np.tanh(z1) # Activation in Layer 1
 z2 = a1.dot(W2) + b2 # Pre-activation in Layer 2
 logits = z2 # Logits for Layer 2

 exp_scores = np.exp(logits)
 probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)
 # Compute for the negative log likelihood
 N = 923 # Number of samples
 corect_logprobs = -np.log(probs[range(N), Y])
 loss = np.sum(corect_logprobs) / N
 return loss


def f(x):
 """Higher-level method to do forward_prop in the
 whole swarm.
 Inputs
 ------
 x: numpy.ndarray of shape (n_particles, dimensions)
 The swarm that will perform the search
 Returns
 -------
 numpy.ndarray of shape (n_particles, )
 The computed loss for each particle
 """
 n_particles = x.shape[0]
 j = [forward_prop(x[i]) for i in range(n_particles)]
 return np.array(j)


##### MAIN FUNCTION #####   
data = load_input();
X = data[:,[0,1]];
Y = data[:,2].astype(int)
# Initialize swarm
options = {'c1': 0.5, 'c2': 0.3, 'w':0.9}
# Call instance of PSO
dimensions = (2 * 20) + (20 * 22) + 20 + 22
optimizer = ps.single.GlobalBestPSO(n_particles=100, 
dimensions=dimensions,options=options)
# Perform optimization
cost, pos = optimizer.optimize(f, print_step=100, iters=1000, verbose=3)

Examples of Inputs and target matrixes : enter image description here

$\endgroup$
1
$\begingroup$

This should be on stackoverflow, with a precise description of the error (line, traceback...), but the error seems to be here:

np.empty((0,3),float)

And then not appending the data as expected.

If you are doing this, use a list, and then at the end, transform your data in a numpy array:

data = []

for...
    data.append(new array)
data = np.array(data) # add the Fortran flags if you need to

There would be just one memory copy instead of O(n-squared) with np.append.

Also the shape for X or Y (?) seems to be 2D ((100,522)), where as you only have 1D. Check your data (which we don't have, so impossible for me to tell you exactly where to look for) so that you can ensure that the dimensions are correct.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Hi. Thanks for answering. But after i make "data = np.array(data)" it only gives me the 1st element of the list. Is there a way to convert all the elements ? $\endgroup$ – Kelve Dec 20 '18 at 17:50
  • $\begingroup$ This converts the full list of data to a numpy array. If there is only one element, then it means that there was only only one entry in data. There will be as many elements as you have calls to data.append. $\endgroup$ – Matthieu Brucher Dec 20 '18 at 17:53
  • $\begingroup$ Btw, i was not having any error with this before. I tried line by line, and the error happens only after this line : <<cost, pos = optimizer.optimize(f, print_step=100, iters=1000, verbose=3)>> $\endgroup$ – Kelve Dec 20 '18 at 17:55
  • $\begingroup$ Yes, but you didn't check what your data was and that it was consistent with what you were expecting. Debug your data preprocessing, this is probably where you have the error with the wrong size. $\endgroup$ – Matthieu Brucher Dec 20 '18 at 18:02
  • $\begingroup$ Could i show you the data ? Its because i really dont think that the problem is with X or Y. They are inputs and targets respectively. ((100,522)) shape has to do with the particles x dimensions matrix created by PSO for optimization of weights and bias. $\endgroup$ – Kelve Dec 20 '18 at 18:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.