1
$\begingroup$

I need to understand how the splitting of labels in the following code happened :

import keras
import librosa
import librosa.feature
import librosa.display
import glob
import numpy as np   
from keras.models import Sequential 
from keras.layers import Dense , Activation
from keras.utils.np_utils import to_categorical


def extract_features_song(f):
    y, _ = librosa.load(f)

    # get mfcc
    mfcc = librosa.feature.mfcc(y)

    # make value between 1 -1 
    mfcc /= np.amax(np.absolute(mfcc))
    return np.ndarray.flatten(mfcc)[:25000]

def extrac_features_and_labels():
    all_features = []
    all_labels = []

    genres = ['blues' , 'classical', 'country' , 'disco' , 'hiphop', 'jazz', 'metal' , 'pop', 'reggae', 'rock']
    for genre in genres:

        sound_files = glob.glob('genres/'+genre+'/*.au')


        print ('prcoessing %d songs in %s genre'% (len(sound_files), genre))

        for f in sound_files:
            features =extract_features_song(f)
            all_features.append(features)
            all_labels.append(genre)

    # one hot encoding
    label_uniq_ids , label_row_ids = np.unique(all_labels, return_inverse= True)
    label_row_ids = label_row_ids.astype(np.int32, copy= False)
    onehot_labels = to_categorical(label_row_ids, len(label_uniq_ids))

    return np.stack(all_features), onehot_labels 

features , labels = extrac_features_and_labels()



print (np.shape(features))
print (np.shape(labels))

training_split = 0.8

alldata = np.column_stack(features , labels)

np.random.shuffle(alldata)
splitidx = int(len(alldata))*training_split
train , test = alldata[:splitidx,:], alldata[splitidx:, :]

print (np.shape(train))
print (np.shape(test))


# the concerned part: begin
train_input = train [:,:-10]
train_labels = train [:,-10:]

test_input = test [:,:-10]
test_labels = test [:,-10:]
#the concerned part: end



print (np.shape(train_input))
print (np.shape(train_labels))

the output is as follows:

(1000, 25000)
(1000, 10)
(800, 25010)
(200, 25010)
(800, 25000)
(800, 10)

Now, when he - code instructor - stacked both arrays together and shuffled them, he can't be sure that the last ten elements are the labels, am I right? If so how did he do it like this, this would cause an error.

$\endgroup$

1 Answer 1

1
$\begingroup$

Nope, the last 10 elements are definitely the labels. np.column_stack appends the elements of the second array to the corresponding row of the first array.

A quick example (Python 3.5), random one-hot encoding snippet was copied from this answer on SO

>>> import numpy as np
>>> np.__version__
'1.15.0'
>>> rand_enc = np.random.randint(0,3,size=(10))
>>> labels = np.zeros((10,3))
>>> labels[np.arange(10), rand_enc] = 1
>>> labels
array([[0., 0., 1.],
       [0., 1., 0.],
       [1., 0., 0.],
       [1., 0., 0.],
       [1., 0., 0.],
       [1., 0., 0.],
       [0., 0., 1.],
       [0., 0., 1.],
       [0., 1., 0.],
       [1., 0., 0.]])
>>> features = np.random.rand(10,4)
>>> features
array([[0.51599703, 0.24758895, 0.70951355, 0.52074341],
       [0.64851074, 0.14481191, 0.88837244, 0.86901571],
       [0.36303914, 0.93464881, 0.21238598, 0.24998789],
       [0.74290604, 0.96704141, 0.16971294, 0.99628383],
       [0.20225753, 0.78450864, 0.42730556, 0.84727098],
       [0.95372573, 0.38092036, 0.73628447, 0.22120431],
       [0.10821134, 0.54467407, 0.00452555, 0.15295404],
       [0.7353348 , 0.26629375, 0.96241551, 0.84573258],
       [0.80659848, 0.34873381, 0.12219345, 0.46671669],
       [0.07837654, 0.82128673, 0.79523531, 0.10652154]])
>>> np.column_stack((features, labels))
array([[0.51599703, 0.24758895, 0.70951355, 0.52074341, 0.        ,
        0.        , 1.        ],
       [0.64851074, 0.14481191, 0.88837244, 0.86901571, 0.        ,
        1.        , 0.        ],
       [0.36303914, 0.93464881, 0.21238598, 0.24998789, 1.        ,
        0.        , 0.        ],
       [0.74290604, 0.96704141, 0.16971294, 0.99628383, 1.        ,
        0.        , 0.        ],
       [0.20225753, 0.78450864, 0.42730556, 0.84727098, 1.        ,
        0.        , 0.        ],
       [0.95372573, 0.38092036, 0.73628447, 0.22120431, 1.        ,
        0.        , 0.        ],
       [0.10821134, 0.54467407, 0.00452555, 0.15295404, 0.        ,
        0.        , 1.        ],
       [0.7353348 , 0.26629375, 0.96241551, 0.84573258, 0.        ,
        0.        , 1.        ],
       [0.80659848, 0.34873381, 0.12219345, 0.46671669, 0.        ,
        1.        , 0.        ],
       [0.07837654, 0.82128673, 0.79523531, 0.10652154, 1.        ,
        0.        , 0.        ]])

As you can see, the labels get appended at the end! The shuffling code only manipulates the rows, so the column ordering is preserved. You do need to pass np.column_stack a tuple of your NumPy arrays, but otherwise this gets the job done.

Hope that clears things up!

Edit: I don't know how the instructor got a size mismatch between the train_input and train_labels, from what I understand from the code you shared they should be (800,25000) and (800,10)

$\endgroup$
2
  • $\begingroup$ Alright I got it, but what do you mean " You do need to pass np.column_stack a tuple of your NumPy arrays, but otherwise this gets the job done." ? and yes you are right about the size, I will edit it $\endgroup$ Aug 8, 2018 at 6:21
  • 1
    $\begingroup$ alldata = np.column_stack(features , labels) needs to be alldata = np.column_stack((features , labels)). This packs the feature and label arrays into a single tuple so that np.column_stack will work. $\endgroup$
    – Tom M.
    Aug 8, 2018 at 12:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.