# Training a sound localization neural network

I am trying to train a neural network, to estimate the location (in degrees from 0 to 180) a sound is coming from.

I am using TensorFlow Keras in python to train the model.

The input data are two binaural cues, specifically the ILD (Interaural Level Difference) and the ITD (Interaural Time Difference), each vector, consisting of the two above described features, is of dimensions [1,71276]. I have a total of 2639 measurements, 10% of which are used as validation data, and another 10% as test data.

The output should be an angle in the range [0,180].

I have normalized the data in the range [-1, 1] and the best loss I've been able to achieve is MSE = 16.

The model that achieved the highest MSE is the following:

model = tf.keras.Sequential(([
tf.keras.layers.Input(shape=(71276,), name='input'),

tf.keras.layers.Dense(units=900,activation='relu', name='dense_1'),
tf.keras.layers.Dense(units=360,activation='relu', name='dense_2'),
tf.keras.layers.Dense(units=180,activation='relu', name='dense_3'),

tf.keras.layers.Dense(units=1,activation='linear', name='output')
]))

model.compile(loss='mse',
metrics=['mae'])

EPOCHS = 500
BATCH_SIZE = 32

callbacks = [
tf.keras.callbacks.EarlyStopping(monitor='val_loss', mode='min', min_delta=0.5, patience=100, verbose=1),
tf.keras.callbacks.ModelCheckpoint('best_model.h5', monitor='val_loss', mode='min', save_best_only=True),
tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=50, verbose=1, mode='min', min_delta=2, cooldown=0, min_lr=0.000001)
]

history = model.fit(X_train, y_train, validation_data=(X_val,y_val), shuffle=True,
batch_size=BATCH_SIZE,
epochs=EPOCHS, verbose=1,
callbacks=callbacks)


Since this is the first neural network I've trained using my own data, I'm wondering whether there is anything obvious I've missed that could reduce the loss function and if not, any suggestion is welcome!

I should note that I'm using google collaboratory and I've already tried adding another hidden layer but I got ran out of memory error. I've also tried increasing/reducing the number of neurons in each layer but I haven't gotten better results and I tried using a CNN architecture as well, with little success as it didn't even converge after 300 epochs.

• In my experience, it is oftentimes easier to train a classification model than a regression model. So perhaps reformulating the problem so that you classify one of 180 classes (i.e. 0, 1, 2, 3 ... 180), using softmax as final activation layer, and cross entropy as loss function may be a little easier. Oct 18 '19 at 9:13

Change your activation layers, use sigmoid or Tanh for your final layer.

I would try CNN again but with different strides, filter sizes, and number of filters. The thing about CNN is that because you have fewer features per layer you will be able to have more layers.

Here is an example of a Convolutional layer used for audio:

import keras
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPooling2D
from keras.optimizers import SGD
model = Sequential()


If you have time, try training an LSTM layer.

• Thank you for your answer, I'll test your suggestions as soon as I can and report back my findings. I've noticed that in the last layer the activation function is a sigmoid instead of linear, isn't that what I should use when training a regression NN? Oct 18 '19 at 5:39
• No, linear defeats the purpose of an activation layer. Also, linear can be greater than 1 and less than 0, this defeats the purpose of your normalization.
– rigo
Oct 18 '19 at 5:49
• Please keep us posted, keep in mind the cnn is only for reference, I didn't rewrite it for you. If you need that, I can though. Also, why don't you pair your observations into a [2,71276] matrix?
– rigo
Oct 18 '19 at 5:50
• The data is [2,35638] before the pre-processing, I just flatten the vector before training. Do you think I shouldn't do that? Oct 18 '19 at 11:50
• You don't need to do that for CNN, they can be designed to make that encoding for you.
– rigo
Oct 18 '19 at 18:03