Determining if a city belongs to a country by coordinates with neural networks

Question

I have a task from NN course from my university: I have worldcities dataset with columns (longitude (-180 to 180), latitude ( -80 to 60), is_russia (0 or 1)). ~15000 rows.

And I have to train NN to determine if city belongs to Russia by its coordinates.

1. I normalized data: calculated expected value and standard deviation of longitude and latitude. Then subtracted expected value and divide on standard deviation.

2. I divided data into 3 groups (70%, 10%, 20%) train, validation, test.

3. I made some layers of NN from scratch (on numpy): Linear, BatchNorm, ELU, ReLU, Sigmoid. Made Loss Class (Binary Cross Entropy) and stole Adam optimizer from github :)

4. Architecture of my NN is:

• Linear(2, hidden_size), BatchNorm, ELU
• ammount_hidden * (Linear(hidden_size, hidden_size), BatchNorm, ELU)
• Linear(hidden_size, 1), Sigmoid

5. Hyperparams is

• hidden_size (from 4 to 256 with x2 step)
• ammount_hidden (from 1 to 20 with step +2)
• batch_size (from 32 to 256 with step x2)
• n_epoch (from 1 to 10 with step +1)
• add_num - Amount of Russia coordinates is only 3%. I decided to add this coordinates to train set add_num times. (from 0 (97% - 3%) to 30 (50% - 50%) with step +5)

Of course learning rate is also, but I made adaptive lr: 0.85^n_epoch*3e-4

6. I made cartesian product of all hyperparams and get 14000 rows of them.

• For example: [hidden_size: 8, ammount_hidden: 5, batch_size: 64, n_epoch:3, add_num: 10]

7. I made functions for save and restore NN and the remaining hyperparams from SSD. save_nn format

 - NN hyperparams
 - dim of weight matrix
 - weight matrix of layer
 - bias of layer
 - No weights (BatchNorm)
 - No weights (ELU)
...
 - dim of weight matrix
 - weight matrix of layer
 - bias of layer
 - No weights (Sigmoid)

save_hyperparams format:

• [hidden_size: 4, ammount_hidden: 1, batch_size: 32, n_epoch:1, add_num: 0]
• [hidden_size: 4, ammount_hidden: 1, batch_size: 32, n_epoch:1, add_num: 5]
• [hidden_size: 4, ammount_hidden: 1, batch_size: 32, n_epoch:1, add_num: 10]
• ...
• [hidden_size: 256, ammount_hidden: 20, batch_size: 256, n_epoch:10, add_num: 30]

8. With python multiprocessing I made worker which get hyperparms:

• made nn
• train nn
• return min validation loss and params of NN

9. Train:

• Get num_of_threads - 1 hyperparams and put it into queue
• Create num_of_threads - 1 workers
• Get from each workers min loss, compare to previous
• If less: save loss, nn params and hyperparams

I train 11 networks (6 cores, 12 - 1 threads) at the same time. 10) Then tests. Accuracy ~ 0.96 but on map here I got this:

What shoud I correct in my lab?