Visualizing correlation between hyper-parameters and metrics for Neural Network

Question

I am working with a neural network and I want to investigate how different settings affect the loss and standard deviation of the network. I can change various parameters such as the loss function, learning rate, epochs, batch size, threshold value for loss calculation, number of hidden layers, and specific parameters like beta for SmoothL1Loss or num_harmonic for HarmonicFunctionLoss. I have a CSV file for each loss function I use, where the columns correspond to the settings mentioned above. I want to use matplotlib to represent how these settings influence the standard deviation and loss. However, if I fix one parameter, there is a chance that others may change between different trainings. Therefore, I am looking for a good representation to see if there is any correlation between the settings I input and the metrics I use to define the precision of my network. What is the best way to visualize this relationship?

For a little bit of context, I am working on a network using pytorch to recognize line orientation in images. And I am trying to find the optimal parameters to improve the performance of the network. The standard deviation helps me get a much more precise idea on the performance.

Answer 1

Scatter plots are standard way of visualizing relation between two variables. Similarly, you can try multi-dimensional scatter plots for visualizing. Since the number of features will be different for different loss functions or metrics (due to specific parameters), you can make different plots for different loss functions. Also, it gets hard to understand the visualization after 6th dimension, so you can fix some hyperparameters, while changing others (if the total is more than 6). You can check out more about multidimensional plots using matplotlib and seaborn and using plotly.