What does it mean when an Actual vs Predicted plot is like this assuming you are using the best model?

Question

I'm trying to predict the monetary value in a fixed time-frame for a project. I wanted to start with a baseline model before doing any feature engineering or advanced pre-processing.

I'm using a feed-forward neural network for regression (Sklearn's MLPRegressor) and I'm exploring a normal, wide and deep neural network.

Usually what I do is feed a dictionary of parameters to my grid-search and get my baseline model from there. Surprisingly enough, the Actual vs Predicted plot I'm getting this time from my best model is confusing.

This is the plot I'm getting here, red is actual and blue is predicted :

This is the loss curve for that model, spoiler alert, it's like I've never seen before :

And these are the parameters from the best model :

{'activation': 'relu', 'alpha': 0.05, 'learning_rate': 'adaptive', 'learning_rate_init': 0.001, 'hidden_layer_sizes': (24, 12, 6), 'solver': 'adam'}

What am I doing wrong ?

Answer 1

It's not said taht you are doing something wrong, some datasets don't contain highly predictable data. However in this case it seems there is a pattern in your data that your models is not capturing. I have a couple of questions about it:

1. Is there a big difference between train and test performances? If so, you must use some regularization techniques, such as dropout, in order to reduce overfitting.
2. What is the architecture of your Neural Network? And did you try different ones?