Dataset I am working with seems to have been artificially created and the target variable has the same average value no matter the split

Question

I am trying to run a Random Forest on a dataset that has 4 synthetic features, with distributions ranging from approximately -5 to 5. It is not possible to say what those features mean.

The dataset also has product_id, customer_id, product_category and customer_segment. There is absolutely no correlation between the synthetic features and the target variable (total sales), and whichever column I group by (product_id, customer_id, product_category or customer_segment) and find the average total_sales, the average value is the same for each level and very close to the average total sales for the whole dataset.

When I run Random Forest, including all the variables above (synthetic which are numerical and others which are categorical) the model fails miserably on the test set and predicts the average total sales for each row, though it performs better on the train set, I think this suggests overfit. If I try to control for overfit by setting max_depth value for example to 10, the model again defaults to average total sales and predicts almost the same value for each row.

Is there any work around about this, or is it just that the dataset is non-sensical?

Thanks in advance

Answer 1

(Before this answer, user @Harshad Patil gave an answer which feels very ChatGPT-like to me. So I tried feeding the question into GPT-4 and got the answer below, which looks very similar to his. @Harshad Patil's answer was deleted afterwards, but I am leaving this answer here as reference.)

Feeding the question into GPT-4, I get similar response as @Harshad's :

From your description, it seems like your dataset might not contain enough information to predict the target variable (total sales). Here are a few things you might want to consider:

1. Feature Importance: Random Forest provides a way to measure the importance of each feature in prediction. You can check the feature importance to see if the synthetic features or any other features are contributing to the prediction. If the synthetic features are not contributing much, you might want to consider dropping them.

2. Correlation: You mentioned that there is no correlation between the synthetic features and the target variable. This could be a major reason why your model is not performing well. Machine learning models, including Random Forest, rely on patterns and relationships between features and the target variable. If there is no correlation, the model might not be able to learn anything useful from these features.

3. Overfitting: Overfitting occurs when a model learns the detail and noise in the training data to the extent that it negatively impacts the performance of the model on new data. This means that the noise or random fluctuations in the training data is picked up and learned as concepts by the model. You can try to control overfitting by using techniques like cross-validation, regularization, or by increasing the size of your dataset if possible.

4. Data Quality: It's also possible that the quality of your dataset is not good enough. You might want to check if there are any errors in your data, or if there are any outliers that might be affecting the performance of your model.

5. Model Selection: Random Forest might not be the best model for your dataset. You might want to try other models like linear regression, support vector machines, or neural networks and see if they perform better.

6. Feature Engineering: You might want to try creating new features from the existing ones. For example, you can create interaction features, or you can try to transform the synthetic features to make them more useful.

Remember, machine learning is an iterative process. It's all about trying different things and seeing what works best. If none of the above suggestions work, it might be that your dataset is indeed non-sensical for the task at hand.