Timeline for How to intrepret low F1 score and high AUC on training set?
Current License: CC BY-SA 4.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 23, 2023 at 20:12 | comment | added | Hai Nguyen | @BenReiniger I will update them shortly, thank you! | |
| May 23, 2023 at 20:05 | comment | added | Hai Nguyen | @StefanPopov I split the data based on the feature Year, the data was indexed by time. I have around 20 million for training, around 1.5 for validation and testing. I have tried random splitting before, with decent results, but I believe it would not be able to generalize well with future data. | |
| May 22, 2023 at 13:35 | comment | added | Ben Reiniger♦ | Can you please provide the actual ROC curves, and the precision and recall separately? | |
| May 22, 2023 at 10:57 | answer | added | Bayrem | timeline score: 2 | |
| May 22, 2023 at 10:39 | comment | added | Stefan Popov | how did you split your training/validation data? looking at f1 scores it seems that the model is underfitting, while the AUC tells us that it is overfitting. in general, for a model that would generalise well, the train and validation errors should be as close as possible. given the extreme imbalance, i suggest (if you haven't already) to split the data with stratification | |
| May 22, 2023 at 10:07 | history | asked | Hai Nguyen | CC BY-SA 4.0 |