I'm using a DNN-48 having the following scenario:
- Features: 8 (48 at the end because I generate conditional sequences of 6 elements each)
- Classes: Y=0 (90%), Y=1 (10%)
Precision and recall are good when Y=0. If I adjust weights properly the recall is good on Y=1 too. The real problem is precision when Y=1.
Using SMOTE results were even worse. Best alternative at the moment was applying a custom stratification based on picking 8 of Y=0 for each Y=1 found.
I've tried moving the threshold to 0.8-0.9 and it slightly works but it sacrifices some recall. I've also tried changing the NN topology adding more layers, units and using regularization like L1/L2 and dropouts. For the time being I've seen best results using simple a topology like:
model = Sequential([ Dense(64, activation='relu', input_shape=(X.shape,)), Dense(32, activation='relu'), Dropout(0.2), Dense(16, activation='relu'), Dropout(0.2), Dense(8, activation='relu'), Dense(4, activation='relu'), Dense(1, activation='sigmoid') ]) model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy', f1_score])
I'd like to keep the good recall without having too much false positives. Any ideas? thank you!