New answers tagged

0

Welcome to Data Science at StackExchange, To prepare the data for your model, you can look at the stratify method in train_test_split. It will detect the ratio of values in the column you choose and keep the distributions equal after the split. For example, if there is a 100:1 ratio in the original dataset, the train and test ratio would also be 100:1. Take ...


-1

Welcome to StackExchange! Yes, the idea of mini-batches is to augment balance in an unbalanced dataset. You should train on balanced datasets (i.e same prevalence of all classes) and measure performance on a representative dataset as discussed here and here. A neural network trained with imbalanced data often has varied levels of precision in determining ...


3

Both Random Forest Classifier and Extra Trees randomly sample the features at each split point, but because Random Forest is greedy it will try to find the optimal split point at each node whereas Extra trees selects the split point randomly. I would choose Random Forest because it's more likely to create a split point that accounts for the imbalanced class, ...


1

With Success already being the larger class, you probably shouldn't be using a scale_pos_weight larger than one: you want to scale the positive class's contribution to the loss function down rather than up. I suspect that's what's happening in the first case. With scale_pos_weight=75, the model ends up basically only caring about the positive class, ...


0

If you use the anomaly detector to label the data directly, there is no way the supervised step that follows can be better than that. One can of course go in an "adjust" the labels afterwards, but there is risk of being biased by the pre-existing labels if a human sees it up front. Instead of sampling data to label randomly, you could sample weighted based ...


0

Imbalanced class means the count of one class is too low compared to the count of other Class. This means Model will have little opportunity to learn the minority Class. We have these option to handle the issue. Key goal is to reduce the fog created by the majority Class and let the Model see the Minority class too - Weighted Class - This instructs the ...


0

your resume is quite good, but I'm not comfortable in dividing the broad discussion to those three more or less sharply separated roads. But indeed, often a technique similar to one of those is chosen. Just let me underline something about them: the Area Under the Curve (AUC) of Precision and Recall has been shown as being slightly better than AUC ROC, but ...


Top 50 recent answers are included