New answers tagged

1

For comparing two rankings Spearman's rank correlation is a good measure. It's probably worth a try, but since your gold truth appears to be binary I would think that top-N accuracy (or some variant of it) would be more appropriate (advantage: easy to interpret). You could also consider using the Area Under the Curve (AUC), using the predicted rank as ...


0

Micro calculates F score globally by counting the total true positives, false negatives and false positives. Macro calculates F score for each label and find their unweighted mean. Macro F score does not take label imbalance into account. Given there is a difference in your performance between the metrics, your data is imbalanced in the base-rate for the ...


0

I think this covers your issue in the Keras documentation https://keras.io/callbacks/#create-a-callback class LossHistory(keras.callbacks.Callback): def on_train_begin(self, logs={}): self.losses = [] def on_batch_end(self, batch, logs={}): self.losses.append(logs.get('loss')) model = Sequential() model.add(Dense(10, input_dim=784, ...


0

Sklearn kappa has 2 variants: simple and weigthed. It seems WKappa refers to weighted (quadratic) kappa in terms of sklearn, and Kappa - to simple.


2

There is a quite detailed comparison with references here: https://towardsdatascience.com/a-tale-of-two-macro-f1s-8811ddcf8f04 Basically the two definitions are used and both can be considered valid. For the sake of clarity I would recommend mentioning which definition you are using when you report your results.


4

Just to clarify (and I think you've got this right, but I'm just being careful), it is best practice to: 1: Split your data into train and test 2: Split train into train and eval 3: Grid search over hyperparameters, for each combination, train on train, evaluate on eval. Select the hyperparameters which allow you to get the best score on the eval set ...


Top 50 recent answers are included