3
$\begingroup$

I just started using keras and would like to use unweighted kappa as a metric when compiling my model. Following the instructions from here, I tried to define my custom metric as follows:

library(DescTools)  # includes function to calculate kappa
library(keras)

metric_kappa <- function(y_true, y_pred) {
   CohenKappa(y_true, y_pred)
} 

model %>% compile(
   loss = 'categorical_crossentropy',
   optimizer = optimizer_rmsprop(),
   metrics = metric_kappa
)

However, there must be something wrong with my definition since I get a segmentation fault. Any idea on how to solve this and how to properly set up a custom metric would be highly appreciated.

$\endgroup$
1
$\begingroup$

This answers suggests a wrapper function . Does that work in your usecase ?

https://stackoverflow.com/a/51415066/655855

| improve this answer | |
$\endgroup$
1
$\begingroup$

The loss function intakes and outputs tensors, not R objects. CohenKappa works on R data frames, no doubt. You're basically limited to TensorFlow's backend functions for whatever you do inside the loss function, or any other function (e.g. a layer activation function) that you want to utilize within the scope of a Keras model.

Here's Keras' nice inventory of all those backend functions, or rather, Keras's accessor functions for those Tensorflow functions. (Confusingly, R's mathematical infixes - * and / at least - do get correctly interpreted as Tensorflow elementwise operations, and so there's no k_multiply, for example.)

So, basically, you need to reverse-engineer CohenKappa() such that you can code out its math using these more basic operations.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.