How to define a custom metric function in R for Keras?

Question

I just started using keras and would like to use unweighted kappa as a metric when compiling my model. Following the instructions from here, I tried to define my custom metric as follows:

library(DescTools)  # includes function to calculate kappa
library(keras)

metric_kappa <- function(y_true, y_pred) {
   CohenKappa(y_true, y_pred)
} 

model %>% compile(
   loss = 'categorical_crossentropy',
   optimizer = optimizer_rmsprop(),
   metrics = metric_kappa
)

However, there must be something wrong with my definition since I get a segmentation fault. Any idea on how to solve this and how to properly set up a custom metric would be highly appreciated.

Answer 1

This answers suggests a wrapper function . Does that work in your usecase ?

https://stackoverflow.com/a/51415066/655855

Answer 2

The loss function intakes and outputs tensors, not R objects. CohenKappa works on R data frames, no doubt. You're basically limited to TensorFlow's backend functions for whatever you do inside the loss function, or any other function (e.g. a layer activation function) that you want to utilize within the scope of a Keras model.

Here's Keras' nice inventory of all those backend functions, or rather, Keras's accessor functions for those Tensorflow functions. (Confusingly, R's mathematical infixes - * and / at least - do get correctly interpreted as Tensorflow elementwise operations, and so there's no k_multiply, for example.)

So, basically, you need to reverse-engineer CohenKappa() such that you can code out its math using these more basic operations.