1
$\begingroup$

I could not carry my question from stackoverflow

I ve been trying to fit a neural network for binary setting using library(keras) and I am interested in class probability (instead of 0/1, probability of the event)

I ve 5.018 times more negative than positive class. I added the code I have been using. I cannot stabilize the predictions. I understand that noise and everything. But I need to put some constraints to get close estimates each time. I am out of ides. Is there anything else I can use to stabilize predictions?

I cannot share the data therefore here is summary of predictions at train data level and I plotted validations/train.

 first run               Second run
 Min.   :0.001843       Min.   :0.0004508 
 1st Qu.:0.012272       1st Qu.:0.0156236 
 Median :0.042264       Median :0.0459510 
 Mean   :0.142551       Mean   :0.1400624  
 3rd Qu.:0.195536       3rd Qu.:0.1937293
 Max.   :0.919892       Max.   :0.9882065 

validation plot for first run first run and validation plot for second runenter image description here

l2_model <- 
  keras_model_sequential() %>%
  layer_dense(units = 512, activation = "relu", input_shape =  ncol(XX_train1),
              kernel_regularizer = regularizer_l2(0.001)) %>% 
  layer_batch_normalization()%>%
  layer_dense(units = 256, activation = "relu", 
              kernel_regularizer = regularizer_l2(0.001)) %>%
  layer_batch_normalization()%>%
  layer_dense(units = 1, activation = "sigmoid",
              bias_initializer = initializer_constant(log(5.0189)))

l2_model %>% compile(
  optimizer="Adam",
  loss = "binary_crossentropy",
  metrics =  c('accuracy')
)

summary(l2_model)

l2_history <- l2_model %>% fit(
  x                = as.matrix(XX_train1), 
  y                = YY_train1,
  epochs = 30,
  batch_size = 1000,
  validation_data = list(XX_test, YY_test[,2]),
  verbose = 2,
  callbacks = list(
    callback_early_stopping(patience = 2) )
 #   ,callback_reduce_lr_on_plateau()  )
)


# Predicted Class Probability
yhat_keras_prob_vec  <- predict_proba(object = l2_model, x = as.matrix(XX_train1)) %>%
  as.matrix()

summary(yhat_keras_prob_vec)

I have look at some questions to get ideas but did not work neural-network-loss-function-for-predicted-probability Understanding neural network probability

$\endgroup$
  • $\begingroup$ Did you try experimenting with adding another layer of dense units or making the network less wide ? Does that affect your results ? If yes, then how ? $\endgroup$ – Ankita Talwar Sep 2 at 1:17
  • $\begingroup$ yes, similar problem. I feel I need to control the gamma parameter or some other parameters but I do not where to look or start. $\endgroup$ – iHermes Sep 2 at 1:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.