I want a loss function which only care about the pecentile head and tail.

for example:

I have a rmse target, but i only care about rmse which y_pred > y_pred.percentile(90) and y_pred < y_pred.percetile(10), in the middle of y_pred, i dont want to consider them in my loss.

here is my code:

class myloss(keras.losses.Loss):
  def __init__(self, coef, name='myloss'):
    self.coef = coef

  def call(self, y, y_pred):
    ## TODO: i want to sum over y_pred > y_pred.percentile(90) and y_pred< y_pred.percentile(10)!!!
    # return tf.math.reduce_mean(tf.square(y * self.coef- y_pred), axis=1)

from util import *
from tensorflow import keras
from tensorflow.keras import layers
import tensorflow as tf
if __name__ == '__main__':
  df = read_cache_data('cache')
  df = df.dropna()
  fv_cols = df.columns[21:-3]
  res_col = 'res_10'
  #model = keras.Sequential([layers.Dense(50, input_shape=(len(fv_cols), ), activation='relu'), layers.Dense(30, activation='relu'), layers.Dense(1)])
  model = keras.Sequential([layers.Dense(1, input_shape=(len(fv_cols), ))])#, activation='relu')])
  model.compile(optimizer=keras.optimizers.Adam(), loss = myloss('loss'))#keras.losses.mean_squared_error)                                                                                                                                           
  model.fit(df[fv_cols].values, df[res_col], batch_size=288)
  • $\begingroup$ What you want to do is weird, and i'm not sure i understand it correctly, but i assume you could use the answer to this post. $\endgroup$
    – Ubikuity
    Commented Apr 29, 2021 at 17:17
  • $\begingroup$ The percentiles are taken over an epoch or batch or what? $\endgroup$ Commented Oct 19, 2022 at 8:40


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.