# Loss drops to NaN after a short time for a time series classification

here is my model code for a binary classification of a time series:

def make_model(feature_columns):
feature_layer = tf.keras.layers.DenseFeatures(feature_columns)
feature_layer_outputs = feature_layer(feature_layer_inputs)
feature_layer_outputs = tf.expand_dims(feature_layer_outputs, 1)

conv = keras.layers.Conv1D(filters=64, kernel_size=3, padding="same",kernel_regularizer=keras.regularizers.l1_l2(l1=0.01, l2=0.01))(feature_layer_outputs)
conv = keras.layers.BatchNormalization()(conv)
conv = keras.layers.ReLU()(conv)

conv = keras.layers.Conv1D(filters=64, kernel_size=3, padding="same",kernel_regularizer=keras.regularizers.l1_l2(l1=0.01, l2=0.01))(conv)
conv = keras.layers.BatchNormalization()(conv)
conv = keras.layers.ReLU()(conv)

conv = keras.layers.Conv1D(filters=64, kernel_size=3, padding="same",kernel_regularizer=keras.regularizers.l1_l2(l1=0.01, l2=0.01))(conv)
conv = keras.layers.BatchNormalization()(conv)
conv = keras.layers.ReLU()(conv)
conv = keras.layers.Dropout(0.25)(conv)

gap = keras.layers.GlobalAveragePooling1D()(conv)

output_layer = keras.layers.Dense(1, activation="Softmax")(gap)

return keras.models.Model(inputs=[v for v in feature_layer_inputs.values()], outputs=output_layer)


So i tried the following. First i had from the beginning nan as loss. I fixed that by using the RobustScaler on numeric values

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, RobustScaler,MinMaxScaler
dataframe = df
ct = ColumnTransformer([
('numeric', RobustScaler(), numerical_features[1:])
], remainder='passthrough')

dataframe = ct.fit_transform(dataframe)
`

numerical_features[0] is my label. I also tried to add keras.regularizers.l1_l2 to fix the loss nan problem and to add dropout after each relu layer (now its only on the last). I also tried to use different losses like MSE and binary crossentropy also i tried using sigmoid/softmax in the outputlayer. Im pretty much clueless at this point and i have the feeling im missing something which is rather basic. Does someone has any idea?