0
$\begingroup$

I am creating a binary classifier in Keras and here's the code

model = Sequential()
model.add(Dense(30, input_dim=60, activation='relu'))
model.add(Dense(1, activation='tanh'))
model.compile(loss='mse', optimizer='adam', metrics=['accuracy'])

The problem though, is that I have variable targets, i.e. they lie between -1 and 1. And here is a histogram of the distibution of my targets

Histogram

I know that this still, could work very well for me, and the model would probably be able to predict the variable outputs, the input data is very complex, and the model might not be able to train to predict the exact outputs. I am perfectly okay of the model cannot predict the exact values, as a long as the sign of the output [-1,1] is correct.

So I thought what if the loss function gave zero loss for a prediction that has the correct sign, but gave whatever the required binary loss is, if the output has the wrong sign, this way, during training it would give importance to the highly incorrect predictions, but as soon as the sign of the predictions are right (just what I need), it doesn't consider them anymore, hopefully giving a better result. The catch here is, that if I just trained it to predict the signs of the targets, it would give all of the inputs equal importance

So my question is, does any such loss function exist in Keras or anywhere else, or do I have to create my own loss function? If I have to create my own loss function, how do I do it?

$\endgroup$
0
$\begingroup$

Actually what you need is a linear regression model and not a binary classifier. You can trying to predict continuous variables using a classifier. So we need to build a regression model which predicts continuous variables in a range i.e from -1 to 1.

You can do this with some changes in your code.

model = Sequential()
model.add(Dense(30, input_dim=60, activation='relu')) 
model.add(Dense(1)) 
model.compile(loss='mse', optimizer='adam', metrics=['accuracy'])

So basically we maintain a linear activation function in the output layer for regression.

Tip:

Instead of accuracy as the metric, use mae or mean absolute error to monitor the model's progress.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ But the problem here is that, suppose my model predicts 0.4, when the true value is 0.6. I do not mind this error, because the sign of the prediction matches. However, when using mse, the model looks at this as an error and tries to correct it $\endgroup$ – ѕняєє ѕιиgнι Aug 18 '19 at 10:15
0
$\begingroup$

If you really just want to guess the sign, you should just build a new target : 0 if the sign is negative 1 if the sign is positive... That would fit with your binary classification approach and the metrics you want to use.

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.