2
$\begingroup$

I'm trying to make an NN that, given the time on the clock, would try to predict which class (out of 32 in this example) is making a request to the system. As a first attempt, I've tried to use categorical_crossentropy, but this will obviously not work because the targets are very sparse, so the system will be heavily rewarded by just always predicting the non-requests.

Now I'm trying to use sparse_categorical_crossentropy, but I keep getting a dimension mismatch error (the train and test sets are the same in this case because I just wanted to evaluate performace in the training set at first):

Error when checking target: expected dense_90 to have shape (1,) but got array with shape (32,)

The DataFrame is here (a simple clock and another column for the requests) and the code is:

from keras.models import Sequential
from keras.layers import Dense
from keras.utils import to_categorical
import tensorflow as tf

requests = df['requests'].values
requests_cat = to_categorical(requests, 32)

length = len(df['clock'])
train = np.reshape(df['clock'].values, (length, 1))
train = train.astype(np.int)
target = requests_cat

model = Sequential()
model.add(Dense(25, activation = 'relu', input_shape = (train.shape[1],)))
model.add(Dense(25, activation = 'relu'))
model.add(Dense(32, activation = 'softmax'))

model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy')
model.fit(x = train, y = target, epochs = 100, validation_data = (train, target))

On a sidenote:

  1. This architecture doesn't seem to be the best in this case. As a second prototype I was thinking about doing something with an LSTM, since past requests can affect later ones. Is there a standard architecture for scheduling?
  2. What would be the proper way of splitting sparse sets into training and testing ones?
$\endgroup$
  • $\begingroup$ So, I think I've found the answer to the main question now. The sparse_categorical_crossentropy doesn't accept the targets to be one-hot encoded. You just feed them as integers. But now, the main problem is that this sparse loss is not working either. Maybe the requests are still too sparse? Or maybe this architecture is not at all suitable for this problem? Anyway, if I don't get an answer sometime soon, I'll answer it myself. $\endgroup$ – Philippe Fanaro Oct 3 '18 at 20:43
2
$\begingroup$

I thing you've misunderstood what the difference between categorical_crossentropy and sparse_categorical_crossentropy is. The sparse part doesn't refer to the sparsity of the data but the format of the labels.

  • If your labels are one-hot encoded: use categorical_crossentropy

  • If your labels are encoded as integers: use sparse_categorical_crossentropy

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Damn, that makes a lot of sense. I had actually studied that at some point in the recent past, but obviously got confused. Do you have any suggestions for how to deal with the sparsity in the data? (I've already even tried to make the event kind of spill to the nearby time steps, just so it relieves a bit of the sparsity in the data.) $\endgroup$ – Philippe Fanaro Oct 4 '18 at 1:58
  • $\begingroup$ Unfortunately not, I haven't worked with many sparse problems in the past. I didn't fully understand your data. You're trying to predict the requests from the clock? If yes it looks like a time series problem. It would make sense to use a recurrent model (like LSTM, as you said), if that were the case. $\endgroup$ – Djib2011 Oct 4 '18 at 8:19
  • $\begingroup$ Yeah, that's exactly what I wanted to do. The LSTM would be my second prototype, but I first wanted to get some results with a simpler architecture. If you want, you can look at it more in this link. $\endgroup$ – Philippe Fanaro Oct 4 '18 at 13:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.