I have a dataset of shape (2600, 95) with first 55 columns are features and 40 columns are label.

Label is a binary matrix of size 10x4 that flattened, and features are real valued numbers ranging (0.0009, 0.6). The goal is to predict this vector using DNN.

here is the model:

model = Sequential()
model.add(Dense(128, activation='tanh', input_shape=(55,)))
model.add(Dense(64, activation='tanh'))
model.add(Dense(40, activation='sigmoid'))
optimizer = keras.optimizers.Adam(learning_rate=0.001)
history = model.fit(X_train, y_train,epochs=50, batch_size=4,validation_data=(X_test, y_test), verbose=1)

but the results are not satisfyingenter image description here

a few questions:

am i modeling the problem right?

what architecture should i use?

what loss function makes more sense?

what else i should take into account?

  • $\begingroup$ Have you tried changing the activations from tanh to relu? Have you tried increasing the number of layers in your network and the number of neurons per layer? $\endgroup$
    – Oxbowerce
    Oct 1 at 14:09
  • $\begingroup$ yes i tried them, tanh is better than relu. but i'm afraid of non appropriate loss function or metric. $\endgroup$
    – PyRe
    Oct 1 at 14:17

Very generally speaking I don't see any major problems with your approach. You can make a few modifications though.

For starters you might want to scale your data. You can use 0-1 scaling or -1,1, shouldn't matter much. Of course each column needs to be scaled separately. I am assuming there is no relation between your columns, if there is a specific structure, you might be better off using convolutions before fully connected layers: CNN are not only used for image but any type of data that has "features" to be extracted. Without knowing specifics of the columns, it is impossible to say anything though.

I also suggest you include dropout layers in your model. You say you tried ReLU but you might want to check again after scaling and adding dropouts. And also give other variations of ReLU a chance: leakyReLU, ELU, SiLU etc..

And finally make sure your model does not have more trainable parameter compared to your data. Your model seems to be too large for your data to me. You can use model.summary() to get a count of your parameters.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.