0
$\begingroup$

I'm trying to build a classification model. Features are purely boolean (not binary) and are in a csv file like 1,0,1,.. The result is an int within a range(0, 128).

I thought, would it make sense not to waste resources on float operations as there are no float data? What models (from Keras, if possible), types of layers, and activators are better for this task?

I'm trying this way and the results are no better than random on ~2mln samples:

  from sklearn.model_selection import train_test_split
  from keras.utils import to_categorical
  from keras.models import Sequential
  from keras.layers import Dense

  X = np.loadtxt("001_X.csv", dtype=bool)
  y = np.loadtxt("001_y.csv", dtype=int)
    
  (X_train, X_test, y_train, y_test) = train_test_split(X, y, shuffle=True, train_size=0.9)

  y_train = to_categorical(y_train)
  y_test = to_categorical(y_test)
        
  model = Sequential()
  model.add(Dense(1000, input_dim=128, activation="relu"))
  ... etc...
  model.add(Dense(128, activation='softmax'))

  model.compile(loss='categorical_crossentropy', optimizer='adam', metrics='accuracy')

  model.fit(X_train, y_train, epochs=50, callbacks=[callback], batch_size=100)

Anything else I can improve for better accuracy?

Improve this question
$\endgroup$
6
  • $\begingroup$ Do you have 128 labels/classes? $\endgroup$
    – Harshad Patil
    Aug 28 at 5:24
  • $\begingroup$ @HarshadPatil Yes. One class, 128 possible values. (The result is one int within a range(0, 128).) $\endgroup$
    – Putnik
    Aug 28 at 5:59
  • $\begingroup$ So you are predicting for 128 different classes. Am I right? $\endgroup$
    – Harshad Patil
    Aug 28 at 6:14
  • $\begingroup$ @HarshadPatilI'm unsure I get the terminology right. My apologies, I'm a newbie. I'm predicting one integer (one column), which can be within the mentioned range only. $\endgroup$
    – Putnik
    Aug 28 at 8:47
  • $\begingroup$ I am asking about 1. You are doing classification rt? If yes then 2. How many different values are there in the prediction column $\endgroup$
    – Harshad Patil
    Aug 28 at 9:08

2 Answers 2

Reset to default
0
$\begingroup$

Most architecture rely on float operations as the optimisation process requires to work with a continum of values. If you want to solve for parameters than can only take discrete values, then it's not a differentiable problem and you cant use gradient descent to optimise the network for your task.

After a quick search it seems that some people worked on creating a Differentiable Logic Gate Networks that allows to learn logic gate combinations to process boolean inputs : https://github.com/Felix-Petersen/difflogic . But's it is not something very common.

If you're network does not learn anything than you may want to change your network, or try to get some insights on your problem in order to help the learning process.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I just want to use the resources wisely. If floats can't be avoided - okay, let's go with that. However, the data does not contain floats. Also, I understand/realize that prediction can say e.g. valueA probability is 0.6, valueB probability is 0.4, etc. $\endgroup$
    – Putnik
    Aug 28 at 8:51
  • $\begingroup$ Indeed, most models wont return a class but rather predict a probability distribution. Then you need to choose a deicision rule in order to obtain the final class. The usual decision use is to take the class with highest predicted probabilities, but this is not the only possible choice. $\endgroup$
    – Lelouch
    Aug 28 at 9:48
  • $\begingroup$ Decision can be made using argmax or prediction[:,1] $\endgroup$
    – Harshad Patil
    Aug 28 at 10:02
  • $\begingroup$ Take the most probable result is not a problem, when you have a decent result. In the initial question I asked advise about model/layers type because those I'm using give really weak results. My apologies if I worded it unclear. $\endgroup$
    – Putnik
    Aug 28 at 10:21
0
$\begingroup$

As you have a lot of labels to predict. I will suggest few basic things which might improve the score or will improve the interpretability:


(X_train, X_test, y_train, y_test) = train_test_split(X, y, shuffle=True, train_size=0.8, stratify = y)

Use stratify. It will make sure that you have equal amount of labels in train and test data.

Try to balance your data by upsampling or downsampling

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.