0
$\begingroup$

First I am new to machine learning if it is an obvious question, I am sorry.

dataset_coefficients = loadtxt(
'in.csv', delimiter=',')
dataset_answers = loadtxt(
    'out.csv')
X = dataset_coefficients[:, 0:4]
y = dataset_answers

model = Sequential()
model.add(Dense(32, input_dim=4, activation='relu'))
model.add(Dense(16, activation='sigmoid'))
model.add(Dense(8, activation='sigmoid'))
model.add(Dense(4, activation='sigmoid'))
model.add(Dense(2, activation='sigmoid'))
model.add(Dense(1, activation='softmax'))

model.compile(loss="mean_squared_error",
              optimizer='adam', metrics=['accuracy'])


model.fit(X, y, epochs=250, batch_size=100, verbose=0)
_, accuracy = model.evaluate(X, y, verbose=0)
print('Accuracy: {}%'.format(accuracy * 100))

in.csv looks like this

     1,       -21,       147,      -343
     1,        19,       115,       225
     1,         1,       -64,       -64
     1,        30,       300,      1000
     1,        16,        64,         0
     1,         3,       -81,      -243
     1,         3,         3,         1
     1,       -13,        16,       192

out.csv

     1
     2
     3
     1
     2
     3
     1
     2

When I test the accuracy with the data which was previously used for training

Output is

Accuracy: 33.33333432674408%

No matter how big my dataset is, the result has the exact same accuracy rate of 33% which seems like the prediction is literally random. What am I doing wrong? Thanks

$\endgroup$
  • 1
    $\begingroup$ MSE is likely not the best objective here. You definitely should be normalizing/standardizing your input. $\endgroup$ – Ben Reiniger Dec 9 '19 at 22:47
  • $\begingroup$ If you are using mean squared error, you should have a linear activation function for the output layer. As @BenReiniger said, try standardizing or normalizing your data for regression. $\endgroup$ – Shubham Panchal Dec 10 '19 at 0:19
2
$\begingroup$

I identified several problems when looking through your codes :

  1. The first column for in.csv is a dummy variable for bias term (all 1 for the whole column). For NN layers in Keras unless you put include_bias=False, weights for bias will be included automatically hence this column would not be necessary.
  2. Your variable is not scaled/standardized, although sometimes it will work without normalization but it is always recommended to do this for NN since input like that may lead to unstable training. Consider using StandardScaler or MinMaxScaler for starter.
  3. You mentioned on the tags that you are working on classification so this is what you should do. First, identify the number of unique classes, this should be the number of hidden unit on the last layer. Next, you have two choices, if you want to use OneHotEncoder then when compiling your model your loss should be 'categorical_crossentropy'. The other choice, you first make sure that your label index starts from 0 and then use 'sparse_categorical_crossentropy' as your loss.
  4. Softmax activation function applies the function AFTER the weights and biases operation. So the output of your model will first be a set of probability of the sample being some class and to get one output simply call .argmax(axis=1) which will return the index with the most probability on each row. You also need to do + 1 since your class labels starts from 1 instead of 0.
| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.