2
$\begingroup$

I am using Keras with Theanos backend in Python.

I have 2117 samples and each sample has an individual target (on purpose) ie. 2117 outputs.

As opposed to categories, the targets are ratings eg. (16.4714494876, 17.4129353234, 17.4476570289) the entirety of the number is important.

I am having problems/don't know where to start.

1) When i run the NN it only outputs the targets as whole integers as opposed to the format of the actual values. eg. 16 instead of 16.xxxxxx

2) Presumably i will only be able to gauge the accuracy of predictions based on how close the output is to the target since there are so many targets, does this type of classification problem have a name that i can research?

3) In 3 research papers i have read that apply NN to my specific classification problem they list the output layer as only having 1 neuron but provide no further explanation, how could this be?

Here is my model.

# fix random seed for reproducibility
seed = 7
np.random.seed(seed)

X = np.array(df[FEATURES].values)
Y = np.array(df["MTPS"].values)

# define baseline model
def baseline_model():
    # create model
    model = Sequential()
    model.add(Dense(10, input_dim=(len(FEATURES)), init='normal', activation='relu'))
    model.add(Dense(2117, init='normal', activation='softmax'))
    # Compile model
    model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model
#build model
estimator = KerasClassifier(build_fn=baseline_model, nb_epoch=100, batch_size=5, verbose=2)

#cross validation
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=seed)
estimator.fit(X_train, Y_train)

#print class predictions
print estimator.predict(X_test)
print Y_test

Thanks for any help.

$\endgroup$
1
  • $\begingroup$ Can you add the links of these 3 papers please? Thanks! $\endgroup$
    – Icyblade
    Commented Feb 19, 2017 at 12:34

1 Answer 1

2
$\begingroup$

I'm confused. As you have said, the targets are ratings. It's definitely a regression problem to me, instead of a classification problem.

There's several problems in your code:

  1. For regression problem, we usually use linear as activation function of the last layer (sometimes relu, even sigmoid).
  2. And also we use mse as metric(sometimes mae, msle, etc). categorical_crossentropy is used for classification problem, and sparse_categorical_crossentropy is used for sparse input classification problem. Ref
  3. KerasClassifier is used for classification problem, use KerasRegressor instead.
  4. metrics=['accuracy'] is used for classification problem, and it's meaningless in regression problem. Ref
  5. Assuming df is a Pandas DataFrame, then df.values is naturally a ndarrary, there's no need to cast np.array.

Now answering your question:

  1. As I've never used wrappers for scikit-learn API, I'm not very sure why you have integer output, my best guess is because of KerasClassifier and .predict(in scikit-learn APIs, .predict returns integer, basically the predicted class and .predict_proba returns float, indicating the probability of each class). Try to use .predict_proba, it would help. BTW, you should really use KerasRegressor.
  2. As you have mentioned, there's so many classes to predict. In fact, it's a regression problem.
  3. Could you please add the links of these 3 papers? Neural networks having only 1 neuron in output layer seem to be a regression NN to me. Regression NN usually uses Dense(1, activation='linear') as the output layer.

Here's my version of your code, it might work:

# fix random seed for reproducibility
seed = 7
np.random.seed(seed)

X = df[FEATURES].values  # no need to cast np.array
Y = df["MTPS"].values

# define baseline model
def baseline_model():
    # create model
    model = Sequential()
    model.add(Dense(10, input_dim=(len(FEATURES)), init='normal', activation='relu'))
    model.add(Dense(1, init='normal', activation='linear'))  # one neuron, linear activation
    # Compile model
    model.compile(loss='mse', optimizer='adam')  # mse loss
    return model
#build model
estimator = KerasRegressor(build_fn=baseline_model, nb_epoch=100, batch_size=5, verbose=2)  # KerasRegressor for regression problem

#cross validation
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=seed)
estimator.fit(X_train, Y_train)

#print class predictions
print estimator.predict(X_test)
print Y_test

References

$\endgroup$
1
  • $\begingroup$ As you say, mistakenly i thought it was a classification problem as opposed to regression. I immediately understand the problem from your great answer. Thank you very much. $\endgroup$
    – Thiedent
    Commented Feb 19, 2017 at 13:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.