1
$\begingroup$

So I'm trying to classify some fashion mnist like photos into either boots or sneakers. I'm using a perception from sklearn to do so. The data set is a CSV containing pixel values. The model is trained through cross validation. Here is the code:

def perceptron_train(ds,target,data,splits):
    clf = linear_model.Perceptron()
    kf = model_selection.KFold(n_splits=splits,shuffle=True)
    X = data
    y = target

    accuracy_model = []
    training_times = []

    for train_index, test_index in kf.split(X):
        X_train, X_test = X.iloc[train_index],X.iloc[test_index]
        y_train, y_test = y.iloc[train_index], y.iloc[test_index]
        train_start = time.time()#time training time
        model = clf.fit(X_train,y_train)
        train_stop = time.time()
        overall_train_time = train_stop-train_start
        training_times.append(overall_train_time)

        #test_start = time.time()
        accuracy_model.append(metrics.accuracy_score(y_test,model.predict(X_test),normalize=True)*100)
        #test_stop = time.time()
    print(np.mean(accuracy_model))
    print(np.mean(training_times))

What I'm worried about in particular is this line:

X_train, X_test = X.iloc[train_index],X.iloc[test_index]
y_train, y_test = y.iloc[train_index], y.iloc[test_index]

The accuracy is very high and I'm worried that I'm either overfitting or I'm training on the wrong part of the dataset. Sorry, I'm just learning cross-validation and just looking for some confirmation that this is not wrong. Thank you

$\endgroup$
3
  • $\begingroup$ Everything looks okay to me. How high is "very high"? Are you getting results that are competitive with SOTA models? $\endgroup$
    – zachdj
    Nov 18, 2020 at 19:40
  • 1
    $\begingroup$ @zachdj around 98%. Thanks for the feedback, I was just concerned that with the accuracy so high. $\endgroup$ Nov 18, 2020 at 20:09
  • $\begingroup$ Hm that is unusually high for a single perceptron trained on fashion MNIST. So you're right to be concerned. But the dataset slicing logic looks fine to me ¯\_(ツ)_/¯ $\endgroup$
    – zachdj
    Nov 18, 2020 at 21:00

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.