SVM produces a constant accuracy when testing with development set, regardless of features

Question

I am currently doing a class project to use a machine learning algorithm (SVM or Regression) to deduce whether two sentences are paraphrases of one another. We were given training, development, and test datasets, and when training my model I am given an accuracy that appears to be constant no matter which features are added/removed.

I believe it is possibly due to the model not properly attaining the features, but my primary concern is that depending on what I use, it produces a different constant accuracy.

Vscode:0.5021459227467812
JupyterLab (Kaggle): 0.7421652421652422

Training DataFrame
Development DataFrame

Code using training and development sets:

#development
X_train = df_train.iloc[:,6:]
y_train = df_train['gold label'].values
X_dev = df_dev.iloc[:,6:]
y_dev = df_dev['gold label'].values

classifier = svm.SVC()
classifier.fit(X_train, y_train)

Y_pred = classifier.predict(X_dev)

print(classifier.score(X_dev, y_dev))

Please let me know what the issue could be or if there is a better way. Thank you!

Answer 1

From the images you posted it looks like df_train and df_dev are identical. When you assign X_train and X_dev they are therefore the same as well as y_train and y_dev. This means that your train and development data are identical and they will therefore produce identical results when the model is applied to them.

Now, I don't know why your dataframes are the same, this does not become apparent from your post. You might have been provided with incorrect data, are reading the data incorrectly or your way to split the data does not work as you intend it to.