0
$\begingroup$

I have a dataset in CSV format, 6 columns and 1877 rows. The full dataset can be viewed at ShareCSV.

The first five columns are characteristics and the final column is a binary result, I want to create a classification network to predict result using the five inputs as seen in the CSV above.

I use the following code to normalize the data with pandas, then split into testing and training groups.

from sklearn import preprocessing
import pandas as pd

df = pd.read_csv(r"D:\path\data.csv", sep=",")

df=(df-df.min())/(df.max()-df.min())

X = df[["Title words", "Hour posted", "Author age", "Is Text", "Subreddit age"]]

Y = df['Front']

X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.2, random_state=42)

I now need to pass this data to scikit-learn and select a classification algorithm, however this is where I am unsure what would be optimal, if anyone could recommend the best algorithm for my data and a rough implementation that would be great.

$\endgroup$
  • $\begingroup$ There is no way of knowing what the "best classification algorithm" will be, you need to try multiple algorithms out and see which one performs the best. $\endgroup$ – Djib2011 Feb 5 at 10:01
1
$\begingroup$

I would suggest you to scale your data using standard scaler and do it before you split it into X and Y here in your case. Why ? Please check this answer on stats sc. Also, keep the target variable (Front in your case) as it is. So, according to me, the right choice looks like this, however you can try and experiment with min max scaler too:

from sklearn.preprocessing import StandardScaler
import pandas as pd
df = pd.read_csv(r"D:\path\data.csv", sep=",")
X = df[["Title words", "Hour posted", "Author age", "Is Text", "Subreddit age"]]
Y = df['Front']
scaler = StandardScaler()
scaled_X = scaler.fit_transform(X)
scaled_X = pd.DataFrame(scaled_X, columns = X.columns)
X_train, X_test, y_train, y_test = train_test_split(scaled_X, Y, test_size=0.2, random_state=42)

To start, you can try with logistic regression and move on to svm, or maybe neural networks, if your data is big enough.

from sklearn.linear_model import LogisticRegression
lr = LogisticRegression(class_weight='balanced', C = 0).fit(X_train, y_train)
y_pred = lr.predict(X_test)
precision, recall, fscore, support = score(y_test, y_pred, pos_label= 1, average='binary')

Now, you can choose a good performance metric for the same, looking at the distribution of the class label, whether they are balanced/imbalanced etc.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.