bad input shape (5634, 2)

Question

I tried everything and I am not sure how to resolve the following error:

ValueError: bad input shape (5634, 2)

This is my first machine learning example so please bear with me. This is the python code:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import warnings
warnings.filterwarnings("ignore")
from pylab import rcParams
from IPython import get_ipython

ipy = get_ipython()
if ipy is not None:
    ipy.run_line_magic('matplotlib', 'inline')
# Loading the CSV with pandas
data = pd.read_csv('...WA_Fn-UseC_-Telco-Customer-Churn.csv')

# Data to plot
sizes = data['Churn'].value_counts(sort = True)
colors = ["grey","purple"]
rcParams['figure.figsize'] = 5,5
explode = (0.1, 0)  # explode 1st slice
labels = "Yes","No"
# Plot
plt.pie(sizes, explode=explode, labels=labels, colors=colors,
        autopct='%1.1f%%', shadow=True, startangle=270,)
plt.title('Percentage of Churn in Dataset')
plt.show()

data.drop(['customerID'], axis=1, inplace=True)
tc = (data['TotalCharges'].str.strip())
#print(tc)
data['TotalCharges'] = pd.to_numeric(tc)

data["Churn"] = data["Churn"].eq('Yes').astype(int)
Y = pd.get_dummies(data["Churn"].values).fillna(0)
X = pd.get_dummies(data.drop(labels = ["Churn"],axis = 1)).fillna(0)


print(X.shape) #(7043, 45)
print(Y.shape) #(7043, 2)

# Create Train & Test Data
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.2, random_state=101)

from sklearn.linear_model import LogisticRegression

model = LogisticRegression()
result = model.fit(X_train, y_train)

Why does the error

bad input shape (5634, 2)" when X.shape is (7043, 45) Y.shape is (7043, 2)?

Answer 1

Logistic Regression expects labelled y_train, you do not need to do OneHotEncoding.

y : array-like, shape (n_samples,)

Above is from sklearn LogisticRegression.fit.

I think as long as it is label encoded(using LabelEncoder), basically this tool will automatically generate label mapping from your categorical values to unique integers between 0 to n-1, where n represent count of distinct value of the category. LogisticRegresion will work fine with label encoded target value.

Protip : Please use labelencoder or OneHotEncoder to make mapping of categorical values. The problem with get_dummies is that it relies on , suppose you are doing transformation on different separated dataset(e.g. test set contained in another dataframe), suppose that on trainset you have 3 unique categorical values but on test set there are only 2 unique values. Then pd.get_dummies will only give you 2 columns (other than 3) when you are perform it on the test set.