0
$\begingroup$

I tried everything and I am not sure how to resolve the following error:

ValueError: bad input shape (5634, 2)

This is my first machine learning example so please bear with me. This is the python code:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import warnings
warnings.filterwarnings("ignore")
from pylab import rcParams
from IPython import get_ipython

ipy = get_ipython()
if ipy is not None:
    ipy.run_line_magic('matplotlib', 'inline')
# Loading the CSV with pandas
data = pd.read_csv('...WA_Fn-UseC_-Telco-Customer-Churn.csv')

# Data to plot
sizes = data['Churn'].value_counts(sort = True)
colors = ["grey","purple"]
rcParams['figure.figsize'] = 5,5
explode = (0.1, 0)  # explode 1st slice
labels = "Yes","No"
# Plot
plt.pie(sizes, explode=explode, labels=labels, colors=colors,
        autopct='%1.1f%%', shadow=True, startangle=270,)
plt.title('Percentage of Churn in Dataset')
plt.show()

data.drop(['customerID'], axis=1, inplace=True)
tc = (data['TotalCharges'].str.strip())
#print(tc)
data['TotalCharges'] = pd.to_numeric(tc)

data["Churn"] = data["Churn"].eq('Yes').astype(int)
Y = pd.get_dummies(data["Churn"].values).fillna(0)
X = pd.get_dummies(data.drop(labels = ["Churn"],axis = 1)).fillna(0)


print(X.shape) #(7043, 45)
print(Y.shape) #(7043, 2)

# Create Train & Test Data
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.2, random_state=101)

from sklearn.linear_model import LogisticRegression

model = LogisticRegression()
result = model.fit(X_train, y_train)

Why does the error

bad input shape (5634, 2)" when X.shape is (7043, 45) Y.shape is (7043, 2)?

$\endgroup$
1
  • $\begingroup$ in which line you got the error exactly? and why shape of Y is (n,2)? $\endgroup$ Commented Nov 28, 2019 at 23:46

1 Answer 1

0
$\begingroup$

Logistic Regression expects labelled y_train, you do not need to do OneHotEncoding.

y : array-like, shape (n_samples,)

Above is from sklearn LogisticRegression.fit.

I think as long as it is label encoded(using LabelEncoder), basically this tool will automatically generate label mapping from your categorical values to unique integers between 0 to n-1, where n represent count of distinct value of the category. LogisticRegresion will work fine with label encoded target value.

Protip : Please use labelencoder or OneHotEncoder to make mapping of categorical values. The problem with get_dummies is that it relies on , suppose you are doing transformation on different separated dataset(e.g. test set contained in another dataframe), suppose that on trainset you have 3 unique categorical values but on test set there are only 2 unique values. Then pd.get_dummies will only give you 2 columns (other than 3) when you are perform it on the test set.

$\endgroup$
3
  • $\begingroup$ Sorry, I'm not sure if I understand what LabelEncoder means $\endgroup$
    – Sindu_
    Commented Nov 28, 2019 at 10:51
  • $\begingroup$ it is a tool from sklearn, I'll edit my answer a bit. $\endgroup$ Commented Nov 28, 2019 at 10:56
  • $\begingroup$ I tried this: from sklearn.preprocessing import LabelEncoder labelencoder = LabelEncoder() data["Churn"] = data["Churn"].eq('Yes').astype(int) Y = labelencoder.fit_transform(data["Churn"].values) X = labelencoder.fit_transform(data.drop(labels = ["Churn"],axis = 1)) I now get ValueError: bad input shape (7043, 19) $\endgroup$
    – Sindu_
    Commented Nov 28, 2019 at 11:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.