1
$\begingroup$

I am trying an Retail ML project, but am stuck on the Error "Found input variables with inconsistent numbers of samples: [982644, 911]". I tired many thing & I know why this error occurs, but I can't figure out a solution for it. Can anybody please help me. I've been stuck on it for past 2 days.

Y_train = train1['Sales']
Y_val = test_val1['Sales']

X_train = train1.drop(['Sales', 'Date', 'Customers'], axis = 1).values
X_val = test_val1.drop(['Sales', 'Date', 'Customers'], axis = 1).values
X_train = X_train.reshape(X_train.shape[0:])

rr = Ridge(alpha=10)
rr.fit(X_train, Y_train)
Y_pred1 = rr.predict(X_val)

print('MSE',np.sqrt(mean_squared_error(Y_pred1,Y_val)))
print('MAE',mean_absolute_error(Y_pred1,Y_val))
print('train model score',rr.score(X_train, Y_train))
print('test model score',rr.score(X_val,Y_val))

I am getting the error on rr.fit(X_train, Y_train). I have performed Linear Regression with the same object variables, but cant seem to perform the Regularization of the Model.

$\endgroup$
6
  • $\begingroup$ Try this X_train = train1.drop(['Sales', 'Date', 'Customers'], axis = 1). Dont put the .values at the end. $\endgroup$
    – spectre
    Sep 23 at 5:56
  • $\begingroup$ @spectre I tried your suggestion, but am still getting the same error msg. $\endgroup$ Sep 23 at 9:04
  • $\begingroup$ Why are you splitting the data manually? You can use sklearn's module train_test_split which might solve the problem. If that doesn't solve your problem, link a collab notebook and i'll look at it.\ $\endgroup$
    – spectre
    Sep 23 at 9:16
  • $\begingroup$ @spectre The train_test_split doesn't seems to work, it throws ValueError ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (4,) + inhomogeneous part. I have shared a collab link : colab.research.google.com/drive/… $\endgroup$ Sep 23 at 10:15
  • $\begingroup$ @ArislanMakhmudov I have mounted my drive have attached the uploaded the csv files. The collab will work now. $\endgroup$ Sep 23 at 17:25
0
$\begingroup$

I have checked the colab example. There seems to be not enough memory to train a Ridge model with your humongous dataset whose shape is (982644, 1169). The notebook crashes when attempting to execute the said line,

rr.fit(X_train, Y_train)

So I tried decreasing the size of the dataset, and everything worked fine.

xt = X_train.loc[:200000].copy()
yt = Y_train.loc[xt.index].copy()

xv = X_val.copy()
yv = Y_val.copy()

print(xt.shape, yt.shape)
print(xv.shape, yv.shape)

rr = Ridge(alpha=10)
rr.fit(xt, yt)

Y_pred1 = rr.predict(xv)

print('MSE:',np.sqrt(mean_squared_error(Y_pred1,yv)))
print('MAE:',mean_absolute_error(Y_pred1,yv))
print('train model score:',rr.score(xt, yt))
print('test model score:',rr.score(xv,yv))

Output:

(200001, 1169) (200001,)
(34565, 1169) (34565,)
MSE: 1431.9094471008812
MAE: 1058.263279968715
train model score: 0.8604548907625048
test model score: 0.8423455437913853

NB: be sure to load all the datasets properly, when dataset files in the mounted drive are corrupt you may get unexpected errors when trying to execute the code.

$\endgroup$
2
  • $\begingroup$ Thank you for your answer, that solution works. I guess the dataset was large, which resulted in the wrong splitting & generated the error. Thanks very much. $\endgroup$ Sep 24 at 12:22
  • $\begingroup$ I am glad to hear this helped you, you're welcome! $\endgroup$ Sep 24 at 12:32
0
$\begingroup$

If you are just dropping the columns like X_train = train1.drop(['Sales', 'Date', 'Customers'], axis = 1) will give a dataframe, when you use .values at the end then the result type will be numpy array. So please remove the .values also the reshape line won't be required. Thanks

$\endgroup$
4
  • $\begingroup$ I tried your suggestion but am still getting the same ValueError message. $\endgroup$ Sep 23 at 9:05
  • $\begingroup$ I checked your notebook, In that i could see print(combi.shape) gives (351613, 12) but when you are splitting the rows using .loc you have started with the index of 982644 which is greater than the total rows (351613). If you check the output of train1.shape, test_val1.shape, test1.shape -> (351613, 1171), (0, 1171), (0, 1171)), Zero rows selected for validation and test variables. So please provide the iloc index within the range and make sure you are getting correct rows count for test and validation. Also follow the steps in my answer as well. it will work for sure. Thanks $\endgroup$ Sep 23 at 16:20
  • $\begingroup$ The output (351613, 12) was incorrect due to incomplete file upload, Sorry for that. And the output for print(combi.shape) is (1051774, 12), also the train1.shape, test_val1.shape, test1.shape gives ((982644, 12), (34565, 9), (34565, 8)). And the iloc index starts from 982644, & I have also tried the steps in your ans as suggested, but it doesn't seems to work. Thanks for your time & help. $\endgroup$ Sep 23 at 18:09
  • $\begingroup$ Welcome and Thanks.. $\endgroup$ Oct 12 at 4:14

Not the answer you're looking for? Browse other questions tagged or ask your own question.