1
$\begingroup$

I have the code below in python to create LinearRegression model. When I train the model with re-sampled data, I get different values for its coefficients. I can't understand why that happens. Can you help me in this please?

[Update]

  • I assume that resampling is the same as shuffling. And that means the order of the data is changed but not the data itself.
  • In the use case presented, the number of rows are are the same as I inspected it and as I understand the order of the data is changed.

Thanks!

from sklearn.linear_model import LinearRegression
from sklearn.utils import resample

model = LinearRegression(fit_intercept=False)

model.fit(X, y)
print('model.coef_',model.coef_)

model.fit(*resample(X, y))
print('model.coef_',model.coef_)

model.fit(*resample(X, y))
print('model.coef_',model.coef_)
$\endgroup$
4
  • 1
    $\begingroup$ The resampled data is not identical to your original dataset (it is done with replacement by default). Why would you expect to get the same coefficients training on different data? They should be roughly similar, but not identical. $\endgroup$ – Nuclear Hoagie Jul 18 '18 at 17:41
  • $\begingroup$ @NuclearWang, resampling is just shuffling the existing data. So the only thing that changes is order of the data and not the data itself, right? $\endgroup$ – karthiks Jul 18 '18 at 17:44
  • 1
    $\begingroup$ No, by default the resampling is performed with replacement, so it's highly likely that your resampled data has repeated data points. If you did a full resampling without replacement, then yes, you'd have a shuffled version of your original data. $\endgroup$ – Nuclear Hoagie Jul 18 '18 at 18:16
  • $\begingroup$ @NuclearWang My bad in mis-reading scikit docs for shuffle() to assume it is an alias for resample(), without looking into the details. You explanation makes sense. Thanks! $\endgroup$ – karthiks Jul 18 '18 at 18:22
1
$\begingroup$

In each sampling, your data is going to be different from the previous sampling. For each sampling, you are going to find the best line which describes your sample with the least error value. Consequently, for each sample you are going to find a model which may be different due to the fact that it reduces the cost for each sample.

$\endgroup$
3
  • $\begingroup$ Sorry, I updated the question with more details with my assumptions and this use-case. I see that resample and shuffle are one and the same per scikit-docs. And so in this case I see that the row count remains the same as before and assume that only the order of the data is changed and not the data itself. Am I wrong here? $\endgroup$ – karthiks Jul 18 '18 at 17:50
  • $\begingroup$ Please add a reference to what you are saying. Shuffling is completely different than resampling. Just to show you what resampling means, consider that you have a population and you need 10 data samples each time out of your population which may consist of over 10 million samples. They may differ each time. shuffling means when you have some data, you change the order of them. I don't know whether these were your problems or not. $\endgroup$ – Media Jul 18 '18 at 18:05
  • 1
    $\begingroup$ Thanks, you made me do rework and I realized that resampling and shuffling are different. I misread scikit-doc for shuffle to think both are same. My bad! $\endgroup$ – karthiks Jul 18 '18 at 18:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.