0
$\begingroup$

I noticed that after each time I execute the following lines of code, my results are different. Any idea why? I think that the main issue here is this line of code But I dont understand why?

test_data['prediction']=sentiment_model.predict_proba(test_matrix)[:,0]

What is the best way to add new column with .predict_prob()?

test_data['prediction'] = sentiment_model.predict_proba(test_matrix)[:,0]
test_data['prediction_label'] = sentiment_model.predict(test_matrix) 
test_data['prediction'] = test_data['prediction'].apply(lambda x: round(x,2)) 
test_data.sort_values(by='prediction', ascending=False, inplace=True) 
test_data[test_data['name'] == 'Britax Decathlon Convertible Car Seat, Tiffany']
test_data.head()
| improve this question | |
$\endgroup$
  • $\begingroup$ there may be randomness at some point along the process e.g. random initialization of weights and/or biases, dataset shuffling, ... $\endgroup$ – tagoma Jul 14 '17 at 19:05
  • $\begingroup$ You need to present more of the algorithm. E.g. the setup of the model, not just getting the model results. $\endgroup$ – Pieter21 Jul 16 '17 at 11:54
2
$\begingroup$

There is some randomness in the results from selecting/shuffling data that is used in the model.

If you don't want that, you could set a fixed random_state (seed) in your model.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Can you please go into details, the answer is a liitle bit unclear, How can i avoid it? Why it's happening ? $\endgroup$ – David Lerech Jul 16 '17 at 6:32
  • $\begingroup$ Sorry but it's not correct I use Jupyter and I execute this lines of code, So it's not an issue of spliiting the datqa, moreever i split the data with fix values for each item $\endgroup$ – David Lerech Jul 19 '17 at 6:51
1
$\begingroup$

My guess - the order of labels entered as training set is different.
From the docs -

The returned estimates for all classes are ordered by the label of classes.

So make sure you know which class prediction probability your slicing when using the [:, 0] in the first row.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.