2
$\begingroup$

I am trying to use train_test_split to split my data. However, I am getting an index error. I pasted part of the error message below. I am using Python 3.5 version and sklearn 0.18.1. The code worked with my previous dataset that was different. Features here are in Pandas DataFrame and labels are in Pandas Series.

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(features, labels, test_size=0.4, random_state=1)

KeyError                                  
Traceback (most recent call last)
/apps/anaconda/anaconda-3.5/lib/python3.5/site-
    packages/pandas/indexes/base.py in get_loc(self, key, method, tolerance)
   2133             try:
-> 2134                 return self._engine.get_loc(key)
   2135             except KeyError:<br><br>
pandas/index.pyx in pandas.index.IndexEngine.get_loc (pandas/index.c:4443)()<br><br>
pandas/index.pyx in pandas.index.IndexEngine.get_loc (pandas/index.c:4289)()<br><br>
pandas/src/hashtable_class_helper.pxi in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:13733)()<br><br>
pandas/src/hashtable_class_helper.pxi in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:13687)()<br><br>
KeyError: 0
$\endgroup$
  • $\begingroup$ You seem to suggest problem comes from that new dataset. But, you just don't describe/show it. Are there partial strings in there? BTW your question might better fit with Stackoverflow. $\endgroup$ – tagoma Jul 28 '17 at 20:45
3
$\begingroup$

Pandas indexes differently:

X[some_slice]   # in Numpy is NOT equal to
df[some_slice]  # in Pandas, but is instead equal to
df.iloc[some_slice]

You can cast your features dataframe into numpy array by calling .values on them right before splitting:

X_train, X_test, y_train, y_test = \
    train_test_split(features.values, labels.values, test_size=0.4, random_state=1)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.