sklearn package with AttributeError: 'MissingValues' object has no attribute 'to_list'

Question

I am currently trying to reproduce this tutorial on building a CNN based time series classifier for human activity recognition.

My setup is:

• Windows 10, Pycharm IDE with a new project for this tutorial, Python3.6, freshly installed the needed packages.

For reproducing, you need to download the activity data here and place it in the project directory under ./Data

The code executes the graphs well until this position:

df[LABEL] = le.fit_transform(df["activity"].values.ravel())

and throws following error:

Traceback (most recent call last):
  File "C:/Users/bobin/PycharmProjects/Mussel/cnn_musseltest.py", line 226, in <module>
    df[LABEL] = le.fit_transform(df["activity"].values.ravel())
  File "C:\Users\bobin\PycharmProjects\Mussel\venv\lib\site-packages\sklearn\preprocessing\_label.py", line 117, in fit_transform
    self.classes_, y = _unique(y, return_inverse=True)
  File "C:\Users\bobin\PycharmProjects\Mussel\venv\lib\site-packages\sklearn\utils\_encode.py", line 31, in _unique
    return _unique_python(values, return_inverse=return_inverse)
  File "C:\Users\bobin\PycharmProjects\Mussel\venv\lib\site-packages\sklearn\utils\_encode.py", line 133, in _unique_python
    uniques.extend(missing_values.to_list())
AttributeError: 'MissingValues' object has no attribute 'to_list'

Related threads that have not helped me so far: Link1 Link2

Answer 1

Not sure what version of the scikit-learn package you are using, but the following works without issues using version 0.24.1:

import pandas as pd
import numpy as np
import sklearn
from sklearn.preprocessing import LabelEncoder

print(sklearn.__version__)
# '0.24.1'

def read_data(file_path):
    column_names = ['user-id', 'activity', 'timestamp', 'x-axis', 'y-axis', 'z-axis']
    df = pd.read_csv(file_path, header=None, names=column_names)
    df['z-axis'].replace(regex=True, inplace=True, to_replace=r';', value=r'')
    df['z-axis'] = df['z-axis'].apply(convert_to_float)
    df.dropna(axis=0, how='any', inplace=True)
    return df

def convert_to_float(x):
    try:
        return np.float(x)
    except:
        return np.nan

df = read_data("WISDM_ar_v1.1//WISDM_ar_v1.1_raw.txt")

LABEL = "ActivityEncoded"
le = LabelEncoder()
df[LABEL] = le.fit_transform(df["activity"].values.ravel())

print(df.head())
# user-id activity       timestamp    x-axis     y-axis    z-axis  ActivityEncoded
#      33  Jogging  49105962326000 -0.694638  12.680544  0.503953                1
#      33  Jogging  49106062271000  5.012288  11.264028  0.953424                1
#      33  Jogging  49106112167000  4.903325  10.882658 -0.081722                1
#      33  Jogging  49106222305000 -0.612916  18.496431  3.023717                1
#      33  Jogging  49106332290000 -1.184970  12.108489  7.205164                1

Answer 2

The problem was with the pandas version. I've used Python3.6 as interpreter and pycharm would only let me install version 0.25 pandas. Now I am on Python3.8 and pandas 1.2.3 and the minimum code example provided by @Oxbowerce works now. I'll be more careful with checking the dependencies in future.

The sketch goes into the epochs now, there was just a minor labelling error:

Traceback (most recent call last):
  File "C:/Users/bobin/PycharmProjects/Mussel/cnn_musseltest.py", line 348, in <module>
    plt.plot(history.history['acc'], "g--", label="Accuracy of training data")
KeyError: 'acc'

Renaming 'acc' to 'accuracy' and val_acc to val_accuracy helped, as stated in the warning log.

WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy