2
$\begingroup$

I'm coding a custom (but simple) dataset for training an MLP with PyTorch. Basically, my data are numeric vectors of two classes so the whole dataset is a pandas.DataFrame object whose first column is 0 or 1 (representing both classes) and the rest of columns are the numeric component of the vectors. Since each row represents a data, I consider that a natural PyTorch's dataset class would be:

class MatrixDataset(Dataset):
    def __init__(self, data: pd.DataFrame, device):
        self.data = data
        self.device = device

    def __len__(self):
        return self.data.shape[0]

    def __getitem__(self, ind):
        row = self.data.iloc[ind]
        x = torch.tensor(row.iloc[1:], dtype=torch.float32, device=self.device)
        y = torch.tensor(row.iloc[0], dtype=torch.float32, device=self.device)
        return x, y

Actually, this Dataset works fine, but I receive the following warning

/path_to_my_script/script.py: FutureWarning: Series.__getitem__ treating 
keys as positions is deprecated. In a future version, integer keys will 
always be treated as labels (consistent with DataFrame behavior). 
To access a value by position, use `ser.iloc[pos]`   
x = torch.tensor(row.iloc[1:], dtype=torch.float32, device=self.device)

After searching about this warning, I still don't get why it triggers because as many people say, the warning is self-explanatory: 'use .iloc method' but I'm already using it. Furthermore, when I try to replicate the error using Python's interpreter in terminal, it doesn't trigger.

Any help would be appreciated it!

$\endgroup$

1 Answer 1

1
$\begingroup$

The warning is showing up not because of how you indexed the row; it's appearing because you're passing a Series object to a tensor constructor. The following (where no indexing happens) would trigger the same warning:

s = pd.Series([1,2,3], index=['a', 'b', 'c'])
torch.tensor(s, dtype=torch.float32)

To silence the warning, pass the underlying numpy array to the tensor constructor:

        x = torch.tensor(row.iloc[1:].values, dtype=torch.float32, device=self.device)
        #                            ^^^^^^^   <--- underlying array
$\endgroup$
2
  • $\begingroup$ Thanks a lot! I came up with that solution after a while, but I thought that working with the numpy array would be less efficient than working with the serie. Do you know if that happens? On the other hand, we could guess that the tensor constructor index the serie 'internally' and it is not up-to-date? $\endgroup$ Mar 9 at 9:49
  • $\begingroup$ @leapofFaith If you're talking about memory, if row is float dtype (which I assume it is since you cast it to torch.float32), .values returns a view, so no issues there. If you're talking about runtime speed, working with the numpy ndarray would almost certainly be faster than the Series. $\endgroup$
    – cottontail
    Mar 13 at 4:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.