0
$\begingroup$

For some reason, when using mps the dataloader is much slower (to a point in which its better to use cpu).

Any idea why?

enter image description here

Code for reproduction:

class Dataset(torch.utils.data.Dataset):
    def __init__(self, device):
        self.a = torch.tensor(1, device=device)
        
    def __len__(self):
        return 100
    
    def __getitem__(self, i):
        return self.a, self.a
    
for device in ['mps', 'cpu']:
    dataloader = torch.utils.data.DataLoader(Dataset(device), 64)
    %time next(iter(dataloader))

Thanks!

$\endgroup$

1 Answer 1

1
$\begingroup$

Depending on how MPS is set up (haven't done any Metal coding in awhile, so...), your data might have to be loaded from disk through cpu first then onto gpu. Note that this is just data loading time, if you're doing training and/or inference that's likely to be much faster on mps.

$\endgroup$
2
  • $\begingroup$ In a real model training takes pretty much the same with/without mps due to that overhead. Haven't set up anything really (just changed device). How would you check such thing? $\endgroup$
    – Ben Lahav
    Commented May 14, 2023 at 6:54
  • $\begingroup$ From jupyter notebook you can just use %time as you're doing above, but try training an actual model. It should be significantly longer than the 5ms you're seeing for the data loading, and in most cases mps should be much faster than cpu. $\endgroup$ Commented May 17, 2023 at 6:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.