0
$\begingroup$

For some reason, when using mps the dataloader is much slower (to a point in which its better to use cpu).

Any idea why?

enter image description here

Code for reproduction:

class Dataset(torch.utils.data.Dataset):
    def __init__(self, device):
        self.a = torch.tensor(1, device=device)
        
    def __len__(self):
        return 100
    
    def __getitem__(self, i):
        return self.a, self.a
    
for device in ['mps', 'cpu']:
    dataloader = torch.utils.data.DataLoader(Dataset(device), 64)
    %time next(iter(dataloader))

Thanks!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Depending on how MPS is set up (haven't done any Metal coding in awhile, so...), your data might have to be loaded from disk through cpu first then onto gpu. Note that this is just data loading time, if you're doing training and/or inference that's likely to be much faster on mps.

Improve this answer
$\endgroup$
2
  • $\begingroup$ In a real model training takes pretty much the same with/without mps due to that overhead. Haven't set up anything really (just changed device). How would you check such thing? $\endgroup$
    – Ben Lahav
    May 14 at 6:54
  • $\begingroup$ From jupyter notebook you can just use %time as you're doing above, but try training an actual model. It should be significantly longer than the 5ms you're seeing for the data loading, and in most cases mps should be much faster than cpu. $\endgroup$
    – brewmaster321
    May 17 at 6:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.