Code does not fully utilize the cpu or m1 gpu (stays around less than 5%) when running locally

[git-tyl]

Hi
I am running
"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/04_pytorch_custom_datasets.ipynb"
locally in vscode using the juypter extension on an m1 mac.

Once I get to "7.7 Train and Evaluate Model 0" the code runs extraordinarily slow

In google colab under cuda i can get around 7s
But when I run using 'cpu' or 'mps' (m1 max 32 gpu) the code runs really slow taking around 4 mins or so
When I look at the mac's activity monitor its around 2% cpu/gpu utilization

Conversely, if I run the code here locally in a juypter notebook in vscode
https://github.com/rasbt/machine-learning-notes/tree/main/benchmark/pytorch-m1-gpu (just paste the code in a notebook and remove the need for arguments and main when you reach main)
The gpu/cpu runs at 100% utilization, so I know there's definitely some major bottleneck in the current code

I've been trying to debug the code but I don't know enough of pytorch to make a good guess on why the code is not utlilizing the cpu/gpu fully. Maybe someone could help?

[haidary99]

Hi there!
I'm also facing a similar apple silicone issue, I have an m1 macbook air. In 01_pytorch_workfkow, after sending the model and the data to the GPU (mps) when running the training loop after setting up the device agnostic code of course, I'm getting an error (NotImplelemntedError) and it prompted me to use a flag (PYTORCH_ENABLE_MPS_FALLBACK=1) which I set before activating my environment. However, the error is still there even after I set the flag. I was wondering how you solved this problem since you also use apple silicone :)

[git-tyl]

Oh i had that but then upgraded to the nightly build instead of the stable build. BUT it seems that even if i use mps or the cpu it still is about the same speed. So my guess is even though I don't see any warnings anymore it still isn't using the mps gpu.

[haidary99]

damn what a bummer :( now I don't feel as bad that I can't utilize the GPU since there's not much speed difference. Thanks for the reply