Does whisper not run on the GPU?

[Vovin155]

I have an RTX 3090 and it's simply not recognizing it for some reason and using my CPU instead.

[dilacerated]

CUDA 12.1 or higher needed.

https://github.com/chidiwilliams/buzz/blob/main/CONTRIBUTING.md#gpu-support

[Vovin155]

I've updated to 12.1 but the speed of transcribing seems no different and my CPU is nearly at full peak with my GPU idle.

[dilacerated]

I'm coming around to try this myself but am nursing a torn tendon...

To be clear you are using Faster Whisper and you did install cuBLAS and cuDNN?

cuBLAS probably easier to get through the CUDA Toolkit: https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64&target_version=10

[raivisdejus]

Faster whisper needs cuDNN and cuBLAS (https://developer.nvidia.com/cublas) that right now does not have the installer for Windows on nvidia site, but regular Whisper should use GPU on any CUDA version. With GPU the speed is +/- similar.

Next release will have necessary GPU support on Windows included. To get it right now you can download latest development build from here https://github.com/chidiwilliams/buzz/actions/workflows/ci.yml?query=branch%3Amain
To see the artifact download button you need to be logged in to the Github.

[hackeranmol]

I am using this new version ? Do i stilll need to install cuda 12 on my pc or these included binaries are enough

[hackeranmol]

And there are so many components. Do i have to install all of them

[raivisdejus]

From the components you need only Runtime for sure. The rest may be optional.

I am not 100% sure if installing CUDA manually is needed, it is bundled with Buzz, so you can try to run it without installing CUDA manually, if it does not work, then install CUDA manually.

Also I recommend you to try Whisper, Faster Whisper and Huggingface whisper, each of them may behave differently.

On my system I have CUDA installed manually, GPU is used, but it may work also with modules that come bundled with the Buzz

[raivisdejus]

And a note. On windows Faster whisper will use the CPU as some necessary libraries are not available from Nvidia side, but regular Whisper and Huggingface whisper will use GPU just fine.