CUDA 12.6 initialization error - PyTorch compatibility issue

[kadmy]

Environment:

• Ubuntu 24.04
• CUDA 12.6
• NVIDIA RTX A5000
• Driver 560.35.03

Issue:
When running with CUDA 12.6, getting error:
CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment

Attempted fixes:

1. Setting CUDA_VISIBLE_DEVICES explicitly - didn't help
2. Installing PyTorch with different CUDA versions - fails as PyTorch supports only up to 12.4

Need guidance on running Whisper-WebUI with CUDA 12.6 system installation.

[jhj0517]

Hi. Can you

nvcc -V

in the terminal and show what it prints out?

I'm using the CUDA 12.6 Windows build and it works fine.

I believe it should work fine on Ubuntu too, and make sure you have the proper CUDA installed for your OS:
https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_network

In Colab, it uses CUDA 12.2 on Linux, and it worked fine with both torch 2.5.0+cu121 and torch 2.5.1+cu124.