PyTorch 'RuntimeError: CUDA error: no kernel image is available for execution on the device', or the warning 'NVIDIA ... with CUDA capability sm_XX is not compatible with the current PyTorch installation'. How do I fix it (new or old GPU)?

What it is: PyTorch's CUDA builds contain GPU code compiled for a specific list of GPU architectures (compute capabilities such as sm_80 for A100 or sm_89 for RTX 40-series). If your GPU's architecture isn't in that list, CUDA has no kernel it can run and you get 'no kernel image is available for execution on the device'; recent PyTorch versions also warn at startup that the GPU's capability isn't compatible with the installed build. It happens in two directions: a brand-new GPU used with a PyTorch build made before support for it was added (or one built against an older CUDA version), or an old GPU whose architecture newer PyTorch releases no longer include. The same applies to other compiled extensions (flash-attn, bitsandbytes, xformers, custom CUDA ops) built without your architecture. Am I affected / how to check: - python -c "import torch; print(torch.__version__, torch.version.cuda); print(torch.cuda.get_device_capability()); print(torch.cuda.get_arch_list())" - Compare your GPU's capability (e.g. (8, 6) means sm_86) with the get_arch_list() output. NVIDIA's CUDA GPUs page lists each GPU's compute capability. - If PyTorch itself works but an extension fails, the extension is the one missing your architecture. Fix: 1. New GPU: install a newer PyTorch build that supports it, using the selector on pytorch.org/get-started/locally and choosing a CUDA version new enough for your GPU (pip install torch --index-url https://download.pytorch.org/whl/<cuda-tag>). Update the NVIDIA driver too, since newer CUDA builds need newer drivers. 2. Old GPU: use the newest PyTorch release whose get_arch_list() still includes your architecture (check its release notes), or build PyTorch from source with TORCH_CUDA_ARCH_LIST set to your capability. 3. Extensions: reinstall them after changing PyTorch so they're rebuilt for your GPU; when building from source, set TORCH_CUDA_ARCH_LIST (e.g. "8.6") to include your capability. 4. In containers, make sure the image's PyTorch/CUDA version supports the host GPU, not just the machine where it was built. 5. To find the failing call, run once with CUDA_LAUNCH_BLOCKING=1 so the error points at the right line. Verify: torch.cuda.get_arch_list() includes your GPU's sm_XX, the startup warning is gone, and a test like torch.ones(2, device='cuda') * 2 runs. Sources: https://pytorch.org/get-started/locally/ https://docs.pytorch.org/docs/stable/generated/torch.cuda.get_arch_list.html https://docs.pytorch.org/docs/stable/generated/torch.cuda.get_device_capability.html https://developer.nvidia.com/cuda-gpus