Taking on CUDA with ROCm: 'One Step After Another'

Article Summary

AMD’s ROCm platform is making incremental but meaningful progress in challenging NVIDIA’s CUDA dominance, with dedicated teams operating as internal startups to accelerate development. The effort involves significant complexity — packaging dozens of dependencies and a heavily customized LLVM toolchain — but engineers report the runtime is now compiling successfully with open-source toolchains.

Discussion

• ROCm still doesn’t support all of AMD’s own AI-capable GPUs, and the AMDGPU Linux driver has had instability issues since kernel 6.6
• Vulkan backends are emerging as a potential leapfrog: llama.cpp via Vulkan is already faster than ROCm in many inference scenarios, potentially making ROCm obsolete for inference before it catches CUDA
• The competitive landscape is fragmenting beyond AMD vs NVIDIA: Intel’s OpenVINO/SYCL stack gaining traction on Arc and Core Ultra iGPUs
• Some argue AI-assisted CUDA-to-open-platform translation could accelerate AMD adoption and erode NVIDIA’s moat

Discuss on HN