Anyone researching hardware for local AI meets the same recurring recommendation: choose NVIDIA. The advice is sound but it is rarely explained. The underlying reason is not the graphics cards themselves so much as a software platform that runs on them called CUDA. Understanding what CUDA is and why it matters so disproportionately clarifies two things. It shows why NVIDIA has become the default for running models at home and what exactly you sacrifice by choosing an alternative.
Why graphics hardware runs artificial intelligence
To understand CUDA it helps to first understand the problem it solves. A central processing unit (CPU) is built around a small number of powerful cores that execute instructions one after another at very high speed. A graphics processing unit (GPU) takes the opposite approach. It contains thousands of comparatively simple cores designed to perform many calculations simultaneously. This architecture was developed for rendering graphics, a task in which the same mathematical operation must be applied to millions of pixels at once.
The computation at the centre of a neural network turns out to have the same shape. Both training and inference are dominated by matrix multiplication, large grids of numbers combined according to a fixed pattern. Each element of the result can be computed independently, so the work distributes naturally across thousands of cores. This is the fundamental reason GPUs rather than CPUs perform the heavy lifting in modern AI. The hardware's parallelism matches the mathematics' parallelism almost exactly.
What CUDA actually is
A GPU is only hardware. On its own it has no notion of how to perform general-purpose mathematics rather than draw triangles. Some software layer must translate ordinary computation into instructions the chip can execute. CUDA(Compute Unified Device Architecture) is NVIDIA's platform for doing exactly that. Introduced in 2007 it provides a programming model and a set of libraries that let developers run conventional numerical work on NVIDIA GPUs without manually managing the underlying graphics pipeline.
Above the base platform sit specialised libraries tuned for particular workloads. The most relevant for AI is cuDNN, a library of highly optimised routines for the operations that neural networks invoke constantly. In practice CUDA functions as the bridge between AI software and the graphics card. When a tool such as Ollama or PyTorch or llama.cpp executes a model on an NVIDIA GPU it is issuing CUDA instructions underneath several layers of abstraction. The end user never writes or even sees this code but it is what makes the hardware usable.
How a programming platform became a competitive moat
CUDA's significance is as much economic as technical. By releasing it well over a decade before the current AI boom, NVIDIA ensured that an entire generation of researchers and frameworks was built against its platform first. This produced a self-reinforcing cycle that economists describe as a network effect. Researchers targeted CUDA because the mature tooling was already there, and the tooling improved precisely because nearly everyone targeted CUDA.
The consequence is an ecosystem that is difficult for competitors to displace regardless of their hardware's raw merits. Today compatibility with NVIDIA is the safe default assumption for any piece of AI software, and the great majority of projects support NVIDIA cards on the day they are released. Support for other vendors arrives later and with caveats when it exists at all. This accumulated advantage rather than a decisive lead in silicon is the principal reason the recommendation to "choose NVIDIA" persists.
The alternatives, and their real trade-offs
Competing platforms exist and are improving but each currently demands more effort or accepts a compromise:
- AMD (ROCm).AMD's equivalent platform is ROCm. Its cards can run local models and often provide more VRAM per dollar, which is a genuine advantage for fitting larger models. The cost is installation that remains rougher on Windows in particular and a subset of tools that lag behind their NVIDIA support.
- Apple Silicon (Metal and MLX).Apple's M-series processors use the Metal graphics API and the MLX framework. These systems share a single pool of memory between CPU and GPU, so a Mac configured with ample RAM can load surprisingly large models. That is a real and distinctive strength even though peak throughput trails high-end discrete NVIDIA cards.
Neither alternative is unusable and both are chosen deliberately by well-informed users. The point is simply that they require you to accept either additional setup friction or a different performance profile, whereas the NVIDIA path tends to work without negotiation.
What this means in practice
If you are assembling or evaluating a system the practical conclusions are narrow and reassuring. There is no need to learn CUDA or install it manually or understand its internals. Modern applications bundle the components they require. The decision it informs is only the choice of card. If your priority is the smoothest best-supported experience with the widest software compatibility, an NVIDIA GPU remains the path of least resistance. Once that is settled the single most important remaining specification is how much VRAM the card carries, since that determines which models will fit. For concrete recommendations by budget see the best GPUs for local LLMs and then confirm any specific build with the WillMyGPURunIt calculator.