For Local AI in Production viewers
Local AI in Production runs language models and automations on its own machines instead of a cloud API. These are the parts that decide what such a setup can run, with what to check before you buy each one.
This guide describes kinds of products and what to check before you buy them. It links to no store, and StackPilot earns nothing from it.
We do not quote memory figures here, and we do not test or rank products. Look up the size of the model you want to run in the Ollama library or on its model card, then compare it with the memory listed for the part.
Start from the model you want to run, then work down to the part: every point links to the matching section below.
More buying guides: PerfectLoop finds: what to check first, Voice gear for narration and clips, AI books and reading subscriptions
Video memory (VRAM) decides which models fit entirely on the graphics card, and a model that fits runs much faster than one split between the card and system memory. Compare the card's VRAM with the size of the model files you plan to run; this search looks for 16 GB cards, so confirm the VRAM on each listing. We have not tested these products.
Ollama and llama.cpp can keep the parts of a model that do not fit on the graphics card in system memory, so more RAM lets a larger model load, though it runs more slowly. Check which memory type, speed and capacity your motherboard supports before you order a kit. We have not tested these products.
Model files are large and every model you keep takes its full size on disk, so storage fills up quickly once you compare a few. A fast NVMe drive also shortens the wait while a model loads into memory. We have not tested these products.
A small always-on box can serve a local model to the rest of your network for automations that run around the clock. Most mini PCs have no separate graphics card and run models from system memory, so check the maximum RAM and the cooling before you choose one. We have not tested these products.
The Jetson Orin Nano is NVIDIA's small developer board for running AI at the edge, next to a camera or sensor. It suits small models and vision projects rather than the largest language models. We have not tested these products.
A Raspberry Pi 5 is a low-cost way to try small models and self-hosted tools on Linux. Expect only small models to run at a usable speed; a kit with the official power supply and a case saves a second order. We have not tested these products.
Hands-on books on building with language models cover prompting, retrieval and evaluation in more depth than a Short can. Check the publication date, because the tools in this field change quickly. We have not tested these products.
Page last edited . We do not list prices or ratings here, because they change.