NVIDIA pushes agentic AI onto local hardware

NVIDIA is putting more weight behind locally run AI, using a series of new models, software tools and hardware updates to make increasingly capable agents practical on PCs, workstations and compact systems rather than relying entirely on cloud infrastructure.

The company’s August announcements span models from NVIDIA and other developers, including Meta, DeepSeek, Poolside AI and Thinking Machines Lab. Taken together, they point to a broader shift in AI deployment. The question is no longer simply whether sophisticated models can run locally, but which workloads organisations may eventually choose to keep closer to users, applications and private data.

Meta’s new Muse Glimmer is one example. The 30-billion-parameter open weight model is designed for coding and local agentic AI and can run on a single NVIDIA RTX 5090. NVIDIA says it can process private files, documents, emails and messages locally, as well as handle credentials and execute multistep workflows.

Local inference gains strategic relevance

Keeping inference on the device changes some of the trade-offs around enterprise AI. Local agents can work with sensitive information without sending every interaction to a remote model, while potentially reducing dependence on cloud-based inference for persistent or high-volume workloads.

NVIDIA is also expanding its Nemotron family with Nemotron 3.5 Lightning, a customisable 30-billion-parameter mixture-of-experts model intended for always-on agents. The company says it can be fine-tuned for specialised tasks and deployed across RTX PCs, DGX Spark, Jetson, workstations, data centres and cloud environments.

Other models underline how quickly the boundaries of local computing are moving. DeepSeek V4 Flash has 284 billion parameters with 13 billion active parameters, while Thinking Machines Lab’s 276-billion-parameter Inkling-Small can run on a DGX Station or two DGX Spark systems. Poolside AI’s 118-billion-parameter Laguna S 2.1 can also run locally on a single DGX Spark using an optimised checkpoint.

Routing becomes part of the cost equation

NVIDIA is pairing those model developments with tools intended to manage where AI work is executed. NeMo Switchyard is an open source routing library that directs individual stages of an agent workflow to different models according to factors including accuracy, speed and cost.

In NVIDIA’s internal benchmarks, this approach maintained what the company describes as frontier-level task completion while reducing benchmark completion cost to roughly one-third of using Opus 4.8 alone.

That is significant because the economics of agentic AI are shaped not only by the cost of a single response, but by repeated tool calls and long-running workflows.

NVIDIA is also making it easier to combine DGX Spark systems through its Sync Cluster Assistant, increasing the memory and compute available for larger local models.

The broader direction is clear. Local AI is moving beyond small offline assistants towards agents capable of working across private data and applications for extended periods. The emerging architecture may therefore be less about choosing between cloud and local AI than deciding which parts of an increasingly complex workflow belong in each.