Skip to content

Topic

Local AI Inference

Running language models on local or on-device hardware instead of remote cloud endpoints.

Current stories

productConfirmed4 publishers

Reviewers find Nvidia's RTX Spark well ahead on local AI and behind the top chips on CPU

Windows Central and Tom's Guide each tested Nvidia's RTX Spark for a week in one laptop model, the Surface Laptop Ultra. Six partners start shipping this week, Windows Central says, so a pilot on any other chassis goes beyond what either review covered.

Perspective Coverage

4 publishers
Builder
Builder 41%
Operator
Operator 38%
Investor
Investor 21%

Reality

Evidence64
Adoption
Not scored
Hype gap+20
Incentives45
Confidence68
buildConfirmed13 publishers

Muse Spark Open's license asks hosted-model sellers above $20M in revenue for a written deal with Meta

Meta released Muse Spark Open, an 816B-parameter open-weight model that activates 42B parameters per token. Per The New Stack, a company above $20M in revenue needs a separate Meta agreement before selling it as a hosted service.

Perspective Coverage

13 publishers
Builder
Builder 40%
Operator
Operator 32%
Investor
Investor 28%

Reality

Evidence55
Adoption30
Hype gap+35
Incentives65
Confidence65
buildConfirmed3 publishers

One early test puts a 120B model at 63 tokens per second on an RTX Spark laptop

NVIDIA's RTX Spark laptops went on sale Oct. 16 from $2,599, with up to 128GB of unified memory. A DEV Community roundup reports 63 tokens per second on a 120B model, but Microsoft's own 128K-prompt test shows a 170-second wait.

Perspective Coverage

3 publishers
Builder
Builder 42%
Operator
Operator 33%
Investor
Investor 25%

Reality

Evidence45
Adoption
Not scored
Hype gap+30
Incentives70
Confidence55
buildOne report1 publisher

Surface Laptop Ultra edges an M4 Max Mac Studio on local models, plugged in

Microsoft's $4,300 Surface Laptop Ultra ran local models generally a bit faster than an M4 Max Mac Studio, The New Stack's reviewer found. The edge was measured on the charger with models sized to the machine, so buyers should check their own model mix.

Publishers:thenewstack.io

Reality

Evidence50
Adoption
Not scored
Hype gap+15
Incentives
Not scored
Confidence55
productConfirmed27 publishers

Apple's M5 Ultra Mac Studio sells 512GB of memory, and the options list is the risk

Apple restored the 512GB configuration it spent last year deleting, and put the RAM shortage on the Mac mini's sticker instead. The ceiling is the product. The ceiling is also what disappears.

Perspective Coverage

28 publishers
Builder
Builder 30%
Operator
Operator 48%
Investor
Investor 22%

Reality

Evidence62
Adoption30
Hype gap+28
Incentives58
Confidence64
productOne report1 publisher

Microsoft builds agentic Windows 11 around free local AI and a $2,599 Surface laptop

Microsoft is turning Windows 11 into an agentic OS that uses free local models in place of paid cloud ones, with a $2,599 Surface Laptop Ultra as its flagship. What each seat's AI costs now depends on its hardware and on how many cloud tokens its user actually burns.

Publishers:theverge.com

Reality

Evidence45
Adoption
Not scored
Hype gap+35
Incentives65
Confidence40
investConfirmed46 publishers

Microsoft's Nvidia-powered Surface Laptop Ultra will start at $2,599 when it ships October 16

Microsoft's Surface Laptop Ultra starts at $2,599 and its Surface RTX Spark Dev Box costs $5,999, both shipping October 16. These are the list prices an IT leader can now put into a refresh budget for running large AI models on staff machines.

Perspective Coverage

46 publishers
Builder
Builder 36%
Operator
Operator 38%
Investor
Investor 26%

Reality

Evidence72
Adoption
Not scored
Hype gap+30
Incentives70
Confidence70
buildOne report1 publisher

Oki's $1,799 home AI computer leaves its privacy to the services owners connect

Oki is taking $59 reservations for Oki Home, a $1,799 computer that runs a 27-billion-parameter AI model on personal data at home. How much of that data stays in the house depends on which outside services each owner connects to it.

Publishers:runtimewire.com

Reality

Evidence40
Adoption
Not scored
Hype gap+20
Incentives65
Confidence40
productConfirmed5 publishers

Ghost asks $3,499 for a dedicated box that keeps personal AI agents running

Ghost opened pre-orders for Core, a $3,499 always-on computer for personal AI agents, after an $11 million seed round led by Andreessen Horowitz. It is a bet that people running agents at home will pay to skip configuring a machine they already own.

Perspective Coverage

5 publishers
Builder
Builder 34%
Operator
Operator 32%
Investor
Investor 34%

Reality

Evidence55
Adoption12
Hype gap+35
Incentives70
Confidence60
buildConfirmed12 publishers

NVIDIA's $4,999 DGX Spark 64GB is a better buy for local agents than for fine-tuning

NVIDIA's 64GB DGX Spark goes on sale from six PC makers on October 23 at $4,999, about $2,000 to $4,000 below in-stock 128GB units today. Two clustered units cost more than one 128GB machine, so the purchase makes most sense for agents on models that fit in 64GB.

Perspective Coverage

13 publishers
Builder
Builder 49%
Operator
Operator 35%
Investor
Investor 16%

Reality

Evidence62
Adoption
Not scored
Hype gap+30
Incentives68
Confidence60
buildOne report1 publisher

NVIDIA's RTX Spark PCs fit a 4-bit 70B model in 128GB with about 90GB to spare

NVIDIA's RTX Spark PCs from Lenovo and Acer ship in October with up to 128GB of shared CPU/GPU memory, enough for a 4-bit 70B model. Long-context agents now fit on owned hardware, though a cost comparison with cloud inference waits on system prices.

Publishers:dev.to

Reality

Evidence30
Adoption
Not scored
Hype gap+35
Incentives
Not scored
Confidence30
productOne report1 publisher

Nvidia adds a 64GB DGX Spark sized for 30B-class local models

Nvidia's DGX Spark 64GB ships October 23 from Acer, ASUS, Dell, Gigabyte, HP and MSI, with the same GB10 chip as the 128GB model and half its memory. Teams comparing it with cloud token bills now start by asking how much of their workload fits in 64GB.

Publishers:pcmag.com

Reality

Evidence35
Adoption
Not scored
Hype gap+40
Incentives75
Confidence40
buildOne report1 publisher

WorldScript Studio confines every AI failure to a single tested provider seam

WorldScript Studio's writing app stays fully usable with no API key, model or network because every AI feature reaches providers through one service. That leaves one module for a test suite to check when a provider goes down.

Publishers:dev.to

Reality

Evidence45
Adoption
Not scored
Hype gap+10
Incentives35
Confidence50
buildOne report1 publisher

Ollaya's latency table sets a local RTX 4090 against Jev's network round trip

Mert Cobanov's open-source Ollaya serves local decision models behind a Jev-compatible API and reports 8 to 10 ms per request on an RTX 4090. Its comparisons with TypeSafe's hosted service are uncontrolled, so teams have to check accuracy on their own traffic before switching.

Publishers:runtimewire.com

Reality

Evidence38
Adoption
Not scored
Hype gap+25
Incentives60
Confidence35
productOne report1 publisher

RTX 50-series prices rising up to 39% put resale value into GPU refresh budgets

Used GPUs with plenty of VRAM keep 40 to 60 percent of their value while RTX 50-series prices climb as much as 39 percent, Engadget reports. For a team planning a refresh, the old card's sale price belongs in the budget, though few models sell above list.

Publishers:engadget.com

Reality

Evidence35
Adoption
Not scored
Hype gap+30
Incentives
Not scored
Confidence40
buildConfirmed7 publishers

Apple's local-AI Mac Studio starts at $5,499, and the price that matters is still blank

The M5 Ultra is pitched as an alternative to CUDA workstations, but the 512GB configuration that carries the argument ships in late October with no announced price.

Perspective Coverage

7 publishers
Builder
Builder 45%
Operator
Operator 31%
Investor
Investor 24%

Reality

Evidence60
Adoption20
Hype gap+35
Incentives70
Confidence65
buildConfirmed5 publishers

NVIDIA PAIR schedules each subagent call onto whichever LAN box already holds the model

PAIR proxies Ollama and LM Studio, so the agent keeps seeing one connection and no harness code changes. The adoption cost moves to disk, because a node is only eligible if it already has the exact model downloaded.

Perspective Coverage

7 publishers
Builder
Builder 45%
Operator
Operator 38%
Investor
Investor 17%

Reality

Evidence62
Adoption
Not scored
Hype gap+30
Incentives72
Confidence60
productConfirmed9 publishers

Apple charges $4,000 to move the Mac Studio from 96GB to 256GB of unified memory

Apple's M5 Ultra Mac Studio starts at $5,499 and reached $12,299 as tested, and the reviewer running local agents on it every day says the most expensive Anthropic subscription would still have cost less for years.

Perspective Coverage

10 publishers
Builder
Builder 36%
Operator
Operator 38%
Investor
Investor 26%

Reality

Evidence82
Adoption41
Hype gap+18
Incentives58
Confidence76
productOne report1 publisher

Asus asks $1,350 for a 16GB mini PC built to run agentic AI locally

Lenovo's ThinkCentre X Ultra starts at $3,700 in November and the cheapest cluster-capable Mac mini is $1,700. The desk-side inference box is now priced like a workstation. Memory decides which one you need.

Publishers:gizmodo.com

Reality

Evidence45
Adoption20
Hype gap+38
Incentives70
Confidence50
productOne report1 publisher

Nvidia's PAIR hands the spare family PC a night shift running sub-agents

Nvidia's Personal AI Router, shown at IFA 2026, farms an agent's subtasks out to whichever household machines happen to be asleep, which buys speed on unattended jobs at the cost of any promise about when they finish.

Publishers:siliconangle.com

Reality

Evidence32
Adoption12
Hype gap+30
Incentives68
Confidence42

Earlier coverage