Topic
Reduced-precision formats (e.g., FP8, GGUF, INT4) for model weights and KV caches that shrink memory use and speed inference at some cost to fidelity.
Nvidia launched RTX Spark laptops on Friday with up to 128GB of shared memory and a 74-billion-parameter model it says fits in 24GB. A local agent pilot is now possible on Windows, though every figure so far comes from Nvidia.
Reality
Nemotron 3.5 Super VL, NVIDIA's 120B model, had GGUF, MLX, FP8 and EXL3 conversions on Hugging Face a day after its October 15 release. The roughly 128 GB memory figure comes from a DEV Community write-up whose author did not run the model.
Reality
The Neuron measured NVIDIA's Nemotron 3.5 Bolt 74B-A9B at 65.5 tokens per second on a laptop in a hands-on test. The build was heavily compressed, and the review's tool-assisted wins went to the hosted Super VL.
Reality
Microsoft's $4,300 Surface Laptop Ultra ran local models generally a bit faster than an M4 Max Mac Studio, The New Stack's reviewer found. The edge was measured on the charger with models sized to the machine, so buyers should check their own model mix.
Reality
NVIDIA says its Nemotron 3.5 Bolt 74B-A9B agent model fits in 24GB and will ship with open weights, data and recipes. The footprint, the license terms and the quality claims stay unverified until the weights are live and outside testers have run them.
Perspective Coverage
12 publishersReality
Meta's Muse Spark Open, an 816B-parameter coding model, is now on SageMaker JumpStart in BF16 and 4-bit NVFP4 variants. The NVFP4 build needs roughly a quarter of the memory, and memory capacity decides which instance you rent.
Reality
Black Forest Labs' open FLUX.3 Image Klein weights now load in Hugging Face Diffusers for self-hosted image generation and editing. Hugging Face times the distilled checkpoint at about 2 seconds per 1024 x 1024 image on an H100, and The Decoder reports an INT8 build for 8 GB cards.
Perspective Coverage
3 publishersReality
PyTorch's ExecuTorch Cortex-M backend ran four MLPerf Tiny models at 21.5 ms or less each, using under 90 KiB of RAM on a single Cortex-M55 core. The test is the PyTorch team's own, on one Alif Ensemble E8 board, so the figures hold for similar cores and INT8 models.
Reality
BAAI's AREX-2 config declares one multi-token-prediction layer, but its weight index lists 0 such tensors against Qwen3.8-27B's 15. Its dev.to testers say the base-model drafter and quant need rechecking too, though they have not run AREX-2.
Reality
Serving a 70B model takes about 141 GB of GPU memory for weights at 16-bit, 73 GB at FP8 and 40 GB at 4-bit, according to a dev.to post. Fine-tuning takes about 50 GB with QLoRA and 1.1 TB with full AdamW. The post maps both onto GPU setups and hourly rental prices.
Reality
NVIDIA and Princeton's Dao AI Lab cache SSM inputs instead of state, targeting a decode step where SSM layers are the largest component up to 100K tokens. ReplaySSM is already integrated into TensorRT-LLM, vLLM and FlashInfer, NVIDIA says.
Reality
SoloQ, a calibration-free quantizer for diffusion LLMs, cuts inference FLOPs about 45%, according to a dev.to post on an arXiv paper dated 2026-10-05. That write-up is the only source, and its cost numbers come from a scenario with an unnamed provider.
Reality
Tencent has open-sourced its Hy-MT2 translation models for 33 languages, with the smallest compressed to 440 MB to run on devices. The claim that the small model beats Microsoft's API is Tencent's own, so teams paying per call have a candidate to test.
Reality
Microsoft plans to run MAI Code 1.1 Flash on developers' machines in GitHub Copilot, with a limited rollout starting by end-October 2026. Its figures come from one high-end laptop and local pricing is undisclosed, so teams cannot yet size the hardware or the bill.
Reality
VIDRAFT released a 111GB, 4-bit build of its 180B-parameter Darwin model that it says runs on a laptop with 8GB of VRAM and 32GB of RAM. Air-gapped teams still need a measured token rate on that laptop before they can price it against a server.
Reality
Strata runs a 125B-class Qwen model at about 100 tok/s on one RTX 4090 because its sparse MoE activates only about 6B parameters per token. The design moves the hardware bill to system RAM, and the 100 tok/s figure holds mainly for code and structured output.
Reality
NVIDIA's RTX Spark PCs from Lenovo and Acer ship in October with up to 128GB of shared CPU/GPU memory, enough for a 4-bit 70B model. Long-context agents now fit on owned hardware, though a cost comparison with cloud inference waits on system prices.
Reality
Salvatore Sanfilippo's open-source ds4 engine runs a short list of large open-weight models locally by compressing their routed experts to about two bits. Even compressed, the supported builds need high-memory Macs or GPU systems that most people do not already own.
Reality
VIDRAFT says POCKET-Darwin-180B, a 111 GB 4-bit GGUF build of its 180B mixture-of-experts model, runs in llama.cpp on about $1,400 of consumer hardware. The accuracy evidence so far is one MMLU-Pro comparison that VIDRAFT reports itself.
Reality
Helion's autotuned GEMM beat vLLM's default CUTLASS and DeepGEMM backends on Hopper GPUs, by more than 10% throughput on some workloads, its authors report. The gain rests on per-shape tuning that can run for hours, a cost teams pay in place of kernel maintenance.
Reality
Build · October 1, 2026 · One report1 publisher
Build · September 30, 2026 · One report1 publisher
Build · September 30, 2026 · One report1 publisher
Build · September 27, 2026 · One report1 publisher
Build · September 26, 2026 · One report1 publisher
Build · September 23, 2026 · One report1 publisher
Build · September 22, 2026 · One report1 publisher
Leadership · September 22, 2026 · One report1 publisher
Build · September 18, 2026 · One report1 publisher
Build · September 17, 2026 · Confirmed2 publishers
Build · September 17, 2026 · One report1 publisher
Build · September 17, 2026 · One report1 publisher
Build · September 16, 2026 · One report1 publisher
Build · September 16, 2026 · One report1 publisher
Build · September 16, 2026 · One report1 publisher
Build · September 15, 2026 · One report1 publisher
Build · September 13, 2026 · One report1 publisher
Build · September 13, 2026 · One report1 publisher
Build · September 11, 2026 · Confirmed2 publishers
Build · September 11, 2026 · One report1 publisher
Build · September 10, 2026 · One report1 publisher
Build · September 9, 2026 · One report1 publisher
Build · September 8, 2026 · One report1 publisher
Build · September 6, 2026 · One report1 publisher
Build · September 5, 2026 · One report1 publisher
Build · August 22, 2026 · One report1 publisher
Build · August 19, 2026 · One report1 publisher
Build · August 17, 2026 · One report1 publisher
Build · August 16, 2026 · One report1 publisher
Build · August 15, 2026 · One report1 publisher