Skip to content

Topic

Small-Model Fine-Tuning

Adapting open base models into narrow classifiers cheap enough to run inline with production workflows.

Current stories

buildOne report1 publisher

One 80 GB GPU holds 4-bit 70B weights and at most 11 conversations of 8,192 tokens

Serving a 70B model takes about 141 GB of GPU memory for weights at 16-bit, 73 GB at FP8 and 40 GB at 4-bit, according to a dev.to post. Fine-tuning takes about 50 GB with QLoRA and 1.1 TB with full AdamW. The post maps both onto GPU setups and hourly rental prices.

Publishers:dev.to

Reality

Evidence72
Adoption
Not scored
Hype gap+5
Incentives
Not scored
Confidence68
buildOne report1 publisher

Models trained on two clashing values let their answers overrule their own reasoning

Researchers who trained DeepSeek V3.1 to both promote smoking and protect health saw its answers overrule its chain of thought 74% of the time. Opus 4.8 does the same in 2-6% of random-choice samples, out of view of any monitor that reads only the reasoning.

Publishers:lesswrong.com

Reality

Evidence40
Adoption
Not scored
Hype gap+15
Incentives
Not scored
Confidence40
buildOne report1 publisher

Google Research steers frozen image models with a small add-on network

Google Research says the fully fine-tuned version of its Diffusion Controller posted a 90% win rate over the base image model. The lighter add-on meant for closed models is reported only as beating the industry standard for human preference.

Publishers:research.google

Reality

Evidence35
Adoption
Not scored
Hype gap+35
Incentives70
Confidence40
buildOne report1 publisher

Ambiguity in Rule 1.3 pushes children's voice app toward on-device design

Apple's Kids Category rule and COPPA's definition of a child's voice both point a preschool voice product at local processing, and Whisper's error on children's speech falls by nearly a factor of three from tiny to large-v3.

Publishers:dev.to

Reality

Evidence47
Adoption38
Hype gap+9
Incentives71
Confidence54
productOne report1 publisher

Red Hat traces the inference bill to 140GB of memory reads per token

Red Hat says 70 to 80 percent of enterprise AI spending goes to inference, and blames a 70-billion-parameter model reading all 140GB of its weights for every token it writes. Its fix needs a draft model you train yourself.

Publishers:redhat.com

Reality

Evidence55
Adoption40
Hype gap+20
Incentives85
Confidence45
buildOne report1 publisher

Google's ToolGrad back-writes the user query from a chain it has already executed

A Google XR team fine-tuned Gemma-3 models on data generated answer-first and says they match proprietary LLMs on tools they never trained on. The post's cost and pass-rate claims carry no numbers.

Publishers:research.google

Reality

Evidence30
Adoption10
Hype gap+40
Incentives60
Confidence35
productOne report1 publisher

Harvey buys Guardrails AI to test agents left working on legal tasks for hours

The $550mn round at $15.6bn gets the attention, but Harvey's fourth acquihire of the year bought a team whose product is testing how agents behave across a long run nobody watched. That is the part a buyer can use.

Publishers:thenextweb.com

Reality

Evidence45
Adoption66
Hype gap+20
Incentives78
Confidence48
leadershipOne report1 publisher

Fine-tuning hands the open-weight safety question to the buyer

The 270-company letter and Anthropic's warning are describing the same lever from opposite ends. Fine-tuning is what lifts a small open model past a frontier one, and it is also what can undo the safeguards shipped with it.

Publishers:forbes.com

Reality

Evidence26
Adoption
Not scored
Hype gap+32
Incentives72
Confidence28
productOne report1 publisher

Capsule puts a 71-millisecond judgment between an agent and the action it intends

The startup fine-tuned two Nvidia Nemotron models to allow, flag or block each action an agent intends, and reports 98% accuracy on a 9,429-trajectory benchmark whose own authors say accuracy is the wrong test.

Publishers:siliconangle.com

Reality

Evidence32
Adoption27
Hype gap+36
Incentives80
Confidence43