Microsoft expects Maia 300 to become its primary AI inference system

Microsoft expects Maia 300 to deliver 40% more tokens per megawatt.

Microsoft unveiled Maia 300, its next AI inference platform, on October 12.

Microsoft plans to begin multi-gigawatt Azure Maia deployments in 2027.

Microsoft claims up to seven times Maia 200's rack-scale token throughput in a full configuration of 96 accelerators.

Independent benchmarks would need to account for differences in model configuration and request volume, as well as how power consumption was measured.

Microsoft has identified future GitHub coding workloads and model serving through Microsoft Foundry as potential uses for its Maia infrastructure.

Microsoft also references built-in AI capabilities across Teams and Copilot, including Cowork experiences.

Microsoft's January Maia 200 announcement identified Microsoft 365 Copilot and Foundry as intended beneficiaries.

An agent that researches a topic or debugs code may call an AI model repeatedly before completing an assignment, and each additional request consumes computing resources.

A coding agent asked to fix a bug might inspect several files before proposing a change, then run tests and consult the model again to interpret failures; each additional step consumes tokens and can increase the time needed to finish.

Microsoft's documentation on agent-based retrieval describes how repeated reasoning steps increase token consumption and latency.

If employees delegate lengthy assignments to agents throughout the workday, Microsoft needs sufficient capacity to process those requests alongside conventional chatbot interactions.

A 40% gain in tokens per megawatt is 1.4 times the output at the same power.

Higher throughput could allow a rack to handle more simultaneous requests, although individual responses might take roughly the same amount of time.

Additional throughput could reduce request backlogs when numerous Copilot users run agents at once; whether users experience faster responses depends on factors including model processing time and network delays.

If the seven-times rack throughput and the 40% tokens-per-megawatt figures share a Maia 200 baseline, rack power would be about five times higher.

Greater efficiency does not necessarily reduce total energy consumption; as Microsoft expands infrastructure and supports more agent activity, overall electricity demand could continue to rise.