Microsoft Maia 300 Explained: Inside the AI Chip Built to Cut Microsoft’s Reliance on Nvidia

On October 12, 2026, Microsoft unveiled Maia 300, the third generation of its in-house AI accelerator, and paired it with a strategic commitment: starting in 2027, the Azure Maia family will scale to multi-gigawatt deployments and become “the primary inference system for Microsoft products and services” (Microsoft).

The announcement is one of the clearest signs yet that the largest buyers of Nvidia hardware are becoming its competitors. This analysis covers what Maia 300 is, why it targets inference rather than training, the architecture details Microsoft has disclosed, the performance claims and their caveats, where the program stands today, and what it means for Microsoft, Nvidia and the wider AI chip market.

Key Takeaways

• What it is: Maia 300 is Microsoft’s third-generation custom AI accelerator, designed as a complete system (silicon, racks, networking, software and datacenter) for AI inference (Azure).
• When: Production deployments begin in 2027 (Microsoft).
• Claims (Microsoft’s): 40% more tokens per megawatt than leading GPU-based systems, and up to 7x the rack-scale token throughput of Maia 200 in a full 96-accelerator rack (Microsoft).
• Key specs: compute chiplets on TSMC N2P, eight HBM4 stacks with 288 GB and 18 TB/s per accelerator, 950 W TDP, 96 accelerators in a 167 kW rack, and superpods of up to 12,288 accelerators (Azure).
• Today: Maia 200 already serves Microsoft Copilot, with GitHub Copilot and Microsoft Foundry to follow by year-end (Microsoft).
• Nvidia still matters: Microsoft continues to train models with Nvidia and AMD hardware (CNBC).

What Is Microsoft Maia 300?

Maia 300 is the third generation of the Azure Maia program, following Maia 100 (introduced in 2023) and Maia 200 (introduced in January 2026) (CNBC). Microsoft describes it as “a complete intelligent system” designed for the AI workloads now emerging: long-context reasoning, latency-sensitive conversational experiences, and sustained, multi-step agentic tasks (Microsoft).

The design philosophy, according to the Azure engineering team, has been consistent since the program began: “optimize the entire system for LLM inference so that every watt and every dollar delivers more intelligence.” With Maia 300, Microsoft co-designed software, silicon, physical rack infrastructure, networking and datacenters “to operate as a single platform” for cloud-scale agentic inference (Azure).

Unlike Google’s TPUs or Amazon’s Trainium and Inferentia, Maia accelerators cannot be rented by the hour by outside companies; they serve Microsoft’s own services (CNBC).

Inference vs. Training: Why Maia 300 Is an Inference Chip

AI computing splits into two broad phases:

• Training builds a model by processing enormous datasets, a phase where Nvidia remains dominant (Axios).
• Inference runs a finished model to answer user requests, such as summarizing a call or answering a research question (Bloomberg).

Microsoft calls Maia 300 an “inference-first architecture” optimized for long-context, memory-intensive and decode-heavy workloads (Microsoft). “We prioritize Maia for inference,” Rani Borkar, president of Azure Hardware Systems and Infrastructure, told Axios, describing inference as the largest and fastest-growing opportunity in AI computing and an area where Microsoft can differentiate on cost and efficiency (Axios).

The focus is strategic. Axios notes that most Nvidia competitors are concentrating on inference, “where margins are highest,” and that inference demand is climbing as more sophisticated models and agents spread (Axios). Agentic workloads, including multi-step execution, long-context decoding and tool use, also change the computational profile of inference, putting more pressure on memory capacity, bandwidth and data movement than on raw compute (Azure).

Deployment Timeline: Production in 2027, Multi-Gigawatt Scale Over Time

Production starts in 2027

Maia 300 “will begin production deployments in 2027” (Microsoft). Microsoft plans to share more at Microsoft Ignite 2026 in San Francisco next month (Azure).

Multi-gigawatt plans

Microsoft says it will scale the Azure Maia family to multi-gigawatt deployments starting in 2027 (Microsoft). Reuters reported that Microsoft plans to deploy “potentially more than a million” Maia 300 chips next year to support its internal AI services (Reuters via Euronext). Microsoft’s own posts do not give a chip count.

The ramp will take time

Microsoft is hedging on pace. Alistair Speirs, a general manager, declined to give a timeline for reaching multi-gigawatt scale: “We reserve a little bit of optionality there, because this is an industry that’s changing very fast, and we want to deploy the right silicon for the right models that we’re using at the right time as well” (CNBC). Borkar likewise told Axios the shift “won’t happen overnight” and that reaching multi-gigawatt scale will extend beyond next year (Axios).

Evercore ISI analysts estimated that Maia will represent about 10% of Microsoft’s AI chip spending by the first quarter of 2027 (CNBC). For scale, Microsoft has said it expects to spend more than $175 billion in capital expenditures this calendar year (Reuters via Euronext).

Microsoft’s Performance Claims and Their Caveats

40% more tokens per megawatt than leading GPU systems

Microsoft says it expects Maia 300 to deliver 40% more tokens per megawatt compared to leading GPU-based systems, crediting efficiency gains across the whole system (Microsoft). Microsoft did not name the GPU systems used for the comparison (Reuters via Euronext).

Power is the metric that matters here. “You can think of power translating pretty directly into operational cost for something like a data center,” Speirs said (CNBC).

Up to 7x rack-scale throughput vs. Maia 200

In a full 96-accelerator configuration, Microsoft says Maia 300 delivers up to 7x the rack-scale token throughput of Maia 200 (Microsoft). The Azure deep dive adds that the rack delivers roughly 2.5x the performance per watt of Maia 200, about 2x the HBM bandwidth per watt, and twice the accelerators per rack (Azure).

What the claims don’t tell us

• All figures are Microsoft’s own. “Nobody has tested them outside of Microsoft yet,” said Patrick Moorhead, CEO of Moor Insights & Strategy (CNBC).
• The GPU baseline is unnamed (Reuters via Euronext).
• The 7x figure compares full racks, and the Maia 300 rack holds twice as many accelerators as Maia 200’s (Azure).
• Microsoft’s blog posts do not state Maia 300’s absolute compute (for example, petaFLOPS at FP4).

Maia 300 Architecture: A Deep Dive

The details below come from the Azure engineering deep dive by Rani Borkar and Saurabh Dighe, and from Microsoft’s announcement (Azure; Microsoft).

Software-defined dataflow (SDLA)

Maia 200 introduced what Microsoft calls a Software Defined Locally Accessed Dataflow Architecture (SDLA). It separates control-path programming from data-path execution so software can explicitly coordinate where data is placed, moved, converted and synchronized across local memories, DMA engines, compute engines and the network. The premise: storing, moving and converting data, not raw compute, is the central bottleneck in large inference systems (Azure).

Maia 300 extends SDLA in three ways:

1. Memory organization moves from uniform memory access (UMA) to a locality-aware non-uniform (NUMA) layout across chiplets, so software can keep data near the compute that uses it.
2. New narrow-precision formats with hardware-accelerated data-type conversion cut the bytes moved per token.
3. Scale doubles to 96 accelerators per rack, and the scale-up domain grows to a 12,288-accelerator superpod.

Microsoft also cites AI-assisted kernel optimization working alongside the dataflow design to maximize locality and utilization (Microsoft), and ISA enhancements in the vector engine for key inference kernels such as softmax and low-precision data casting (Azure).

Multi-chiplet design on TSMC N2P

Maia 300 uses a multi-chiplet architecture that separates compute, memory-controller and I/O functions. Compute chiplets are built on TSMC’s N2P process, where the advanced node gives the biggest performance-per-watt return. Memory-controller and I/O functions sit on dies optimized for bandwidth, connectivity and cost (Azure). Reuters reports those other chiplets use cheaper 3-nanometer technology, a different path from Nvidia’s approach of one large compute die surrounded by memory (Reuters via Euronext).

Microsoft says this strategy increases system compute FLOPs by 36%, adds 2.5x HBM bandwidth, and provides 14% more scale-up bandwidth (Azure).

Memory: eight HBM4 stacks, 288 GB, 18 TB/s

Each accelerator pairs software-managed on-die SRAM with eight HBM4 stacks providing 288 GB of capacity and 18 TB/s of peak bandwidth, all within a 950-watt TDP. Compared with Maia 200, that is 2.5x the memory bandwidth, 33% more capacity, and about 2x the HBM bandwidth per watt (Azure). Specialized kernels use the on-die SRAM as a scratchpad to reduce pressure on HBM (Microsoft). Microsoft describes the result as “industry-leading HBM bandwidth per watt.”

Low-precision formats: enhanced MXFP4 and sub-3-bit storage

Maia 300 introduces enhanced MXFP4 and sub-3-bit storage formats. Narrower number formats reduce the memory footprint and data movement for both computation and KV-cache storage, which lets the system support larger models, longer context windows and more concurrent sessions within the same power envelope (Azure; Microsoft).

Networking: one scale-up fabric from rack to superpod

Maia 300 treats the network as an extension of its dataflow architecture. An integrated AI network controller runs a custom Ethernet-based AI transport, and a two-tier network builds superpods of up to 12,288 accelerators (Azure):

• Half-switched in-rack topology: direct links connect the two accelerators in a pair within a tray, while a switched fabric connects all 96 accelerators in a rack (a “pod”). All in-rack links are electrical to save power and cost.
• Optical second tier: second-tier switches connect up to 128 racks into a superpod.
• One fabric, not two: the same communication semantics apply inside the pod and across the superpod, so accelerators can talk to any peer without involving the host or a discrete scale-out NIC.
• Three latency domains: a directly connected pair (lowest latency), one switch hop within a pod, and three switch hops across pods.
• Configurable bandwidth: the split between in-rack and cross-rack bandwidth is set per job; an accelerator can drive up to 750 GB/s to any peer in the superpod when only a subset of accelerators go cross-rack.
• Flexible placement: tensor parallelism stays within a rack while expert and pipeline parallelism can span racks, reducing stranded capacity.
• Disaggregated inference: prefill and decode workers can be sized and placed independently, with KV state moving directly over the scale-up fabric.
• Resilience: schedulers can use hot-spare capacity in another rack, transparently to software kernels.

Rack design: 96 accelerators in 167 kW

The Maia 300 rack integrates 96 accelerators across 24 four-accelerator trays within 167 kW, double Maia 200’s accelerator density. Each tray includes its own host processor, memory, storage and cooling, eliminating separate host trays (Azure). A full rack delivers 27.6 TB of HBM capacity and 1.7 PB/s of aggregate HBM bandwidth (Microsoft). For context, Reuters notes Nvidia’s latest GB300 racks hold 72 GPUs and draw about 142 kW (Reuters via Euronext).

Switching is consolidated into six high-radix switch ASICs across three liquid-cooled switch trays (Azure).

Power, cooling and telemetry

• Power: dual power feeds, high-capacity power shelves and a 50 V busbar, plus an integrated rack automatic transfer switch that keeps the rack available when one feed is offline for servicing.
• Cooling: liquid cooling removes more than 80% of the rack’s thermal load (accelerators, voltage regulators and switch ASICs), while air cools memory, storage and other components. Racks can connect directly to facility water or use a heat-exchanger unit in air-cooled datacenters.
• Telemetry: rack- and row-level power telemetry feeds datacenter operations, so algorithms can dynamically allocate power and optimize workload placement and density. Microsoft also highlights improved reliability, availability and serviceability (RAS).

(Source for this section: Azure.)

Maia 300 vs. Maia 200 at a glance

Sources: Microsoft (Maia 200), Azure (Maia 300).

Maia 200: Where the Program Stands Today

From Maia 100 to Maia 200

Microsoft introduced its first AI chip, Maia 100, in 2023 (CNBC). Maia 200, announced on January 26, 2026, is built on TSMC’s 3nm process with more than 140 billion transistors, and delivers over 10 petaFLOPS at FP4 and over 5 petaFLOPS at FP8 within a 750 W envelope. Microsoft claimed three times the FP4 performance of Amazon’s third-generation Trainium and FP8 performance above Google’s seventh-generation TPU (Microsoft).

Maia 200 launched in the US Central region near Des Moines, Iowa, with US West 3 near Phoenix next, alongside a Maia SDK preview that includes PyTorch integration, a Triton compiler, an optimized kernel library, the low-level NPL language, and a simulator and cost calculator (Microsoft). In July, CEO Satya Nadella told analysts Maia 200 “delivers 30% better performance per dollar than the latest generation hardware in our fleet” and was handling OpenAI models (CNBC).

Current use: Copilot, GitHub Copilot and Foundry

As of October 2026, Microsoft says (Microsoft):

• Maia 200 is serving production workloads, including Microsoft Copilot, and supporting the MAI team as it develops next-generation models.
• GitHub Copilot and Microsoft Foundry will follow by the end of the year.
• Across these workloads, Maia 200 runs OpenAI models, Microsoft’s own MAI models and leading open-source model families.
• Maia 200 is deploying across the United States, Europe and Australia, delivering up to 1.7x more tokens per watt for built-in AI in Microsoft Teams, Copilot and Cowork compared with leading GPU infrastructure.

Together, Maia 200 and Maia 300 are meant to support agentic coding in GitHub, model serving in Microsoft Foundry, inference and data generation for the MAI team, and latency-sensitive workflows in Dragon Copilot for clinicians. Jim Kleewein, a Microsoft technical fellow, said parts of Microsoft 365 Copilot should speed up with Maia 300 (CNBC).

The Big Tech Custom-Chip Race

Microsoft’s move is part of a broader shift in which “every major Nvidia customer is now a competitor,” seeking lower costs, better performance and more control (Axios).

Google TPU

Google has built Tensor Processing Units for years. Its eighth generation, announced in April 2026, splits into two chips for the first time: TPU 8t for large-scale training and TPU 8i for inference and reinforcement learning, with 288 GB of HBM and 384 MB of on-chip SRAM (Google Cloud). Google Cloud customers can rent TPUs (CNBC). Google is even testing TPUs in orbit, as we covered in Google Just Put AI Chips in Space: Project Suncatcher Explained Simply.

Amazon Trainium and Inferentia

AWS sells two custom families: Inferentia for inference and Trainium, which AWS now positions for both training and inference (AWS). Trainium customers include Anthropic, and OpenAI has committed to using Amazon’s custom silicon (Axios). AWS says Trainium4 is designed to support Nvidia’s NVLink Fusion interconnect (Amazon), and CEO Andy Jassy has said it arrives in 2027 (CRN).

Meta MTIA

Meta’s MTIA program has four new generations (MTIA 300, 400, 450 and 500) that are deployed or scheduled for 2026–2027, with MTIA 450 and 500 focused on generative-AI inference (Meta). Meta plans to start deploying MTIA 450 in the first half of 2027, with MTIA 500 following at the end of 2027 (Bloomberg). Meta still buys substantial capacity from Nvidia (Axios).

OpenAI and Anthropic

OpenAI is developing its Jalapeño chip with Broadcom, initially for inference, and Anthropic confirmed in August that it is exploring its own chip designs (Axios).

Where Nvidia and AMD Fit

Nvidia is not being replaced. Nvidia GPUs fill Microsoft’s existing datacenters (CNBC), and for model training, Microsoft partners with Nvidia and AMD, Speirs said. Microsoft also recently announced a Surface laptop with an Nvidia processor (CNBC). “For us, it’s not an either-or strategy,” Borkar said. “We want a broad ecosystem while investing deeply in our first-party silicon” (Axios).

Three further limits apply:

• Scope: Making Maia the primary inference platform applies to Microsoft’s own services, not necessarily to third-party Azure customers such as OpenAI (Axios).
• Demand: Nvidia’s revenue more than doubled to $96 billion in its second quarter, a sign that overall demand leaves room for everyone (Axios).
• Supply chain: Custom-chip makers depend on the same limited manufacturing, advanced packaging and high-bandwidth memory capacity as Nvidia (Axios).

What Maia 300 Means

For Microsoft: cost and margins

Power is the binding constraint on AI datacenters, and Microsoft’s pitch is more useful output from every megawatt. CFO Amy Hood said last month that using its own chips has helped with “margin improvement over the course of a decade” (CNBC). Owning the inference stack, from silicon to systems to agents, is central to what Microsoft calls its “Frontier Transformation” strategy (Microsoft).

For Nvidia: less centrality, not less demand

Axios argues that even a small erosion of Nvidia’s position is notable, because Nvidia increasingly uses its balance sheet to fund the AI buildout and depends on staying central to it (Axios). Near term, however, Maia is an inference play inside Microsoft, and training remains on Nvidia and AMD.

For users and Azure customers

Users may see faster and cheaper AI features in Copilot, GitHub Copilot, Teams and Foundry. But Maia is not sold as rentable capacity, so Azure customers will keep choosing among GPU-based options for their own workloads (CNBC).

Open questions

• How Maia 300 performs in independent testing against named GPU systems.
• How quickly Microsoft reaches multi-gigawatt scale amid tight supplies of HBM and advanced packaging.
• Whether Microsoft will ever offer Maia capacity to external customers.
• What further details emerge at Ignite 2026.

FAQ: Microsoft Maia 300

What is Microsoft Maia 300?

Maia 300 is Microsoft’s third-generation custom AI accelerator, announced on October 12, 2026. It is designed as a complete system for AI inference, covering silicon, racks, networking and software, and will power Microsoft’s own AI products and services.

When will Maia 300 be deployed?

Microsoft says Maia 300 will begin production deployments in 2027. Reaching multi-gigawatt scale is expected to take longer, extending beyond 2027.

Is Maia 300 for AI training or inference?

Inference. Microsoft calls it an inference-first architecture for long-context, memory-intensive and agentic workloads. Microsoft continues to train models with Nvidia and AMD hardware.

How does Maia 300 compare with Nvidia GPUs?

Microsoft claims Maia 300 delivers 40% more tokens per megawatt than leading GPU-based systems, but it has not named the systems it compared against, and the chip has not been tested independently.

What are the key Maia 300 specs?

Compute chiplets are built on TSMC N2P. Each accelerator has eight HBM4 stacks with 288 GB and 18 TB/s of bandwidth within a 950 W TDP. A rack holds 96 accelerators in 167 kW, and superpods scale to 12,288 accelerators.

Where is Maia 200 used today?

Maia 200 serves Microsoft Copilot in production and supports Microsoft’s MAI team, with GitHub Copilot and Microsoft Foundry to follow by the end of 2026. It runs OpenAI, Microsoft MAI and open-source models.

Can businesses rent Maia chips on Azure?

Not currently. Unlike Google’s TPUs or Amazon’s Trainium and Inferentia, Microsoft’s Maia accelerators cannot be rented by the hour by other companies.

Conclusion: A Strategic Bet on Inference Economics

Maia 300 is less a single chip than a statement of intent. Microsoft has designed an inference system from the transistor up to the rack and the datacenter, and it plans to make that system the default engine for its own AI services starting in 2027. If the claimed efficiency gains hold up in production, Microsoft gets more AI output from every megawatt it can secure. Nvidia keeps its central role in training and remains a key partner, but it faces another major customer building its own alternative.

The next markers to watch are independent benchmarks, the pace of the 2027 ramp and any further disclosures at Ignite.

👉 Follow us on Instagram for clear, accurate tech news and analysis. Follow on Instagram

Sources

• Microsoft (Scott Guthrie), “Microsoft to expand Azure Maia system to multi-gigawatt deployments,” Oct 12, 2026: https://blogs.microsoft.com/blog/2026/10/12/microsoft-to-expand-azure-maia-system-to-multi-gigawatt-deployments/
• Microsoft Azure (Rani Borkar, Saurabh Dighe), “Deep dive into Azure Maia 300 architecture,” Oct 12, 2026: https://azure.microsoft.com/en-us/blog/deep-dive-into-azure-maia-300-architecture/
• CNBC (Jordan Novet), Oct 12, 2026: https://www.cnbc.com/2026/10/12/microsoft-will-mainly-use-its-maia-chips-for-ai-inference.html
• Axios (Ina Fried), Oct 13, 2026: https://www.axios.com/2026/10/13/big-tech-ai-chips-nvidia-rivals
• Reuters (Stephen Nellis, Max A. Cherney), Oct 12, 2026, via Euronext: https://live.euronext.com/en/financial-news/microsoft-plans-huge-deployment-its-own-ai-chips-eye-saving-costs
• Bloomberg (Matt Day), Oct 12, 2026: https://www.bloomberg.com/news/articles/2026-10-12/microsoft-prepares-to-put-gigawatts-of-its-chips-in-data-centers
• Microsoft, “Maia 200: The AI accelerator built for inference,” Jan 26, 2026: https://blogs.microsoft.com/blog/2026/01/26/maia-200-the-ai-accelerator-built-for-inference/
• Google Cloud, “AI infrastructure at Next ’26”: https://cloud.google.com/blog/products/compute/ai-infrastructure-at-next26/
• AWS Trainium: https://aws.amazon.com/ai/machine-learning/trainium/
• Amazon, Trainium3 UltraServer announcement: https://www.aboutamazon.com/news/aws/trainium-3-ultraserver-faster-ai-training-lower-cost
• CRN, Andy Jassy on Trainium4: https://www.crn.com/news/cloud/2026/andy-jassy-on-aws-244b-backlog-trainium4-and-ai-chips-strategy
• Meta, “Four MTIA Chips in Two Years”: https://ai.meta.com/blog/meta-mtia-scale-ai-chips-for-billions/
• Bloomberg, Meta MTIA, Sep 15, 2026: https://www.bloomberg.com/news/articles/2026-09-15/meta-touts-the-cost-saving-benefits-of-latest-in-house-ai-chips

Photo Credits

Microsoft images are used for editorial purposes under Microsoft's usage terms; Wikimedia Commons photos are used under their licences (credit required).

• Azure Maia 300 (hero image): Image: Microsoft, editorial use
• Azure datacenter aisle: Image: Microsoft, editorial use
• Azure Maia 300 prototype chip: Image: Microsoft, editorial use
• Azure Maia 200 hardware: Image: Microsoft, editorial use
• Nvidia DGX B200 systems: Pokiiri, Wikimedia Commons, CC BY-SA 4.0
• NVIDIA Headquarters: Coolcaesar, Wikimedia Commons, CC BY-SA 4.0