Google TPUs Break Into Meta’s AI Infrastructure

A tuned configuration of the vLLM inference engine now tops 700 tokens per second on a single user, a figure that highlights the growing speed of Google’s custom silicon. This performance benchmark, achieved on the seventh-generation Ironwood TPU, comes from Inferact, a startup that collaborated with Google to build the new software stack. The speed was measured while serving Moonshot AI’s open-weight Kimi K3 model, which has 2.8 trillion parameters. In the same test, the system outperformed a setup using 16 Nvidia GB200 chips, which reached 452 tokens per second.

The ability to achieve these speeds is the result of a strategic shift in how AI workloads are deployed. For years, running code on Google TPUs required using the company’s own JAX framework. That barrier has now been removed with the release of PyTorch-native versions of SGLang and vLLM. This change means developers can use their existing PyTorch codebases on Google hardware without extensive rewrites. The move is particularly significant because it allows companies to treat TPUs as a standard option rather than a niche alternative.

The path to this milestone has been gradual. From 2020 to 2023, Meta research teams used TPUs through an older bridge called PyTorch/XLA. However, those contracts were eventually canceled. The current integration represents a deeper collaboration, with Google co-developing the new TorchTPU stack alongside Meta’s PyTorch team. Zongwei Zhou, Meta’s PyTorch-TPU lead, noted that the engine provides a mature serving stack. He credited a three-way collaboration between Meta, Google, and the SGLang team for getting the system into production.

Jana van Greunen, director of PyTorch engineering at Meta, emphasized the strategic importance of this flexibility. She stated that Meta cannot be constrained by a single hardware architecture. By enabling the deployment of PyTorch workloads to Google TPUs, the company gains a critical alternative. Although the exact location of Meta’s production TPUs is not public, reports from last year suggested talks about renting Google Cloud TPUs in 2026, with potential purchases following in 2027. Meta has since reportedly signed a multi-year rental agreement.

This announcement arrives at a pivotal moment in the AI chip market. Just one day prior, Amazon open-sourced TorchNeuron, a similar native PyTorch backend for its Trainium chips. Both moves appear designed to loosen Nvidia’s software lock-in, which has long defined the industry. By making it easier to run standard PyTorch models on rival hardware, Google and Amazon are lowering the switching costs for large enterprises. This could force Nvidia to compete more aggressively on both performance and software ecosystem.

Looking ahead, the ecosystem plans to expand quickly. SGLang-TorchTPU will soon add support for DeepSeek V4.1 Flash and Reflection Beam. RadixArk, the startup behind SGLang, aims to provide day-zero support for new open models by the first quarter of 2027. This means that as soon as a new model is released, it should be ready to run on TPUs. Google, Meta, and RadixArk are scheduled to share further details at the SGLang Summit in San Francisco on November 12 and 13.

The performance gains are not limited to single-user scenarios. According to Inferact, the kernels for Kimi K3 and Qwen 3.8 27B deliver roughly 1.4 to 2 times the throughput of the GB200 baseline at small batch sizes, even without speculative decoding. This broader efficiency suggests that the new stack is becoming a viable primary option for many workloads. As these tools become more robust, the center of gravity in AI infrastructure may continue to shift away from a single dominant player.