🔍 Read the full analysis: Running AI On TPUs, Natively: Meet Hugging Face ❤️ TorchTPU on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Hugging Face announced TorchTPU, a PyTorch backend that lets supported Transformers, Diffusers and Kernels workflows use Google TPUs through familiar device APIs. The company reports faster decoding for two models on a specific TPU setup, but has not supplied a full compatibility list or detailed benchmark methodology.
Hugging Face has announced TorchTPU, a PyTorch backend that lets supported workflows in its Transformers, Diffusers and Kernels libraries run on Google Tensor Processing Units. The release offers familiar ways to select a TPU device, while the company says it is still expanding testing and has not published a full compatibility list or detailed methodology for its performance claims.
Hugging Face’s announcement presents TorchTPU as a way to direct models and tensors to a TPU through existing PyTorch-based library interfaces. Its examples load a Transformers model with device_map=”tpu” and move a Diffusers pipeline with pipe.to(“tpu”). The company says compiled execution is available as an option, alongside integration with torch.compile.
The release also includes Pallas kernels tuned for TPUs and distributed training support. Hugging Face says large models can be split across chips during loading using a TensorParallelConfig. Its installation instructions target a TPU virtual machine and specify a CPU PyTorch wheel, rather than a CUDA build.
For selected decoding workloads, Hugging Face reports that kernels from its Kernels Hub and Tokamax achieved 21.5 times faster decoding for Qwen3-30B-A3B and 12.8 times faster for Gemma 4 26B-A4B at batch size 64 on a TPU v6e-8, compared with each model’s default implementation. The announcement also gives example image-generation timings on a TPU v6e-4. These are company-reported results; the supplied material does not include detailed measurement procedures or independent comparisons.
A Simpler Path to TPU Workloads
TorchTPU addresses a practical choice for teams developing with PyTorch: whether to evaluate Google’s TPU hardware without maintaining a separate, TPU-specific path for supported models. Familiar device selection in Transformers and Diffusers could reduce the amount of code adaptation needed to try that infrastructure. That may make it easier for developers to compare TPU capacity with other accelerators in their own environments.
That convenience does not establish that every PyTorch model or operation will work, or that TPUs will be faster or less expensive for a particular deployment. The published decoding comparisons concern two named models, one batch size and one TPU configuration. Real-world results could vary with model versions, workload, software setup, TPU generation and cost. Teams will need compatibility details and workload-specific tests before drawing conclusions about production use.
Google TPU compatible PyTorch models
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
From XLA to TorchTPU
Before this announcement, Hugging Face says PyTorch users generally accessed TPUs through PyTorch/XLA, also known as torch_xla. The company describes that route as workable, but says lazy tensor execution, graph recompilations after code changes and more difficult debugging could complicate development. Those are Hugging Face’s stated reasons for introducing a different backend; the announcement does not provide a comparative engineering study of the two approaches.
Hugging Face also previously offered Optimum TPU for inference and training on its models. The company says that package is no longer required to run Transformers and Diffusers models using the new native support. Google TPUs are specialized accelerators available through Google Cloud, ranging from individual chips to larger pods. TorchTPU’s stated role is to connect that hardware with Hugging Face’s PyTorch tools.
“With TorchTPU, a native PyTorch backend, TPU is now just another device.”
— Hugging Face
As an affiliate, we earn on qualifying purchases.
Compatibility and Results Still Open
The announcement does not list which models, operators or workloads are unsupported, and it does not establish that every model or operation in the named libraries will run on a TPU. It also provides no independent benchmark results, complete measurement procedures or cost comparisons for the reported speedups. The results should not be generalized beyond the named models and stated hardware and batch-size conditions.
Hugging Face says it is running the Transformers test suite against TPU support and working toward nightly TPU continuous integration. The announcement does not say when that testing will be available, what results will be published or how broad the eventual support coverage will be. Operational stability and performance across other workloads remain to be demonstrated.
As an affiliate, we earn on qualifying purchases.
Testing and Support Will Expand
Hugging Face says its next work includes continuing to run the Transformers test suite and adding nightly testing on TPU. Developers can consult the company’s Diffusers TPU guide for distributed and compiled usage, and follow the installation path for TPU virtual machines. The announcement gives no date for a broader compatibility matrix or for publication of further benchmark results.
As testing develops, developers and infrastructure teams will be able to judge how well the backend covers their models and whether the reported gains hold for their own workloads. Until then, TorchTPU establishes a new route for supported PyTorch workflows, while the breadth and practical performance of that support remain open questions.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is TorchTPU?
TorchTPU is a PyTorch backend that Hugging Face says lets supported PyTorch workflows use Google TPUs as a device.
Which Hugging Face libraries are named in the announcement?
Hugging Face names Transformers, Diffusers and Kernels. The announcement does not establish that every model or operation in those libraries is supported.
Does TorchTPU replace PyTorch/XLA?
Hugging Face presents TorchTPU as a native PyTorch backend and says its Optimum TPU package is no longer required for Transformers and Diffusers models using the new support. The announcement does not say that TorchTPU replaces PyTorch/XLA for every TPU workload.
How strong is the evidence behind the reported speedups?
The figures are Hugging Face-reported results for two models at batch size 64 on a TPU v6e-8, compared with each model’s default implementation. The supplied announcement does not include detailed benchmark methodology or independent results.
What should developers watch for next?
Hugging Face says it is expanding Transformers testing and working toward nightly TPU CI. A compatibility list, broader test results and additional benchmark details would help clarify which workloads are supported and how they perform.
Primary source: Hugging Face · via ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
