"Turns out there are still people here who can think clearly." btw why yout think there is around 30%-50% advantage of TPU，I have never seen clear sources of the Comparation. hope to figure out
Internally to Google yes. Almost 2x better on Tokens per Watt, ergo 6c-10c per 1m tokens for TPUv7 vs 12c-18c for Blackwells. Many sources around, this is pretty standard benchmarking.
But this says nothing to the buyer / renter of TPUs...META won't pay for TPUs what Google pays...there's a take rate.
Another highly misleading claim from Google is regarding the power consumption of TPU v7. They did not provide the TDP of TPU v7; instead, they presented an exaggerated figure of 29.3 TFLOPS/W, and stated that the peak performance of TPU v7 is 4619 TFLOPS. From this, one can infer that they are implicitly suggesting a TPU v7 TDP of 157 watts.
From a technical standpoint, this is impossible. TPU v7 significantly increases per-die SerDes and HBM capacity. Structurally, from a chip architecture perspective, it is essentially no different from the NVIDIA B200. Yet Google is implying a TDP an order of magnitude lower than the B200. I am not sure what conceptual sleight of hand they are using here, but this chart is frequently cited in the market as evidence that TPU v7 has extremely low power consumption.
First, thank you for sharing the data on TPU costs.
Second, I agree with your point regarding TPU rental pricing. However, I believe there are additional issues that deserve attention but have not yet been incorporated into the discussion.
The topology, compilation model, and cluster scheduling mechanisms of TPU clusters are fundamentally different from those of GPU clusters. As a result, TPUs come with many constraints, which in turn force models from Anthropic, Meta, and OpenAI to undergo substantial re-adaptation. This adaptation is not a simple optimization or tuning exercise—it requires large-scale adjustments to the model architecture. In many cases, it is almost necessary to redesign the number of MoE experts, the number of MoE layers, and the routing logic of MoE experts. In addition, precision-related adaptations are required for quantization. Fortunately, TPU v7 finally supports FP8 quantization. However, Chinese AI labs such as DeepSeek and Kimi have already demonstrated that FP4 quantization can preserve model performance while significantly reducing compute. They have also demonstrated the feasibility of moving from linear attention to sparse attention, both of which further reduce inference cost. These techniques are difficult to apply on TPU v7 clusters—even assuming TPU v7 clusters reach meaningful deployment scale. (There is also the issue of Google’s insufficient assurance of its upstream supply chain. For example, T-Glass used in substrate manufacturing: expansion plans from Japanese suppliers have been delayed until 2027.)
For other AI labs migrating their models to TPU clusters, once all this adaptation work is done, they would still carry three major risks:
Changes in model architecture may introduce uncertainty in model performance.
The model architecture was not originally designed for TPU cluster topology, so its efficiency would be far below that of Gemini + TPU.
If the model architecture needs to evolve, TPU clusters may not be able to accommodate such changes. In other words, future architectural innovation could become locked in by the constraints of TPU clusters.
Finally, I believe the most critical issue is that both model architectures and inference methodologies are still evolving rapidly. These shifts drive changes in the compute characteristics required by training and inference clusters. Simply put, the demand profile for compute clusters is not converging—it is diverging rapidly.
I couldn’t agree more with your last point. About diverging. LLMs want dense matmults. MoE want dynamic routing— we can argue another time about mesh topologies being misunderstood, recommendation engines want giant embeddings, multi-modal wands long context (audio) and so on and so on.
But this isn’t about “replacing GPUs” - I haven’t even mentioned the lack of engineering talent to re-architect TPUs if people were to wholesale buy them.
Hence my view on NVDA above.
This is really about Google converging on model side and leading on ASIC side and arguably (still TBD) distribution side.
"Turns out there are still people here who can think clearly." btw why yout think there is around 30%-50% advantage of TPU，I have never seen clear sources of the Comparation. hope to figure out
Internally to Google yes. Almost 2x better on Tokens per Watt, ergo 6c-10c per 1m tokens for TPUv7 vs 12c-18c for Blackwells. Many sources around, this is pretty standard benchmarking.
But this says nothing to the buyer / renter of TPUs...META won't pay for TPUs what Google pays...there's a take rate.
Another highly misleading claim from Google is regarding the power consumption of TPU v7. They did not provide the TDP of TPU v7; instead, they presented an exaggerated figure of 29.3 TFLOPS/W, and stated that the peak performance of TPU v7 is 4619 TFLOPS. From this, one can infer that they are implicitly suggesting a TPU v7 TDP of 157 watts.
From a technical standpoint, this is impossible. TPU v7 significantly increases per-die SerDes and HBM capacity. Structurally, from a chip architecture perspective, it is essentially no different from the NVIDIA B200. Yet Google is implying a TDP an order of magnitude lower than the B200. I am not sure what conceptual sleight of hand they are using here, but this chart is frequently cited in the market as evidence that TPU v7 has extremely low power consumption.
First, thank you for sharing the data on TPU costs.
Second, I agree with your point regarding TPU rental pricing. However, I believe there are additional issues that deserve attention but have not yet been incorporated into the discussion.
The topology, compilation model, and cluster scheduling mechanisms of TPU clusters are fundamentally different from those of GPU clusters. As a result, TPUs come with many constraints, which in turn force models from Anthropic, Meta, and OpenAI to undergo substantial re-adaptation. This adaptation is not a simple optimization or tuning exercise—it requires large-scale adjustments to the model architecture. In many cases, it is almost necessary to redesign the number of MoE experts, the number of MoE layers, and the routing logic of MoE experts. In addition, precision-related adaptations are required for quantization. Fortunately, TPU v7 finally supports FP8 quantization. However, Chinese AI labs such as DeepSeek and Kimi have already demonstrated that FP4 quantization can preserve model performance while significantly reducing compute. They have also demonstrated the feasibility of moving from linear attention to sparse attention, both of which further reduce inference cost. These techniques are difficult to apply on TPU v7 clusters—even assuming TPU v7 clusters reach meaningful deployment scale. (There is also the issue of Google’s insufficient assurance of its upstream supply chain. For example, T-Glass used in substrate manufacturing: expansion plans from Japanese suppliers have been delayed until 2027.)
For other AI labs migrating their models to TPU clusters, once all this adaptation work is done, they would still carry three major risks:
Changes in model architecture may introduce uncertainty in model performance.
The model architecture was not originally designed for TPU cluster topology, so its efficiency would be far below that of Gemini + TPU.
If the model architecture needs to evolve, TPU clusters may not be able to accommodate such changes. In other words, future architectural innovation could become locked in by the constraints of TPU clusters.
Finally, I believe the most critical issue is that both model architectures and inference methodologies are still evolving rapidly. These shifts drive changes in the compute characteristics required by training and inference clusters. Simply put, the demand profile for compute clusters is not converging—it is diverging rapidly.
I couldn’t agree more with your last point. About diverging. LLMs want dense matmults. MoE want dynamic routing— we can argue another time about mesh topologies being misunderstood, recommendation engines want giant embeddings, multi-modal wands long context (audio) and so on and so on.
But this isn’t about “replacing GPUs” - I haven’t even mentioned the lack of engineering talent to re-architect TPUs if people were to wholesale buy them.
Hence my view on NVDA above.
This is really about Google converging on model side and leading on ASIC side and arguably (still TBD) distribution side.
Are those JPM research tables? Can you provide name of the report?
Those tables are mine. Images aren’t - they have source mentioned underneath. Anything with no source mentioned is mine.