Designed with large language model (LLM) inferencing in mind, each TPU boasts as much as 192 GB of high bandwidth memory (HBM) good for somewhere between 7.2 TB/s and 7.4 TB/s of bandwidth. Both are quoted in Google's launch announcement, one in the text and one in a graphic.

Google (NASDAQ:GOOG) (NASDAQ:GOOGL) unveiled the seventh-generation tensor processing unit chips, which will be the first designed specifically for inference. The new chip, which is called Ironwood, will come in two sizes based on AI workload demands: a 256 chip configuration and a 9,

Google unveils Ironwood, its 7th-generation TPU Ironwood is designed for inference, the new big challenge for AI It offers huge advances in power and efficiency, and even outperforms El Capitain supercomputer Google has revealed its most powerful AI training hardware to date as it looks to take another major step forward in inference.