AI Computing Design Trends for LLMs in the Generative AI Era

Large Language Models (LLMs) have demonstrated exceptional performance across numerous generative AI applications, but they require significant computation for both AI training and inference. The growth rate of these computational requirements significantly outpaces advancements in semiconductor process technology. Consequently, innovative IC and system design techniques are essential to address challenges related to computing power, memory, bandwidth, energy consumption to meet AI computing needs. In this talk, we will explore the evolution of LLMs in the generative AI era and their influence on AI computing design trends. We will discuss the evolution of LLMs, from focusing on training, focusing on inference, and then agentic AI and physical AI to explore Ai capability beyond single LLM. About AI computing design trends, we will discuss how scale-up and scale-out techniques to affect AI computing performance and energy efficiency, and different strategies to design Ai computing chips for data center and edge devices. These trends will significantly shape the design of future computing architectures and influence the advancement of circuit and system designs.