Webinar

CASS-Wide Webinar XX: Design Issues for Mobile Processors with Large-Language Models (LLMs) and Large Multimodal Models (LMMs)

Presentation Menu

Abstract

Design Issues for Mobile Processors with Large-Language Models (LLMs) and Large Multimodal Models (LMMs)

Large-language models (LLMs) have achieved remarkable performance in many AI applications, but they require large parameter size in their models. The parameter size ranges from several billions to trillion parameters, and results in huge computation requirements on both training and inference. To fulfill huge computation requirements, Domain Specific Architecture is important to make trade-offs among different design parameters.

Large Multimodal Models (LMMs), also called Multimodal Large Language Models, integrate multiple data types as input. Multimodal information can provide rich and or environment information for LMMs to generate better user experience. LMM is also a trend for mobile devices, because mobile devices often connect with many sensors, such as video, audio, touch, etc.

Recently, there is a trend to run smaller LLMs/LMMs (near or less than 10 billion parameters) on edge device-side, especially for smartphones and PCs. Even LLMs/LMMs model size are reduced, they still require more computing resources than previous mobile processor workloads, and face challenges on memory size, bandwidth, and power efficiency requirements. Besides, device-side LLMs/LMMs in mobile processors can collaborate with cloud-side LLMs/LMMs in the data center to deliver better performance. They can off-load computing from cloud-side models to provide seamless response, or to become an agent to prompt cloud-side LLMs/LMMs, or be fine-tuned locally by user data to keep privacy.

Those LLMs/LMMs trends and new usage scenarios will shape future computing architecture design. In this talk we will discuss those issues, and especially their impacts on mobile processor design.

Description

This talk will take place on 8 May 2024 at 9:00 AM EDT (-4:00 UTC) and features a talk by Bor-Sung Liang, titled "Design Issues for Mobile Processors with Large-Language Models (LLMs) and Large Multimodal Models (LMMs)".

Registration for this series is entirely free and will be limited to the first 1,000 registrants per event. If you cannot register, you can also attend the webinar via LinkedIn Live. Following the webinar, the recording will be available on the CASS Resource Center and as a lesson in the CASS Microlearning (CASS MiLe) e-learning platform. In CASS MiLe, interested practitioners can learn through short didactic units (micro-lessons) with practical questions, and upon lesson completion, learners receive digital badges/certificates.

Register Now

Biography

Bor-Sung Liang is currently a Senior Director of Corporate Strategy & Strategic Technology, MediaTek, Hsinchu Science Park, Taiwan, and concurrently serving as a visiting professor at Department of Computer Science and Information Engineering, College of Electrical Engineering & Computer Science and Graduate School of Advanced Technology, National Taiwan University, as well as a Professor Ranked Specialist at Institute of AI Innovation, Industry Academia Innovation School in National Yang Ming Chiao Tung University. He is also the chair of IEEE CASS (Circuits and Systems Society) Taipei chapter.

He received his Ph.D degree from Institute of Electronics, National Chiao Tung University, and graduated from EMBA, College of Management, National Taiwan University. Dr. Liang has received several important awards, such as Ten Outstanding Young Persons, Taiwan, R.O.C.; National Invention and Creation Award for three times on Invention (one Gold Medal and two Silver Medals)  from Intellectual Property Bureau of the Ministry of Economic Affairs, Taiwan; Outstanding Youth Innovation Award of Industrial Technology Development of the Economic Affairs, Taiwan; Outstanding ICT Elite Award of ICT Month, Taiwan, R.O.C.;  and K. T. Li Young  Researcher  Award  from  Institute of Information & Computing Machinery and ACM Taipei/ Taiwan Chapter.