Sparse Computing Empowers Efficient AI Computation

As artificial intelligence (AI) algorithms become increasingly complex and Moore’s Law continues to slow down, sparsification has emerged as a key technique for compressing neural network models and reducing computational workloads. However, a fundamental mismatch exists between the unstructured nature of sparse data and the structured characteristics of hardware circuits, leading to several challenges. (1) Structured sparsity methods cause large accuracy loss. (2) Dynamic sparse data is difficult to align with the fixed computation flow on general-purpose hardware (GPUs). (3) The fine-grained parallelism of sparse tasks conflicts with the coarse-grained parallelism of custom hardware (ASICs and FPGAs). To address these issues, the speaker proposes a multi-level “algorithm-compilation-hardware” co-optimization method, which includes knowledge-driven structured sparsity, machine learning-based dynamic compilation, and fine-grained parallel sparse architectures. At the algorithm level, a flexible structured sparsity method guided by domain knowledge is introduced, achieving 1.7× reduction in computation with negligible accuracy loss. At the compilation level, a machine learning-based dynamic compilation strategy is proposed, which adaptively selects optimal computation flows based on the characteristics of input sparse data, resulting in 1.3-3.8× improvement in GPU utilization. At the hardware level, a fine-grained parallel sparse hardware design is developed, featuring compressed redundant sparse data access and configurable sparse computing paths, achieving 1.6× improvement in custom hardware efficiency. Overall, this multi-level optimization approach improves effective computing performance by approximately an order of magnitude.