avatar

Shang Yang

Ph.D. Student
MIT EECS
Cambridge, MA
shangy [at] mit [dot] edu


Shang Yang

I am a final-year Ph.D. student at HAN LAB of MIT EECS, advised by Prof. Song Han. My research focuses on machine learning systems, particularly model compression, LLM serving, and LLM post-training. Recently, I am activately working on the efficient inference and post-training of LLM Agents.

News

Selected Publications

  1. ASPLOS
    Taming the Long-Tail: Efficient Reasoning RL Training with Adaptive Drafter
    Qinghao Hu*, Shang Yang*, Junxian Guo, Xiaozhe Yao, Yujun Lin, Yuxian Gu, Han Cai, Chuang Gan, Ana Klimovic, Song Han
    The 31th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2026.
    Code

  2. CVPR
    ForeAct: Steering Your VLA with Efficient Visual Foresight Planning
    Zhuoyang Zhang*, Shang Yang*, Qinghao Hu, Luke J. Huang, James Hou, Yufei Sun, Yao Lu, Song Han
    The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026.
    Code CVPR Highlight

  3. MLSys
    LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention
    Shang Yang*, Junxian Guo*, Haotian Tang, Qinghao Hu, Guangxuan Xiao, Jiaming Tang, Yujun Lin, Zhijian Liu, Yao Lu, Song Han.
    The Eighth Annual Conference on Machine Learning and Systems (MLSys), 2025.
    Code

  4. MLSys
    QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
    Yujun Lin*, Haotian Tang*, Shang Yang*, Zhekai Zhang, Guangxuan Xiao, Chuang Gan, Song Han.
    The Eighth Annual Conference on Machine Learning and Systems (MLSys), 2025.
    Code

  5. MLSys
    AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration
    Ji Lin*, Jiaming Tang*, Haotian Tang†, Shang Yang†, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, Song Han.
    The Seventh Annual Conference on Machine Learning and Systems (MLSys), 2024.
    Code Best Paper Award

  6. MICRO
    TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUs
    Haotian Tang*, Shang Yang*, Zhijian Liu, Ke Hong, Zhongming Yu, Xiuyu Li, Guohao Dai, Yu Wang, Song Han.
    56th IEEE/ACM International Symposium on Microarchitecture (MICRO), 2023.
    Code

Blogs

  1. TinyChat 2.0: Accelerating Edge AI with Efficient LLM and VLM Deployment
    Explore the latest advancement in TinyChat - the 2.0 version with significant advancements in prefilling speed of Edge LLMs and VLMs. Apart from the 3-4x decoding speedups achieved with AWQ quantization, TinyChat 2.0 now delivers state-of-the-art Time-To-First-Token, which is 1.5-1.7x faster than the legacy version of TinyChat.
    Code

  2. TinyChat: Visual Language Models & Edge AI 2.0
    Explore the latest advancement in TinyChat and AWQ - the integration of Visual Language Models (VLM) on the edge! The exciting advancements in VLM allows LLMs to comprehend visual inputs, enabling seamless image understanding tasks like caption generation, question answering, and more. With the latest release, TinyChat now supports leading VLMs such as VILA, which can be easily quantized with AWQ, empowering users with seamless experience for image understanding tasks.
    Code

  3. TinyChat: Large Language Model on the Edge
    Running large language models (LLMs) on the edge is of great importance. In this blog, we introduce TinyChat, an efficient and lightweight system for LLM deployment on the edge. It runs Meta's latest LLaMA-2 model at 30 tokens / second on NVIDIA Jetson Orin and can easily support different models and hardware.
    Code


© Copyright 2024 Shang Yang. Powered by Jekyll and Minimal Light theme.