Ph.D./Postdoc, Berkeley AI Research (BAIR), UC Berkeley                           Aug 2018 — Jun 2023

Advisor: Prof. Kurt Keutzer

Research on Hessian-AWare Quantization: HAWQ (ICCV’19), HAWQ-V2 (NeurIPS’20), ZeroQ (CVPR’20), HAP (WACV’22), Quantization Review (BLPCV’22), QD-BEV (ICCV’23), NoisyQuant (CVPR’23)

• Propose a Hessian-based method to decide mixed-precision configuration and block-wise fine-tuning order.
• Prove theorem to use the trace of Hessian as sensitivity metric and conduct fast Pareto frontier optimization.
• Generalize to segmentation, 2D/3D object detection tasks and achieve state-of-the-art results.
• Conduct fast end-to-end quantization without fine-tuning and without using any training/test data.

Research on HW-SW Co-design: HAWQ-V3 (ICML’21), CoDeNet (FPGA’21), HAO (FCCM’21), CSQ (DAC’23), EPIM (DAC’24)

• Achieve hardware-aware quantization and utilize 4-bit Tensor Cores for inference acceleration.
• Implement 4-bit kernels and mixed-precision on TVM, achieve 7.4x compression and 5.4x speedup against fp32.
• Propose efficient deformable op on embedded FPGAs, design new FPGA-core with ultra-low precision arithmetic.
• HW-SW joint architecture search and efficient implementation of mixed-precision NNs on CPU/GPU/FPGAs/PIM.

Research on Efficient LLMs and Diffusion Models: Q-BERT (AAAI’20), Q-Diffusion (ICCV’23), SqueezeLLM (ICML’24), PB-LLM (ICLR’24)

• Propose sensitivity-based non-uniform quantization and dense-and-sparse decompose for handling of outliers.
• Pioneer the usage of Hessian information to guide LLM quantization in both PTQ and QAT.
• Implement 3/4-bit CUDA kernels and achieve 4.6x compression compared to fp16 and 2.4x speedup on an A6000.
• Propose timestep-aware calibration and split shortcut quantize to achieve 4-bit diffusion models at the first time.

Research on Multi-agent Systems: MAgIC (EMNLP’24)

• Pioneer the integration of probabilistic graphical modeling (PGM) to enhance the cognitive abilities of LLMs.
• Present a framework to evaluate LLM-powered multi-agent systems by employing social deduction games.

Research on Image & Video Generative Models: PromptCoT (CVPR’24), ViewControl (IJCAI’24), D-Edit (AAAI’25), Meissonic, Magic-Me, VEditBench, K-Sort Arena

• Propose new methods to achieve better control ability of generative diffusion models.
• Develop novel efficient Arena algorithms for human-in-the-loop evaluation and alignment.
• Present Meissonic-1B that elevates masked image modeling (MIM) text-to-image models to SDXL-level.

Research on AI for Science: FastML (Frontiers in Big Data’22), High-Momentum Particle Trigger (TRETS’24)

• Review AI inference acceleration methods and how they help dark matter search, morphology characterization, etc.
• Implement efficient AI on ASICs and FPGAs to reduce time cost and enable particle trigger decisions at CERN LHC.

Research Intern, Bytedance AI Lab                                                                          Jan 2023 — Apr 2023

Research Intern, NVIDIA AI Lab                                                                                 May 2021 — Aug 2021

Research Intern, Facebook AI Research                                                               May 2020 — Aug 2020