ZHU LEI (祝磊)

I got my PhD degree from CityUHK under the supervision of Prof. Rynson W.H. Lau in July, 2024. Before that, I received my bachelor degree in Computer Science and Engineering from Dalian University of Technology (rank 1/102) and spent one year as PhD student at CUHK under the supervision of Eric Lo.

My research focus on hardware-efficient architecture design, LLM inference acceleration, and (vision)-language foundation models. Before finalizing PhD research topic on effective attention mechanisms, I actively explored several topics such as 3D vision, data-efficient learning and low-level vision. Through all these trials, I have built a good understanding on GPUs, deep learning systems, and strong skills on experiment automation. During the journey, I published 8 papers in top venues (including 4 first-author ones), and earned 750+ citations (640+ from first-author papers).

News

Oct 30, 2024	I’m pleased to receive NeurIPS travel award and be selected as Top Reviewers.
Sep 26, 2024	One first-author paper on vision backbone is accepted by NeurIPS 2024.
Jul 02, 2024	I passed my oral defense and became a Dr.
Jul 02, 2024	One co-authored paper (Analytic-Splatting) is accepted by ECCV 2024.
May 16, 2024	RelayAttention got accepted by ACL 2024 main conference.

Selected Publications

ACL

RelayAttention for Efficient Large Language Model Serving with Long System Prompts

Lei Zhu, Xinjiang Wang, Wayne Zhang, and 1 more author

In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2024

arXiv Bib Code

@inproceedings{zhu2023relayattention,
  title = {RelayAttention for Efficient Large Language Model Serving with Long System Prompts},
  author = {Zhu, Lei and Wang, Xinjiang and Zhang, Wayne and Lau, Rynson WH},
  booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2024},
}

CVPR

BiFormer: Vision Transformer with Bi-Level Routing Attention

Lei Zhu, Xinjiang Wang, Zhanghan Ke, and 2 more authors

In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023

arXiv Bib Code Poster

@inproceedings{zhu2023biformer,
  title = {BiFormer: Vision Transformer with Bi-Level Routing Attention},
  author = {Zhu, Lei and Wang, Xinjiang and Ke, Zhanghan and Zhang, Wayne and Lau, Rynson WH},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages = {10323--10333},
  year = {2023},
  google_scholar_id = {d1gkVwhDpl0C},
}