ZHU LEI (祝磊)

I am a PhD candiate at City University of Hong Kong under the supervision of Prof. Rynson W.H. Lau. I also work closely with Dr. Wayne Zhang. I received my bachelor degree in Computer Science and Engineering from Dalian University of Technology (rank 1/102).

Currently, I focus on vision transformer design and large language model (LLM) inference. I am particularly intrested in hardware-aware deep learning (e.g. works like FlashAttention and TinyCudaNN). I believe in simple and scalable architectures and data-centric deep learning.

Previously, I have some experience in 3D vision (camera-LiDAR calibration and neural radiance field) and semi-supervised learning.

I am on the job market. If you have any openings related to large (vision-)language models or hardware-aware/efficient neural network design & deployment, I would like to get in contact!.

News

Mar 1, 2023	Two papers (BiFormer & Neural Preset) are accepted by CVPR 2023.

Selected Publications

CVPR

BiFormer: Vision Transformer with Bi-Level Routing Attention

Lei Zhu, Xinjiang Wang, Zhanghan Ke, and 2 more authors

In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

arXiv Bib Code Poster

@inproceedings{zhu2023biformer,
  title = {BiFormer: Vision Transformer with Bi-Level Routing Attention},
  author = {Zhu, Lei and Wang, Xinjiang and Ke, Zhanghan and Zhang, Wayne and Lau, Rynson WH},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages = {10323--10333},
  year = {2023},
}