Zhiqiang Shen

I am currently a tenure-track Assistant Professor leading VILA Lab at MBZUAI. I was a postdoctoral researcher in Professor Marios Savvides and Professor Eric Xing's group (2019-2022). My research interests span machine learning, computer vision, efficient deep learning, etc. Prior to CMU, I was fortunate to be a joint-training Ph.D student (2017-2019) in IFP group/UIUC, advised by Prof. Thomas S. Huang.

1. Multiple positions are available, including PhD, master, visiting student, Postdoc, etc. Please send me your CV if you are interested in working with me at MBZUAI.
2. We also have a few joint postdoc positions with CMU (starting from 2025), please reach out if you are interested.
(For RAs and visiting students, currently I only accept those who either plan to pursue your PhD in my group or are already PhD students at other universities seeking a short/long-term visit. If you do not meet these criteria, it is no need to apply for RA positions. Also, I may not be able to respond to every inquiry, but I do go through all the emails I receive. Thanks for your understanding.)

Email: zhiqiangshen0214 AT gmail.com | Zhiqiang.Shen AT mbzuai.ac.ae
[Google Scholar] | [Github] | [Zhihu] | [Twitter]

Research Interest

My research interests focus on the broad areas of efficient deep learning, machine learning, computer vision and natural language processing. Specifically, I am interested in deep learning methods for image recognition and object detection, efficient architecture design and parameter-efficient finetuning strategies, etc. Recently, I focus on

Foundation Models in CV and NLP
Low-bit and Efficient Networks
Knowledge Distillation on Models and Datasets
Designing and Training Highly-efficient Network Architectures for CNNs and Transformers
Un(Self-)supervised / Weakly-supervised Learning
Image Understanding, Including Object Detection, Captioning and Fine-grained Recognition
Few-shot and Zero-shot Learning

Prospective Students: I am actively looking for self-motivated students (including graduate students and research assistants) who are interested in the areas of artificial intelligence, machine learning, VLM&LLM, deep generative networks, etc. I have several PhD/Master/RA openings starting in Fall 2025/2026 at MBZUAI. Please drop me an email with your CV if you are interested in joining.

Some ongoing projects in our lab for prospective students:
1. MBZUAI-LLM Project (SlimPajama-DC , LLM360, GBLM-Pruner): Huggingface
2. OptiML: Optimizing Efficiency in Machine Learning (GLoRA, Partial Transfer, ViT-Slim): Github
3. Evolving Knowledge Distillation: The Role of Pre-generated Soft Labels (FKD, FerKD, SRe²L): Github
4. Pure Machine Learning Problems, such as: Dropout, Knowledge Distillation, BNN Optimization, etc.
...

News

[Sep 28 & Oct 19, 2024] Two papers are accepted in TMLR 2024. Congrats to Xijie and Zeyuan.
[Sep 25/26, 2024] Four papers (three in main conference and one in datasets and benchmarks track) are accepted in NeurIPS 2024. Congrats to Shitong, Liang, Kai and Sukmin.
[May 2, 2024] Two papers are accepted in ICML 2024. Congrats to Tianjun and Kirill.
[Nov 10, 2023] I received Google Research Award.
[Mar 30, 2023] [Workshop & Competition]: We organized a workshop and two competitions (hosted on Kaggle platform) in ICCV'23 regarding "Resource-Efficient Deep Learning". More details are available at website and competition-1, competition-2.
[Nov 28, 2022] [Talk]: I gave a talk at Fudan University to introduce recent works on Efficient Deep Learning. Thanks Prof. Zuxuan for the invitation.
[Jul 12, 2022] New: I will serve as an SPC (Meta-Reviewer) in AAAI 2023.
[Jan 19, 2022] [Talk]: I will give a talk at AI Drive to introduce our Un-Mix paper, thanks Haitao for the invitation.
[Jan 19, 2022] [Talk]: I will give a talk at POSTECH (Pohang University of Science and Technology) to introduce our recent KD related works, thanks Prof. Sungsoo (Peter) for the invitation.
[Jan 19, 2022] [Talk]: I will give a talk at UT Austin, thanks Tianlong for the invitation.
[Jan 7, 2022] New: We have released a demo of Un-Mix, please check it out on Github!
[Dec 13, 2021] [Talk]: I will give a talk/lecture on 2d3d.ai invited by Peter Naftaliev to systematically introduce our recent works on Knowledge Distillation. Please join us if you are interested in this topic. More details are available on meetup and reddit.
[Dec 1, 2021] Two papers accepted to AAAI 2022.
[Oct 31, 2021] One paper accepted to NeurIPS 2021, AI for Science workshop, one paper accepted to ICCV 2021 and one paper accepted to TIP 2021.
[Jul 12, 2021] Our code and models for S²-BNN (Self-supervised Binary Neural Networks Using Distillation Loss, CVPR 2021) have been released on Github.
[May 8, 2021] One paper accepted to ICML 2021 (regarding optimization on binary networks). Code and models are available on Github.
[Mar 1, 2021] Three papers accepted to CVPR 2021.
[Jan 12, 2021] One paper accepted to ICLR 2021. Our project page is here.
[Dec 23, 2020] One paper accepted to AAAI 2021.
[Nov 3, 2020] Short version of MEAL V2 has been accepted in NeurIPS 2020 Beyond BackPropagation: Novel Ideas for Training Neural Architectures workshop. Long version is coming soon.
[Oct 14, 2020] Our extended version of SCL (a domain adaptive object detection framework via gradient detach and stacked complementary losses) together with INIT has been accepted in IJCV 2020. Code and models are available on GitHub.
[Sep 19, 2020] We released MEAL V2, the first approach that can boost vanilla ResNet-50 to 80%+ Top-1 accuracy on ImageNet without tricks. Our code and models are available on GitHub.
[July 2, 2020] Three papers accepted to ECCV 2020.
[June 14, 2020] One paper accepted to CVPR 2020.
[July 14, 2019] INIT dataset has been released, please check out our Project Page.
[June 5, 2019] DSOD V2 accepted to IEEE transactions on pattern analysis and machine intelligence (TPAMI). In this version, we have provided more ablation studies and some preliminary results on exploring the factors of training two-stage detectors from scratch.
[Feb 26, 2019] One paper accepted to CVPR 2019.
[Dec 21, 2018] One paper accepted to ICLR 2019.

More News

[Nov 1, 2018] Our paper MEAL: Multi-Model Ensemble via Adversarial Learning accepted in AAAI 2019 as Oral Presentation. Code and models are available at Github.

[Sep 27, 2018] An extended version of DSOD is available on: arXiv.

[July 29, 2018] One paper accepted to ECCV 2018.

[Jan 12, 2018] I gave an invited talk at the Baidu IDL, Sunnyvale, CA, USA on the topic of learning object detectors from scratch. My talk involved our recent two papers DSOD and GRP-DSOD. Slides can be downloaded here (or Google Drive).

[Dec 22, 2017] We released the code and model for MSR-VTT Challenge (Video Captioning) on Github.

[Dec 04, 2017] Our new paper GRP-DSOD is available at: arXiv. Code and models are available at Github.

Code and models for DSOD are available at: Github.

Code and models for Network Slimming are available at: Github.

Two papers accepted to ICCV 2017.

~~Two papers submitted to ICCV 2017.~~

Our paper "Weakly Supervised Dense Video Captioning" accepted to CVPR 2017.

During my internship, our team won the 2016 Intel China Award (ICA), the highest award for team achievement in Intel China.

4th Place (Human Evaluation) and 5th Place (Automatic Evaluation Metrics) Winners at the MSR-VTT Challenge (Video Captioning). Code is here .

Recent & Selected Publications (Full List)

(*: equal contribution; ✝: corresponding author)

Zeyuan Yin, Zhiqiang Shen^✝.
Dataset Distillation via Curriculum Data Synthesis in Large Data Era
Transactions on Machine Learning Research (TMLR), 2024.
Code | Paper | arXiv

Xijie Huang, Zhiqiang Shen, Pingcheng Dong, Kwang-Ting Cheng.
Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precision
Transactions on Machine Learning Research (TMLR), 2024.
Code | Paper | arXiv

Sukmin Yun^*, Haokun Lin^*, Rusiru Thushara^*, Mohammad Qazim Bhat^*, Yongxin Wang^*, Zutao Jiang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, Haonan Li, Preslav Nakov, Timothy Baldwin, Zhengzhong Liu, Eric P. Xing, Xiaodan Liang, Zhiqiang Shen.
Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs
NeurIPS Datasets and Benchmarks Track, 2024.
Code | arXiv Paper

Shitong Shao, Zikai Zhou, Huanran Chen, Zhiqiang Shen^✝.
Elucidating the Design Space of Dataset Condensation
The Thirty-eighth Conference on Neural Information Processing Systems (NeurIPS), 2024.
Code | arXiv Paper

Kai Hu, Weichen Yu, Tianjun Yao, Xiang Li, Wenhe Liu, Lijun Yu, Yining Li, Kai Chen, Zhiqiang Shen, Matt Fredrikson.
Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained Optimization
The Thirty-eighth Conference on Neural Information Processing Systems (NeurIPS), 2024.
arXiv Paper

Liang Chen, Yong Zhang, Yibing Song, Zhiqiang Shen, Lingqiao Liu.
LFME: A Simple Framework for Learning from Multiple Experts in Domain Generalization
The Thirty-eighth Conference on Neural Information Processing Systems (NeurIPS), 2024.
arXiv Paper

Zhengzhong Liu, Aurick Qiao, Willie Neiswanger, Hongyi Wang, Bowen Tan, Tianhua Tao, Junbo Li, Yuqi Wang, Suqi Sun, Omkar Pangarkar, Richard Fan, Yi Gu, Victor Miller, Yonghao Zhuang, Guowei He, Haonan Li, Fajri Koto, Liping Tang, Nikhil Ranjan, Zhiqiang Shen, Roberto Iriondo, Cun Mu, Zhiting Hu, Mark Schulze, Preslav Nakov, Timothy Baldwin, Eric P. Xing.
LLM360: Towards Fully Transparent Open-Source LLMs
First Conference on Language Modeling (COLM), 2024.
Project Page | arXiv Paper

Tianhua Tao, Junbo Li, Bowen Tan, Hongyi Wang, William Marshall, Bhargav M Kanakiya, Joel Hestness, Natalia Vassilieva, Zhiqiang Shen, Eric P. Xing, Zhengzhong Liu.
Crystal: Illuminating LLM Abilities on Language and Code
First Conference on Language Modeling (COLM), 2024.
arXiv Paper

Kirill Vishniakov, Zhiqiang Shen, Zhuang Liu.
ConvNet vs Transformer, Supervised vs CLIP: Beyond ImageNet Accuracy
International Conference on Machine Learning (ICML), 2024.
Project Page | Code | arXiv Paper

Tianjun Yao^*,✝, Yongqiang Chen^*, Zhenhao Chen, Kai Hu, Zhiqiang Shen^✝, Kun Zhang^✝.
Empowering Graph Invariance Learning with Deep Spurious Infomax
International Conference on Machine Learning (ICML), 2024.
OpenReview | arXiv Paper

Shitong Shao, Zeyuan Yin, Muxin Zhou, Xindong Zhang, Zhiqiang Shen^✝.
Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matching
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. Highlight.
Code | arXiv Paper

Kirill Vishniakov, Eric Xing, Zhiqiang Shen.
MixMask: Revisiting Masking Strategy for Siamese ConvNets
British Machine Vision Conference (BMVC), 2024.
Code | arXiv Paper

Zhili Chen, Kien T. PHAM, Maosheng Ye, Zhiqiang Shen, Qifeng Chen.
Cross-Cluster Shifting for Efficient and Effective 3D Object Detection in Autonomous Driving
International Conference on Robotics and Automation (ICRA), 2024.
arXiv Paper

Zhiqiu Xu, Yanjie Chen, Kirill Vishniakov, Yida Yin, Zhiqiang Shen, Trevor Darrell, Lingjie Liu, Zhuang Liu.
Initializing Models with Larger Ones
International Conference on Learning Representations (ICLR), 2024. Spotlight.
Code | arXiv Paper

Sheng Zhang, Muzammal Naseer, Guangyi Chen, Zhiqiang Shen, Salman Khan, Kun Zhang, Fahad Khan.
Towards Realistic Zero-Shot Classification via Self Structural Semantic Alignment
Association for the Advancement of Artificial Intelligence (AAAI), 2024. Oral.
Code | arXiv Paper

Zeyuan Yin^*, Eric Xing, Zhiqiang Shen^*.
Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective
Conference on Neural Information Processing Systems (NeurIPS), 2023. Spotlight.
Project Page | Code | arXiv Paper

Zhiqiang Shen, Tianhua Tao, Liqun Ma, Willie Neiswanger, Zhengzhong Liu, Hongyi Wang, Bowen Tan, Joel Hestness, Natalia Vassilieva, Daria Soboleva, Eric Xing.
SlimPajama-DC: Understanding Data Combinations for LLM Training
Technical report, 2023.
Code | arXiv Paper

Zhiqiang Shen.
FerKD: Surgical Label Adaptation for Efficient Distillation
International Conference on Computer Vision (ICCV), 2023.
Code | arXiv Paper

Zhuang Liu^*, Zhiqiu Xu^*, Joseph Jin, Zhiqiang Shen, Trevor Darrell.
Dropout Reduces Underfitting
International Conference on Machine Learning (ICML), 2023.
Code | arXiv Paper

Sheng Zhang, Salman Khan, Zhiqiang Shen, Muzammal Naseer, Guangyi Chen, Fahad Khan.
PromptCAL: Contrastive Affinity Learning via Auxiliary Prompts for Generalized Novel Category Discovery
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Code | arXiv Paper

Zhiqiang Shen, Eric Xing.
A Fast Knowledge Distillation Framework for Visual Recognition
European Conference on Computer Vision (ECCV), 2022.
State-of-the-art accuracy of 80.1% (SGD) and 80.5% (AdamW) on ResNet-50 with a plain training and 16% faster than regular classification frameworks.
Project Page | Code & Models | Camera-Ready | arXiv Paper

Zhiqiang Shen, Zechun Liu, Eric Xing.
Sliced Recursive Transformer
European Conference on Computer Vision (ECCV), 2022.
Code & Models | arXiv Paper | Media (In Chinese)

Zhiqiang Shen, Zechun Liu, Zhuang Liu, Marios Savvides, Trevor Darrell, Eric Xing.
Un-Mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learning
Association for the Advancement of Artificial Intelligence (AAAI), 2022.
State-of-the-art accuracy on tiny datasets, such as CIFAR-10/100, Tiny-ImageNet.
Code & Models | arXiv Paper | Media & Zhihu (In Chinese)

Zechun Liu, Zhiqiang Shen^✝, Yun Long, Eric Xing, Kwang-Ting Cheng, Chas Leichner.
Data-Free Neural Architecture Search via Recursive Label Calibration
European Conference on Computer Vision (ECCV), 2022.
arXiv Paper

Xijie Huang, Zhiqiang Shen^✝, Shichao Li, Zechun Liu, Xianghong Hu, Jeffry Wicaksana, Eric Xing, Kwang-Ting Cheng.
SDQ: Stochastic Differentiable Quantization with Mixed Precision
International Conference on Machine Learning (ICML), 2022.
Project Page | arXiv Paper

Arnav Chavan^*, Zhiqiang Shen^*,✝, Zhuang Liu, Zechun Liu, Kwang-Ting Cheng, Eric Xing.
Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Space
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Code & Models | arXiv Paper

Zechun Liu, Kwang-Ting Cheng, Dong Huang, Eric P Xing, Zhiqiang Shen.
Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Code & Models | arXiv Paper

Zhuang Liu, Hungju Wang, Tinghui Zhou, Zhiqiang Shen, Bingyi Kang, Evan Shelhamer, Trevor Darrell.
Exploring Simple and Transferable Recognition-Aware Image Processing
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022.
Code & Models | Paper

Zechun Liu^*,✝, Zhiqiang Shen^*,✝, Shichao Li, Koen Helwegen, Dong Huang, Kwang-Ting Cheng.
How Do Adam and Training Strategies Help BNNs Optimization
International Conference on Machine Learning (ICML), 2021.
Code & Models | Paper

Zhiqiang Shen, Zechun Liu, Dejia Xu, Zitian Chen, Kwang-Ting Cheng, Marios Savvides.
Is Label Smoothing Truly Incompatible with Knowledge Distillation: An Empirical Study
International Conference on Learning Representations (ICLR), 2021.
OpenReview (Rating: 8 6 6 6) | Project Page | Paper | Zhihu (in Chinese)
A new perspective on the relationship between knowledge distillation and label smoothing. Reviewer acknowledges that this paper made a breakthrough regarding the correlation between label smoothing and knowledge distillation.

Zhiqiang Shen*, Zechun Liu*, Jie Qin, Marios Savvides, Kwang-Ting Cheng.
Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning
Association for the Advancement of Artificial Intelligence (AAAI), 2021.
arXiv Paper (Code of evolutionary searching can be referred here.)
A searching based fine-tuning method for few-shot learning.

Zhiqiang Shen, Marios Savvides.
MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks
Technical report. Short version has been accepted in NeurIPS 2020 Beyond BackPropagation: Novel Ideas for Training Neural Architectures workshop.
Code & Models | arXiv Paper
We achieve 80.67% top-1 accuracy using a single crop-size of 224×224 on the vanilla ResNet-50, the first work that is able to boost vanilla ResNet-50 to surpass 80% on ImageNet without architecture modification or additional training data. Our result can be regarded as a new strong baseline on ResNet-50 using knowledge distillation.

Zhiqiang Shen, Zechun Liu, Jie Qin, Lei Huang, Kwang-Ting Cheng, Marios Savvides.
S²-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Self-supervised BNNs using distillation loss (5.5~15% improvement over contrastive baseline).
Code & Models | arXiv Paper

Zhiqiang Shen, Mingyang Huang, Jianping Shi, Zechun Liu, Harsh Maheshwari, Yutong Zheng, Xiangyang Xue, Marios Savvides, Thomas S. Huang.
CDTD: A Large-Scale Cross-Domain Benchmark for Instance-Level Image-to-Image Translation and Domain Adaptive Object Detection
International Journal of Computer Vision (IJCV), 2020.
Code & Models | Paper

Zhiqiang Shen, Zhuang Liu, Jianguo Li, Yu-Gang Jiang, Yurong Chen, Xiangyang Xue.
Object Detection from Scratch with Deep Supervision
IEEE transactions on pattern analysis and machine intelligence (T-PAMI), 2019.
Code & Models | arXiv Paper

Zhiqiang Shen, Mingyang Huang, Jianping Shi, Xiangyang Xue, Thomas S. Huang.
Towards Instance-level Image-to-Image Translation
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Project | Paper | Dataset

Zhiqiang Shen*, Zhankui He*, Xiangyang Xue.
MEAL: Multi-Model Ensemble via Adversarial Learning
33rd AAAI Conference on Artificial Intelligence (AAAI), 2019. (Oral)
Code & Models | Our ResNet-50 (Top-1/5: 21.70%/5.99%) [PyTorch Model (102.5M)]

Zhiqiang Shen, Honghui Shi, Jiahui Yu, Hai Phan, Rogerio Feris, Liangliang Cao, Ding Liu, Xinchao Wang, Thomas Huang, Marios Savvides.
Improving Object Detection from Scratch via Gated Feature Reuse
30th British Machine Vision Conference (BMVC), 2019.

Zhiqiang Shen*, Zhuang Liu*, Jianguo Li, Yu-Gang Jiang, Yurong Chen, Xiangyang Xue.
DSOD: Learning Deeply Supervised Object Detectors from Scratch
Proceedings of 16th IEEE International Conference on Computer Vision (ICCV2017).
(* indicates equal contribution)
Code & Models | Paper

Zhiqiang Shen, Jianguo Li, Zhou Su, Minjun Li, Yurong Chen, Yu-Gang Jiang, Xiangyang Xue.
Weakly Supervised Dense Video Captioning
Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR2017).
Project | Paper

Academic Activities

Meta-Reviewer (SPC): AAAI 2023, 2024.
Conference reviewer: ICLR 2023, NeurIPS 2022, ICML 2022, ICLR 2022, ECCV 2022, CVPR 2022, NeurIPS 2021, ICML 2021, CVPR 2021, AAAI 2021, WACV 2021, NeurIPS 2020, ECCV 2020, BMVC 2020, IJCAI 2020, CVPR 2020, AAAI 2020, ICCV 2019, CVPR 2019, AAAI 2019, CVPR 2018, ACCV 2018, NIPS 2016.
Journal reviewer: TPAMI, IJCV, TMLR, TIP, TMM, JVCI, etc.

Awards and Honors

CVPR 2019 Doctoral Consortium travel award. Mentor: Prof. Trevor Darrell.
ICLR 2019 travel award, 2019
AAAI 2019 student scholarship award, 2018
ICCV 2017 student volunteer, 2017
Huawei scholarship, 2017
During my internship, our team won the 2016 Intel China Award (ICA), the highest award for team achievement in Intel China, 2016
Tung OOCL scholarship, 2015
Special Grade Scholarship, 2013
University-level Outstanding Students, 2013

Competitions

iMaterialist Challenge on Product Recognition (Fine-grained image classification of products at FGVC6, CVPR'19 workshop): ranked 4th globally (Team leader).
MSR-VTT Challenge (video captioning): ranked 4th in human evaluation and ranked 5th in the automatic evaluation metrics (Team leader), 2016
Top 10% in Kaggle Competition of Right Whale Recognition, 2016
Second Prize in DataCastle Competition of the Verification Code Recognition, 2016
Second Prize (National-level) in China Graduate Student Mathematical Contest in Modeling, 2015
MCM/ICM -- Honorable Mention, 2012
First Prize (National-level) in Electrical Engineering Mathematical Contest in Modeling, 2012
First Prize (National-level) in China Undergraduate Mathematical Contest in Modeling, 2011 (大学生数学建模竞赛全国一等奖)
Second Prize of Jiangsu High School Physics Competition (江苏省高中物理竞赛二等奖)

Teaching Assistant

2015.9- 2016.1, Fudan University, COMP120008.02, C++ language programming