Xin Ma

Biography

Welcome 👋 I am currently a Ph.D candidate at Monash University. Previously, I received the M.S degree from University of Chinese Academy of Sciences, where I studied at CRIPAC under the leadership of Prof. Tieniu Tan and was supervised by Prof. Ran He. Before that, I obtained the B.E degree from Jiangsu University. My research interests include image super-resolution and inpainting, model compression, face recognition, video generation, large-scale generative models, etc. I am always pursuing research collaborations on deep generative models for images and videos. Feel free to contact me if you are interested.

Work Experiences

Research Intern

Shanghai Artificial Intelligence Laboratory

December 2022 – Present Shanghai

Research on generative models. During this period,

A high-quality text-to-video generation framework LaVie is proposed.
A general Transformer-based latent video diffusion model, referred to as Latte, is introduced.

Algorithm Engineer

Meituan

June 2021 – September 2022 Beijing

Research on model compression. A model compression tool has been developed to assist developers in rapidly deploying models to edge devices without compromising model accuracy, while simultaneously enhancing model inference speed. This tool has been extensively implemented across various businesses at Meituan. One paper was accepted by CVPR 2022 during this period.

Algorithm Intern

Meituan

April 2020 – August 2020 Beijing

Research on Image Dewatermarking Algorithm. An image dewatermarking algorithm was proposed based on attention mechanism and self-supervised learning. The service is now launched on Meituan App. Related work was accepted by ICPR 2020 and selected as an oral presentation.

Recent Publications

Quickly discover relevant content by filtering publications.

InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation

Published in International Conference on Learning Representations (ICLR), Spotlight, 2024, Stars

Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, Conghui He, Ping Luo, Ziwei Liu, Yali Wang, Limin Wang, Yu Qiao

InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation

SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction

Published in International Conference on Learning Representations (ICLR), 2024, Stars

Xinyuan Chen, Yaohui Wang, Lingjun Zhang, Shaobin Zhuang, Xin Ma, Jiashuo Yu, Yali Wang, Dahua Lin, Yu Qiao, Ziwei Liu

SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction

Latte: Latent Diffusion Transformer for Video Generation

arXiv preprint arXiv:2401.03048, Stars

Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, Yu Qiao

Latte: Latent Diffusion Transformer for Video Generation

LaVie: High-Quality Video Generation with Cascaded Latent Diffusion Models

arXiv preprint arXiv:2309.15103, Stars

Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, Yuwei Guo, Tianxing Wu, Chenyang Si, Yuming Jiang, Cunjian Chen, Chen Change Loy, Bo Dai, Dahua Lin, Yu Qiao, Ziwei Liu

LaVie: High-Quality Video Generation with Cascaded Latent Diffusion Models

Uncertainty-Aware Image Inpainting with Adaptive Feedback Network

Published in Expert Systems with Applications (ESWA), 2023, IF 8.5

Xin Ma, Xiaoqiang Zhou, Huaibo Huang, Gengyun Jia, Yaohui Wang, Xinyuan Chen, Cunjian Chen

Uncertainty-Aware Image Inpainting with Adaptive Feedback Network

LEO: Generative Latent Image Animator for Human Video Synthesis

arXiv preprint arXiv:2305.03989

Yaohui Wang, Xin Ma, Xinyuan Chen, Antitza Dantcheva, Bo Dai, Yu Qiao

Style-Based Attentive Network for Real-World Face Hallucination

Published in Pattern Recognition and Computer Vision (PRCV), 2022, CCF-C

Mandi Luo, Xin Ma, Huaibo Huang, Ran He

Style-Based Attentive Network for Real-World Face Hallucination

Compressing Models with Few Samples: Mimicking then Replacing

Published in Computer Vision and Pattern Recognition (CVPR), 2022, CCF-A

Huanyu Wang, Junjie Liu, Xin Ma, Yang Yong, Zhenhua Chai, Jianxin Wu

Compressing Models with Few Samples: Mimicking then Replacing

Contrastive attention network with dense field estimation for face completion

Published in Pattern Recognition (PR), 2022, IF 8.0

Xin Ma, Xiaoqiang Zhou, Huaibo Huang, Gengyun Jia, Zhenhua Chai, Xiaolin Wei

Contrastive attention network with dense field estimation for face completion

Partial NIR-VIS heterogeneous face recognition with automatic saliency search

Published in IEEE Transactions on Information Forensics and Security (T-IFS), 2021, IF 6.8

Mandi Luo, Xin Ma, Zhihang Li, Jie Cao, Ran He

Free-form image inpainting via contrastive attention network

Published in International Conference on Pattern Recognition (ICPR), Oral, 2021, CCF-C

Xin Ma, Xiaoqiang Zhou, Huaibo Huang, Zhenhua Chai, Xiaolin Wei, Ran He

Free-form image inpainting via contrastive attention network

Unsupervised Contrastive Photo-to-Caricature Translation based on Auto-distortion

Published in International Conference on Pattern Recognition (ICPR), 2021, CCF-C

Yuhe Ding, Xin Ma, Mandi Luo, Aihua Zheng, Ran He

Unsupervised Contrastive Photo-to-Caricature Translation based on Auto-distortion

Inconsistency-aware wavelet dual-branch network for face forgery detection

Published in IEEE Transactions on Biometrics, Behavior, and Identity Science (T-BIOM), 2021

Gengyun Jia, Meisong Zheng, Chuanrui Hu, Xin Ma, Yuting Xu, Luoqi Liu, Yafeng Deng, Ran He

Inconsistency-aware wavelet dual-branch network for face forgery detection

FA-GAN: face augmentation GAN for deformation-invariant face recognition

Published in IEEE Transactions on Information Forensics and Security (T-IFS), 2021, IF 6.8

Mandi Luo, Jie Cao, Xin Ma, Xiaoyu Zhang, Ran He

FA-GAN: face augmentation GAN for deformation-invariant face recognition

Recent Projects

Latte - A Transformer-based Video Diffusion Generation Framework

A simple and general latent video diffusion model incorporating sptio-temporal Transformers for video generation.

LaVie - A High-Quality Video Generation Framework

A large-scale text-to-video framework that produces high-quality and temporally coherent videos. This framework operates on cascaded video latent diffusion models, comprising a base T2V model, a temporal interpolation model, and a video super-resolution model.

Granted Patents

Image super-resolution method of deep neural network fusing mutual information, CN110211035B
Attention-mechanism-based image completion method and device, CN112184582B
Cartoon style image conversion model training method, image generation method and device, CN112232485B
Image completion method based on uncertainty estimation, CN112686817B

Academic Activities

Conference Reviewers:
- IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- IEEE International Conference on Multimedia and Expo (ICME)
- Chinese Conference on Pattern Recognition and Computer Vision (PRCV)
Journal Reviewers:
- Signal, Image and Video Processing
- IEEE Transactions on Circuits and Systems for Video Technology
- International Journal of Computer Vision

Contact

xin.ma1@monash.edu