HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling

Zhongang Cai, Daxuan Ren, Ailing Zeng, Zhengyu Lin, Tao Yu, Wenjia Wang, Xiangyu Fan, Yang Gao, Yifan Yu, Liang Pan, Fangzhou Hong, Mingyuan Zhang, Chen Change Loy, Lei Yang, Ziwei Liu

ECCV 2022

doi:10.1007/978-3-031-20071-7_33 /eccv/2022/cai2022eccv-humman/

Abstract

4D human sensing and modeling are fundamental tasks in vision and graphics with numerous applications. With the advances of new sensors and algorithms, there is an increasing demand for more versatile datasets. In this work, we contribute HuMMan, a large-scale multi-modal 4D human dataset with 1000 human subjects, 400k sequences and 60M frames. HuMMan has several appealing properties: 1) multi-modal data and annotations including color images, point clouds, keypoints, SMPL parameters, and textured meshes; 2) popular mobile device is included in the sensor suite; 3) a set of 500 actions, designed to cover fundamental movements; 4) multiple tasks such as action recognition, pose estimation, parametric human recovery, and textured mesh reconstruction are supported and evaluated. Extensive experiments on HuMMan voice the need for further study on challenges such as fine-grained action recognition, dynamic human mesh reconstruction, point cloud-based parametric human recovery, and cross-device domain gaps.

PDF ECCV Semantic Scholar

Cite

Text

Cai et al. "HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling." Proceedings of the European Conference on Computer Vision (ECCV), 2022. doi:10.1007/978-3-031-20071-7_33

Markdown

[Cai et al. "HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling." Proceedings of the European Conference on Computer Vision (ECCV), 2022.](https://mlanthology.org/eccv/2022/cai2022eccv-humman/) doi:10.1007/978-3-031-20071-7_33

BibTeX

@inproceedings{cai2022eccv-humman,
  title     = {{HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling}},
  author    = {Cai, Zhongang and Ren, Daxuan and Zeng, Ailing and Lin, Zhengyu and Yu, Tao and Wang, Wenjia and Fan, Xiangyu and Gao, Yang and Yu, Yifan and Pan, Liang and Hong, Fangzhou and Zhang, Mingyuan and Loy, Chen Change and Yang, Lei and Liu, Ziwei},
  booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
  year      = {2022},
  doi       = {10.1007/978-3-031-20071-7_33},
  url       = {https://mlanthology.org/eccv/2022/cai2022eccv-humman/}
}