IKEA-Manual: Seeing Shape Assembly Step by Step
Abstract
Human-designed visual manuals are crucial components in shape assembly activities. They provide step-by-step guidance on how we should move and connect different parts in a convenient and physically-realizable way. While there has been an ongoing effort in building agents that perform assembly tasks, the information in human-design manuals has been largely overlooked. We identify that this is due to 1) a lack of realistic 3D assembly objects that have paired manuals and 2) the difficulty of extracting structured information from purely image-based manuals. Motivated by this observation, we present IKEA-Manual, a dataset consisting of 102 IKEA objects paired with assembly manuals. We provide fine-grained annotations on the IKEA objects and assembly manuals, including decomposed assembly parts, assembly plans, manual segmentation, and 2D-3D correspondence between 3D parts and visual manuals. We illustrate the broad application of our dataset on four tasks related to shape assembly: assembly plan generation, part segmentation, pose estimationand 3D part assembly.
Cite
Text
Wang et al. "IKEA-Manual: Seeing Shape Assembly Step by Step." Neural Information Processing Systems, 2022.Markdown
[Wang et al. "IKEA-Manual: Seeing Shape Assembly Step by Step." Neural Information Processing Systems, 2022.](https://mlanthology.org/neurips/2022/wang2022neurips-ikeamanual/)BibTeX
@inproceedings{wang2022neurips-ikeamanual,
title = {{IKEA-Manual: Seeing Shape Assembly Step by Step}},
author = {Wang, Ruocheng and Zhang, Yunzhi and Mao, Jiayuan and Zhang, Ran and Cheng, Chin-Yi and Wu, Jiajun},
booktitle = {Neural Information Processing Systems},
year = {2022},
url = {https://mlanthology.org/neurips/2022/wang2022neurips-ikeamanual/}
}