Fragment Relation Networks for Geometric Shape Assembly

Abstract

A geometric shape is often made of multiple fragments or parts. Assembling the fragments into the target object can be viewed as an interesting combinatorial problem with a variety of applications in science and engineering. Previous related work, however, focuses on tackling limited cases, e.g., primitive fragments of identical shapes or jigsaw-style fragments of textured shapes, which greatly mitigate the combinatorial challenge. In this work we introduce a challenging problem of shape assembly with textureless fragments of arbitrary shapes and propose a learning-based approach to solving it. Given a target object and a set of candidate fragments, the proposed model learns to select one of the fragments and place it into a right place. Our model processes the candidate fragments in a permutation-equivariant manner and can generalize to cases with an arbitrary number of fragments and even with a different target object. We demonstrate our method on shape assembly tasks with different shapes and assembling scenarios.