EasyRet3D: Uncalibrated Multi-View Multi-Human 3D Reconstruction and Tracking

Abstract

Current methods performing 3D human pose estimation from multi-view still bear several key limitations. First most methods require manual intrinsic and extrinsic camera calibration which is laborious and difficult in many settings. Second more accurate models rely on further training on the same datasets they evaluate severely limiting their generalizability in real-world settings. We address these limitations with EasyRet3D (Easy REconstruction and Tracking in 3D) which simultaneously reconstructs and tracks 3D humans in a global coordinate frame across all views with uncalibrated cameras and videos in the wild. EasyRet3D is a compositional framework that composes our proposed modules (Automatic Calibration module Adaptive Stitching Module and Optimization Module) and off-the-shelf large pre-trained models at intermediate steps to avoid manual intrinsic and extrinsic calibration and task-specific training. EasyRet3D outperforms all existing multi-view 3D tracking or pose estimation methods in Panoptic EgoHumans Shelf and Human3.6M datasets. Codebase and demos will be released on the project website.