Reconstruction of 3D Interaction Models from Images using Shape Prior

Mehrshad Mirmohammadi1, Parham Saremi1, Yen-Ling Kuo2, Xi Wang3
1Sharif University of Technology, Iran
2University of Virginia, USA
3ETH Zurich, Switzerland
In Proceedings of the International Conference on Computer Vision Workshop, 2023.
Paper Code
Overview

Abstract

We investigate the reconstruction of 3D human-object interactions from images, encompassing 3D human shape and pose estimation as well as object shape and pose estimation. To address this task, we introduce an autoregressive transformer-based variational autoencoder capable of learning a robust shape prior from extensive 3D shape datasets. Additionally, we leverage the reconstructed 3D human body as supplementary features for object shape and pose estimation. In contrast, prior methods only predict object pose and rely on shape templates for shape prediction. Experimental findings on the BEHAVE dataset underscore the effectiveness of our proposed approach, achieving a 40.7cm Chamfer distance and demonstrating the advantages of learning a shape prior.

Poster

Overview

BibTeX

@inproceedings{Mirmohammadi2023reconstruction,
      title={Reconstruction of 3D Interaction Models from Images using Shape Prior},
      author={Mirmohammadi, Mehrshad and Saremi, Parham and Kuo, Yen-Ling and Wang, Xi},
      booktitle={International Conference on Computer Vision (ICCV) Workshops},
      year={2023}
    }