Neural Mesh Flow: 3D Manifold Mesh Generation via Diffeomorphic Flows

Mesh RCNN [1]

Ours

Ground Truth

Mesh RCNN [1]

Ours

Ground Truth

Abstract

Meshes are important representations of physical 3D entities in the virtual world. Applications like rendering, simulations and 3D printing require meshes to be manifold so that they can interact with the world like the real objects they represent. Prior methods generate meshes with great geometric accuracy but poor manifoldness. In this work we propose Neural Mesh Flow (NMF) to generate two-manifold meshes for genus-0 shapes. Specifically, NMF is a shape auto-encoder consisting of several Neural Ordinary Differential Equation (NODE)[2] blocks that learn accurate mesh geometry by progressively deforming a spherical mesh. Training NMF is simpler compared to state-of-the-art methods since it does not require any explicit mesh-based regularization. Our experiments demonstrate that NMF facilitates several applications such as single-view mesh reconstruction, global shape parameterization, texture mapping, shape deformation and correspondence. Importantly, we demonstrate that manifold meshes generated using NMF are better-suited for physically-based rendering and simulation.

Supplementary Video

Paper

@misc{gupta2020neural, title={Neural Mesh Flow: 3D Manifold Mesh Generationvia Diffeomorphic Flows}, author={Kunal Gupta and Manmohan Chandraker}, year={2020}, eprint={2007.10973}, archivePrefix={arXiv}, primaryClass={cs.CV} }

Acknowledgement

We would like to thank Krishna Murthy Jatavallabhula for valuable discussions. We would also like to thank Pengcheng Cao with UCSD CHEI for providing 3D printed models and Shreyam Natani for helping with Blender.

References

[1] Gkioxari, Georgia, Jitendra Malik, and Justin Johnson. "Mesh r-cnn." Proceedings of the IEEE International Conference on Computer Vision. 2019.
[2] Chen, Ricky TQ, et al. "Neural ordinary differential equations." Advances in neural information processing systems. 2018.
[3] Groueix, Thibault, et al. "A papier-mache approach to learning 3d surface generation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
[4] Wang, Nanyang, et al. "Pixel2mesh: Generating 3d mesh models from single rgb images." Proceedings of the European Conference on Computer Vision (ECCV). 2018.