March 18, 2025

ikayaniaamirshahzad@gmail.com

Unleashing Zero-shot Single-view 3D Indoor Scene Modeling


View a PDF of the paper titled Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modeling, by Qirui Wu and 4 other authors

View PDF
HTML (experimental)

Abstract:Reconstructing structured 3D scenes from RGB images using CAD objects unlocks efficient and compact scene representations that maintain compositionality and interactability. Existing works propose training-heavy methods relying on either expensive yet inaccurate real-world annotations or controllable yet monotonous synthetic data that do not generalize well to unseen objects or domains. We present Diorama, the first zero-shot open-world system that holistically models 3D scenes from single-view RGB observations without requiring end-to-end training or human annotations. We show the feasibility of our approach by decomposing the problem into subtasks and introduce robust, generalizable solutions to each: architecture reconstruction, 3D shape retrieval, object pose estimation, and scene layout optimization. We evaluate our system on both synthetic and real-world data to show we significantly outperform baselines from prior work. We also demonstrate generalization to internet images and the text-to-scene task.

Submission history

From: Qirui Wu [view email]
[v1]
Fri, 29 Nov 2024 06:19:04 UTC (47,960 KB)
[v2]
Fri, 14 Mar 2025 22:54:30 UTC (49,121 KB)



Source link

Leave a Comment