3D scene synthesis using natural language instructions has become a popular direction in computer graphics, with significant progress made by data-driven generative models recently. However, previous methods have mainly focused on one-time scene generation, lacking the interactive capability to generate, update, or correct scenes according to user instructions. To overcome this limitation, this paper focuses on text-guided interactive scene synthesis. First, we introduce the SceneMod dataset, which comprises 168k paired scenes with textual descriptions of the modifications. To support the interactive scene synthesis task, we propose a two-stage diffusion generative model that integrates scene-prior guidance into the denoising process to explicitly enforce physical constraints and foster more realistic scenes. Experimental results demonstrate that our approach outperforms baseline methods in text-guided scene synthesis tasks. Our system expands the scope of data-driven scene synthesis tasks and provides a novel, more flexible tool for users and designers in 3D scene generation. Code and dataset are available at https://github.com/bshfang/SceneMod.

PDF

European Association for Computer Graphics (2025).

@article{fang:eurographics25, title={Text-Guided Interactive Scene Synthesis with Scene Prior Guidance}, author={Shaoheng Fang and Haitao Yang and Raymond Mooney and Qixing Huang}, booktitle={European Association for Computer Graphics}, month={May}, url="http://www.cs.utexas.edu/users/ai-labpub-view.php?PubID=128119", year={2025} }