PRISM
Photorealistic Reconstruction
& Intrinsic Scene Modeling

Alara Dirik1*, Tuanfeng Wang2, Duygu Ceylan2, Stefanos Zafeiriou1, Anna Frühstück2

1Imperial College London  ·  2Adobe Research

*Work done during an internship at Adobe.

ICPR 2026
Paper

Overview

PRISM overview
PRISM is a unified framework for conditional generation of RGB images and their intrinsic channels (referred to as X layers) simultaneously. It supports text-to-RGBX generation, RGB-to-X decomposition, and X-to-RGBX conditional generation — enabling both local material editing and global image relighting through conditioning on selected intrinsic layers and text prompts.

Contributions

Unified Multi-task Model

A single model for inverse and forward rendering, and text-to-RGBX generation that demonstrates strong generalization to diverse real-world scenes.

Joint RGBX Generation

Joint generation of intrinsic layers and RGB images for improved cross-modality alignment, addressing ambiguity between geometry and material properties.

Versatile Editing

Competitive performance across intrinsic decomposition and diverse editing applications including text-guided relighting and local material editing.

Abstract

We present PRISM, a unified framework that enables multiple image generation and editing tasks in a single foundational model. Starting from a pre-trained text-to-image diffusion model, PRISM proposes an effective fine-tuning strategy to produce RGB images along with intrinsic maps (referred to as X layers) simultaneously. Unlike previous approaches, which infer intrinsic properties individually or require separate models for decomposition and conditional generation, PRISM maintains consistency across modalities by generating all intrinsic layers jointly. It supports diverse tasks, including text-to-RGBX generation, RGB-to-X decomposition, and X-to-RGBX conditional generation. Additionally, PRISM enables both global and local image editing through conditioning on selected intrinsic layers and text prompts. Extensive experiments demonstrate the competitive performance of PRISM both for intrinsic image decomposition and conditional image generation while preserving the base model's text-to-image generation capability.

Method

PRISM pipeline
Pipeline. RGB image and its corresponding X intrinsic channels are encoded into latent space via a fixed VAE Encoder. A Diffusion Transformer operates on the tokens of all channels simultaneously, conditioned by text embeddings. During training, intrinsic channels are randomly ablated or provided as condition (orange), making PRISM a unified framework for text-to-RGBX generation, intrinsic decomposition, and conditional image generation.

Results

PRISM Modes Unified

PRISM supports multiple generation modes in a single model: text-to-RGBX, RGB-to-X decomposition, and X-to-RGBX conditional generation. Any subset of text, RGB, and X inputs can be combined to jointly generate RGBX images.

PRISM modes overview

Overview of operation modes and their applications

Text-to-RGBX Generation

Generate RGB images along with albedo, normals, depth, and irradiance from text descriptions alone.

Text-to-RGBX results

Diverse text prompts generating consistent RGB and intrinsic representations

RGB-to-X Intrinsic Decomposition

Given an RGB input, PRISM decomposes it into albedo, surface normals, depth, and irradiance — matching or outperforming state-of-the-art methods while being a single unified model.

Decomposition results

Comparison with State-of-the-Art

Comparison with baselines

Visual comparison against baseline methods on synthetic datasets

Text-Guided Relighting Global

Decompose an input image, then condition generation on all intrinsic layers except irradiance with a new lighting text prompt. Geometric and material properties are preserved while generating plausible new lighting, including shadows, specular highlights, and indirect illumination.

Relighting results

Comparison with State-of-the-Art

Relighting comparison

Comparison with RGB↔X, IC-Light-V2, Latent Intrinsic, and LumiNet

Local Material Editing Local

Masked conditioning lets PRISM edit specific objects while preserving lighting consistency. A mask and text prompt describing desired material properties generates realistic appearance changes while maintaining scene identity.

Material editing results

Appearance changes on masked objects with consistent lighting

BibTeX

@inproceedings{Dirik2025PRISM,
        title = {PRISM: A Unified Framework for Photorealistic Reconstruction and Intrinsic Scene Modeling},
        author = {Dirik, Alara and Wang, Tuanfeng and Ceylan, Duygu and Zafeiriou, Stefanos and Fr{\"u}hst{\"u}ck, Anna},
        booktitle = {Proceedings of the International Conference on Pattern Recognition (ICPR)},
        year = {2026}
}