Light Source Estimation via Intrinsic Decomposition for Novel View Synthesis
OPEN ACCESS
Author / Producer
Date
2023
Publication Type
Master Thesis
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
In this thesis, we propose a method for light source estimation via intrinsic decomposition within a Neural Radiance Field-like setting. We begin by exploring a learned method to recover light sources from intrinsic components - reflectance, shading, and surface normals. As demonstrated, the learned mapping is unable to properly learn the relationship between the different intrinsic components, and instead overfits some components while ignoring the rest. However, inference of local SH lighting, while somewhat spatially discontinuous, shows promise for future extensions. For these reasons, we later pivot our approach to directly optimize global spherical harmonics lighting, in which the in-scene radiance is represented by coefficients of a single global basis function. With this optimization-based approach, we successfully reconstruct a coarse estimate of the lighting of our scenes. To validate and support our method, we introduce a novel synthetic dataset, which consists of intrinsic components of several scenes. For each scene in our dataset, we provide geometry, shading, reflectance, and fully diffuse and rendered passes. We hope that extensions of our work could serve as modules of future NeRF models, enabling the joint optimization and refinement of lighting, geometry, and radiance.
Permanent link
Publication status
published
External links
Editor
Contributors
Examiner : Van Gool, Luc
Examiner : Oswald, Martin Ralf
Examiner : Das, Partha
Examiner : Sandström, Erik
Book title
Journal / series
Volume
Pages / Article No.
Publisher
ETH Zurich
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
03514 - Van Gool, Luc (emeritus) / Van Gool, Luc (emeritus)