Light Path Gradients for Forward and Inverse Rendering
PhD Thesis, École Polytechnique Fédérale de Lausanne, December 2021
Thesis jury:
Sabine Süsstrunk,
Wojciech Jarosz,
Tzu-Mao Li, and
Mark Pauly

Abstract
Physically based rendering is a process for photorealistic digital image synthesis and one of the core problems in computer graphics. It involves simulating the light transport, i.e. the emission, propagation, and scattering of light through a virtual scene that is defined by a detailed description of object geometry and appearance. Research over the last decades has led to sophisticated rendering techniques and recently, the inversion of this process, i.e. recovering scene parameters from image observations, has also received significant attention. In this thesis, we investigate methods in both physically based forward and inverse rendering that exploit light path gradients.
The first part is concerned with scattering from specular surfaces, which
produces complex optical effects that are frequently encountered in realistic
scenes: intricate caustics due to focused reflection, multiple refractions, and
high-frequency glints from specular microstructure. Yet, despite their
importance and considerable research to this end, sampling of light paths that
cause these effects remains a formidable challenge of forward rendering.
We propose a surprisingly simple and general path sampling strategy that targets
the examples above. Valid light path configurations need to fulfill the physical
laws of reflection and refraction, and we find these using a numerical
root-finding process that is driven by geometric light path gradients. In
contrast to prior work, our method supports high-frequency normal- or
displacement-mapped geometry, samples specular-diffuse-specular (SDS) paths, and
is compatible with standard Monte Carlo methods including unidirectional path
tracing. We demonstrate our method on a range of challenging scenes and evaluate
it against state-of-the-art methods for rendering caustics and glints.
In the second part, we consider differentiable rendering algorithms.
These propagate derivatives through the full light transport simulation to solve
inverse rendering problems via gradient-based optimization. Recent progress has
led to methods that can simultaneously compute derivatives with respect to
millions of scene parameters. At the same time, elementary properties of these
methods remain poorly understood.
Current algorithms for differentiable rendering are constructed by mechanically
differentiating a given primal algorithm. As differentiation fundamentally
changes the underlying problem, this is often suboptimal and instead, primal and
differential algorithms should be decoupled so that the latter can suitably
adapt. This is surprisingly complex. Even the most basic Monte Carlo path tracer
already involves several design choices concerning the techniques for sampling
materials and emitters, and their combination, e.g. via multiple importance
sampling (MIS). Differentiation causes a veritable explosion of this decision
tree: should we differentiate only the estimator, or also the sampling
technique? Should MIS be applied before or after differentiation? Are
specialized derivative sampling strategies of any use? How should
visibility-related discontinuities be handled when millions of parameters are
differentiated simultaneously? We provide a taxonomy and analysis of different
estimators for differential light transport to provide intuition about these and
related questions.
Downloads
Thesis
BibTeX Reference
@PhdThesis{Zeltner2021Thesis,
title = "Light Path Gradients for Forward and Inverse Rendering",
author = "Tizian Zeltner",
school = "École Polytechnique Fédérale de Lausanne",
month = dec,
year = "2021"
}