Tizian Zeltner

Mitsuba 2: A Retargetable Forward and Inverse Renderer

Merlin Nimier-David * Delio Vicini * Tizian Zeltner Wenzel Jakob

* Joint first authors

In Transactions on Graphics (Proceedings of SIGGRAPH Asia 2019)

Four applications enabled using automated transformations of a generic renderer. (a) Polarized spectral rendering of an optical experiment that analyzes light with elliptical polarization. (b) A coherent MCMC rendering algorithm that explores bundles of nearby light paths to improve convergence at equal render time. (c) A refractive slab optimized by inverse rendering to focus light with three primary colors into a rendition of the painting A Sunday Afternoon on the Island of La Grande Jatte by Georges Seurat. (d) Reconstruction of a smoke volume from reference images; multiple iterations of the optimization are shown. Please refer to the supplemental video for animated visualizations of many results shown in this paper.

Abstract

Modern rendering systems are confronted with a dauntingly large and growing set of requirements: in their pursuit of realism, physically based techniques must increasingly account for intricate properties of light, such as its spectral composition or polarization. To reduce prohibitive rendering times, vectorized renderers exploit coherence via instruction-level parallelism on CPUs and GPUs. Differentiable rendering algorithms propagate derivatives through a simulation to optimize an objective function, e.g., to reconstruct a scene from reference images. Catering to such diverse use cases is challenging and has led to numerous purpose-built systems—partly, because retrofitting features of this complexity onto an existing renderer involves an error-prone and infeasibly intrusive transformation of elementary data structures, interfaces between components, and their implementations (in other words, everything).

We propose Mitsuba 2, a versatile renderer that is intrinsically retargetable to various applications including the ones listed above. Mitsuba 2 is implemented in modern C++ and leverages template metaprogramming to replace types and instrument the control flow of components such as BSDFs, volumes, emitters, and rendering algorithms. At compile time, it automatically transforms arithmetic, data structures, and function dispatch, turning generic algorithms into a variety of efficient implementations without the tedium of manual redesign. Possible transformations include changing the representation of color, generating a “wide” renderer that operates on bundles of light paths, just-in-time compilation to create computational kernels that run on the GPU, and forward/reverse-mode automatic differentiation. Transformations can be chained, which further enriches the space of algorithms derived from a single generic implementation.

We demonstrate the effectiveness and simplicity of our approach on several applications that would be very challenging to create without assistance: a rendering algorithm based on coherent MCMC exploration, a caustic design method for gradient-index optics, and a technique for reconstructing heterogeneous media in the presence of multiple scattering.

Video

Downloads

Publication

Code

Supplemental

BibTeX Reference

@article{NimierDavidVicini2019Mitsuba2,
    author = {Merlin Nimier-David and Delio Vicini and Tizian Zeltner and Wenzel Jakob},
    title = {Mitsuba 2: A Retargetable Forward and Inverse Renderer},
    journal = {Transactions on Graphics (Proceedings of SIGGRAPH Asia)},
    volume = {38},
    number = {6},
    year = {2019},
    month = dec,
    doi = {10.1145/3355089.3356498}
}