Data-driven Material Synthesis

Research from Sema Berkiten, PhD student in Computer Graphics at Princeton

Editor's note: We were so thrilled to have Sema join us for a summer and a semester to work with us on material rendering and synthesis in our graphics engine, Luma. We asked her to pull together a few slides/images to share with the world the nature of her work from the past 9 months with us.

In this project, we aim to automatically create realistic normal and texture maps from example imagery. We focus on soft materials found in residential & commercial furniture (couches, pillows, beds, chairs, etc.).

Given a piece of low-poly geometry, we would like to produce high resolution normal and texture maps with wrinkling/folding, compression, and stretching effects from our database of example 3D models with detailed material maps (texture and normal maps).

In the figure below, control maps are created by unfolding the triangles starting from the center of each hemisphere for different resolution of 3D meshes. These control maps will be used as guidance maps for the synthesis of wrinkles in the future.

To create small patches of wrinkles from the bump map database, we use an SLIC [Achanta'12] algorithm to segment the bump map into small coherent patches. We regroup a few neighboring coherent segments to form single wrinkle patches as shown below:

Given a small image patch as an example, we can synthesize the texture over arbitrary 3D surfaces by using [Turk'01] algorithm:

Sema is going to continue on with her research at Princeton and we should be seeing the fruits of her labor in models inside of Luma later this year!

If these topics interest you on either a technical or business development level, please reach out and say hello.