parametric-face-image-generator

This software enables you to generate fully parametric face images from the Basel Face Model 2017 as proposed in:

• [1] Adam Kortylewski, Andreas Schneider, Thomas Gerig, Bernhard Egger, Andreas Morel-Forster and Thomas Vetter "Training Deep Face Recognition Systems with Synthetic Data", IN: arXiv preprint (2018)

• [2] Adam Kortylewski, Bernhard Egger, Andreas Schneider, Thomas Gerig, Andreas Morel-Forster and Thomas Vetter "Empirically Analyzing the Effect of Dataset Biases on Deep Face Recognition Systems", IN: CVPRW (2018)

You can control the variation of parameters such as pose, shape, color, camera and illumination based on your demand and application. This dataset can be used for training and comparing machine learning techniques such as CNNs on a common ground as proposed in [1] by generating fully controlled training and test data.

Rendering Setup

Above you can see example face images sampled from this data generator. Each row shows different images of the same facial identity.

In the "controlled" setup (top row), the model parameters are sampled at equidistant positions along a certain parameter , e.g. the yaw pose.

In the "random" setup (bottom row), the model parameters are sampled randomly from a custom distribution.

Rendering Different Image Modalities

You can render different image modalities such as e.g. depth images (top row), color coded correspondence images (bottom row), normals, albedo or illumination.

Facial Landmarks

For each face image the location and visibilty of 19 facial landmarks is written in a .tlms file in the following format:

"facial landmark name" "visibility" "x-position" "y-position"

Usage

Setup

• installed Java (Version 8.0 or higher recommended)
• download jar and config file under release
• download Basel Face Model 2017
• download Basel Illumination Prior 2017
• get a dataset with backgrounds, e.g. the Describable Textures Dataset

Run

• adapt paths and configuration in data/config_files/example_config_controlled.json
• For generating images in the controlled setup execute:
• java -Xmx2g -cp generator.jar faces.apps.ControlledFaces -c data/config_files/example_config_controlled.json
• For generating images in the random setup execute:
• java -Xmx2g -cp generator.jar faces.apps.RandomFaces -c data/config_files/example_config_random.json

For Developers:

• installed Java (Version 8.0 or higher recommended)
• installed sbt (only for compiling from sources)
• clone repository
• compile and run using sbt run -mem 2000

Singularity:

• we provide a singularity container recipe file to run the data generator directly on compute servers

Help needed

There is a scalismo-faces google group for general questions and discussion.

Background

Besides the publications listed next, we have also freely available lectures and tutorials. Some of the topics covered are statistical shape modeling and model-based image analysis as part of our research about Probabilistic Morphable Models.

Publications

If you use this software you will need the Basel Face Model 2017, the Basel Illumination Prior 2017 and a dataset of backgrounds. Please cite the following papers:

Data Generator - Random Mode

• [1] Adam Kortylewski, Andreas Schneider, Thomas Gerig, Bernhard Egger, Andreas Morel-Forster and Thomas Vetter "Training Deep Face Recognition Systems with Synthetic Data", IN: arXiv preprint (2018)

Data Generator - Controlled Mode

• [2] Adam Kortylewski, Bernhard Egger, Andreas Schneider, Thomas Gerig, Andreas Morel-Forster and Thomas Vetter "Empirically Analyzing the Effect of Dataset Biases on Deep Face Recognition Systems", IN: CVPRW (2018)

Basel Face Model 2017

• [3] Thomas Gerig, Andreas Morel-Forster, Clemens Blumer, Bernhard Egger, Marcel Luethi, Sandro Schoenborn and Thomas Vetter " Morphable Face Models - An Open Framework", IN: arXiv preprint (2017)

Basel Illumination Prior 2017

• [4] Bernhard Egger, Sandro Schoenborn, Andreas Schneider, Adam Kortylewski, Andreas Morel-Forster, Clemens Blumer and Thomas Vetter "Occlusion-aware 3D Morphable Models and an Illumination Prior for Face Image Analysis", IN: International Journal of Computer Vision, 2018

Background Dataset

• A background dataset of your choice

Contributors

• Bernhard Egger
• Adam Kortylewski
• Andreas Morel-Forster
• Andreas Schneider

Maintainers

• University of Basel, Graphics and Vision research: @unibas-gravis, homepage

License

Apache License, Version 2.0, details see LICENSE

Copyright 2017, University of Basel, Graphics and Vision Research

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.