BioSkin

Copyright (c) Meta Platforms, Inc. and affiliates. This source code is licensed under the MIT license found in the LICENSE file in the root directory of this source tree.

This is the implementation of "A Hyperspectral Space of Skin Tones for Inverse Rendering of Biophysical Skin Properties". Carlos Aliaga, Mandy Xia, Hao Xie, Adrian Jarabo, Gustav Braun, Christophe Hery. Computer Graphics Forum (EGSR 2023) [Paper]. Further references: [ArXiv 2022 Paper] [SIG'22 Course].

Citation

@article {10.1111:cgf.14887,
journal = {Computer Graphics Forum},
title = {{A Hyperspectral Space of Skin Tones for Inverse Rendering of Biophysical Skin Properties}},
author = {Aliaga, Carlos and Xia, Mengqi and Xie, Hao and Jarabo, Adrian and Braun, Gustav and Hery, Christophe},
year = {2023},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {10.1111/cgf.14887}
}

Installation

Download

git clone [email protected]:facebookresearch/BioSkin.git

Requirements

python -m venv ./venv
.\venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Packaging

Simply run python -m pip install . and the package should be available by import bioskin.

Running

To have an overview of the different use cases and practical applications, check examples.py. Run selecting the application with --mode. e.g.: --mode "Reconstruction".

• GUI: Interactive estimation and editing tool. See next section.
• Reconstruction: Estimates skin properties from diffuse albedo textures.
• Sampler: Skin tones sampler.
• Augment: Augment dataset with skin properties.
• OptimizeCCM: Optimize color correction matrix given an image or a set of images (with consistent illumination).
• Homogenize: Match skin properties and reflectance to a target (all textures to the last one loaded).
• OptimizeProps: Optimize bio maps using the decoder only (skin properties --> reflectance spectra).

For training new models, RGB or spectral of different wavelength ranges, different losses, etc. please check bioskin/train.py or bioskin/train_exposure_aware.py.

GUI/Demo

• The GUI displays the first character from the input folder. Switch using 'Character' menu.
• The main window can be interacted with like general maps (zooming, moving, etc).
• Clicking on the component images (and small reflectance) on the right switches the main view to that specific texture for more details.
• Use 'View' menu to switch from visible range reflectance, to near infrared reflectance, original and edited (key e toggles the original and the edited versions).
• Edits are stacked one after another. Once you are done, press 'Push Edits and Update Reflectance' to see the effect on the albedo. Also use:
  • 'Reset Reflectance' button to recover the original skin reflectance and all unedited property maps.
  • 'Reset Property' button to undo and recover the original skin property map.
  • 'Load from File' button to perform edits outside the program. Edit any map (the .jpeg) in the output_folder and press the 'Load from File' button.

License

BioSkin is MIT licensed, as found in the LICENSE file.