ReCoDi is a research project funded by the FFG.

Motivation

Colorization, i.e. adding color to monochrome images or videos, makes historic movies more accessible to audiences used to the high quality imagery of today’s cinema productions. The unique impression obtained by adding color removes the archaic aesthetics associated with black-and-white or grayscale imagery and thus makes the depictions of historic events visually appealing for a wider audience. Despite significant improvements in available image and video editing techniques, however, creating full-color versions of a traditional movie or old black-and-white documentaries still requires an enormous amount of manual labor to achieve precise and natural-looking colorization results that keep a historically authentic look.

This project, ReCoDi (Refinable Colorization of Historic Imagery via Conditional Diffusion Models),  aims to develop novel deep learning-based approaches to efficiently colorize monochrome or black-and-white imagery at a quality level that meets the requirements of the film and restoration industry. Such a colorization technique would significantly reduce the duration of professionally conducted restoration and can furthermore be applied for both single images (e.g. historic documents), as well as video colorization by applying it to selected keyframes of a video that can be temporally propagated by the existing innovative techniques.

Main Objectives

• Significantly reducing the currently high cost and large amount of manual labor needed for the professional colorization of historic imagery.
• Analysis and development of novel deep learning-based approaches to automate the professional colorization via multi-modal conditioning signals.
• A novel interactive refinement approach that allows fast adjustments of the colorization results to meet the user’s expectations by model guidance, while minimizing the amount of required user inputs.
• Validation of the investigated colorization approaches by human colorization experts to ensure visually aesthetic and historically authentic results, while allowing efficient user interaction with the model to support the artistic freedom.