Date: Thursday 19th January, at 4 pm Irish time
Title:
Colourisation and Any-resolution Generation
Abstract:
Part I: Given a grayscale photograph as input, we attack the problem of hallucinating a plausible color version of the photograph. This problem is clearly underconstrained, so previous approaches have either relied on significant user interaction or resulted in desaturated colorizations. We leverage big data and deep networks to develop (a) a fully automatic approach that produces vibrant and realistic colorizations and (b) a user-guided approach which also incorporates sparse, local user “hints” to an output colorization with a CNN. We validate our method by using real human judges.
Part II: Generative models operate at fixed resolution, even though natural images come in a variety of sizes. As high-resolution details are downsampled away, and low-resolution images are discarded altogether, precious supervision is lost. We argue that every pixel matters and create datasets with variable-size images, collected at their native resolutions. Taking advantage of this data is challenging; high-resolution processing is costly, and current architectures can only process fixed-resolution data. We introduce continuous-scale training, a process that samples patches at random scales to train a new generator with variable output resolutions. We generate higher resolutions images than previously possible, without adding layers to the model. We train on natural image domains including churches, mountains, birds, and faces, and demonstrate arbitrary scale synthesis with both coherent glob
Speaker Biography:
Richard Zhang is a Senior Research Scientist at Adobe Research, with interests in computer vision, deep learning, machine learning, and graphics. He obtained his PhD in EECS, advised by Professor Alexei A. Efros, at UC Berkeley in 2018. He graduated summa cum laude with BS and MEng degrees from Cornell University in ECE. He is a recipient of the 2017 Adobe Research Fellowship.
More information can be found on his webpage:
