Annals of Mathematical Sciences and Applications

Volume 3 (2018)

Number 1

Special issue in honor of Professor David Mumford, dedicated to the memory of Jennifer Mumford

Guest Editors: Stuart Geman, David Gu, Stanley Osher, Chi-Wang Shu, Yang Wang, and Shing-Tung Yau

A survey of exemplar-based texture synthesis

Pages: 89 – 148

DOI: https://dx.doi.org/10.4310/AMSA.2018.v3.n1.a4

Authors

Lara Raad (Dept. of Information & Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain)

Axel Davy (Centre de Mathématiques et Leurs Applications, Ecóle Normale Supérieure Paris-Saclay, Cachan, France)

Agnès Desolneux (Centre de Mathématiques et Leurs Applications, Ecóle Normale Supérieure Paris-Saclay, Cachan, France)

Jean-Michel Morel (Centre de Mathématiques et Leurs Applications, Ecóle Normale Supérieure Paris-Saclay, Cachan, France)

Abstract

Exemplar-based texture synthesis is the process of generating, from an input sample, new texture images of arbitrary size and which are perceptually equivalent to the sample. The two main approaches are statistics-based methods and patch re-arrangement methods. In the first class, a texture is characterized by a statistical signature; then, a random sampling conditioned to this signature produces genuinely different texture images. The second class boils down to a clever “copy-paste” procedure, which stitches together large regions of the sample. Hybrid methods try to combine ideas from both approaches to avoid their hurdles. The recent approaches using convolutional neural networks fit to this classification, some being statistical and others performing patch re-arrangement in the feature space. They produce impressive synthesis on various kinds of textures. Nevertheless, we found that most real textures are organized at multiple scales, with global structures revealed at coarse scales and highly varying details at finer ones. Thus, when confronted with large natural images of textures the results of state-of-the-art methods degrade rapidly, and the problem of modeling them remains wide open.

Keywords

texture synthesis, exemplar-based, image statistics, images patches, Julesz conjecture, convolutional neural networks

Full Text (PDF format)

We thank Rafael Grompone von Gioi for valuable corrections and suggestions, Arthur Leclaire and Yang Lu for their feedback and for helping produce some of the experiments, and the anonymous referees for valuable advice.

Received 18 June 2017

Published 27 March 2018