Communications in Mathematical Sciences

Volume 19 (2021)

Number 5

A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients

Pages: 1167 – 1205

DOI: https://dx.doi.org/10.4310/CMS.2021.v19.n5.a1

Authors

Arnulf Jentzen (Department of Mathematics, ETH Zürich, Switzerland; Institute for Analysis and Numerics, University of Münster, Germany; and School of Data Science and Shenzhen Research Institute of Big Data, Chinese University of Hong Kong)

Diyora Salimova (Department of Mathematics & Department of Information Technology and Electrical Engineering, ETH Zürich, Switzerland)

Timo Welti (Department of Mathematics, ETH Zürich Switzerland; and D ONE Solutions AG, Zürich, Switzerland)

Abstract

In recent years deep artificial neural networks (DNNs) have been successfully employed in numerical simulations for a multitude of computational problems including, for example, object and face recognition, natural language processing, fraud detection, computational advertisement, and numerical approximations of partial differential equations (PDEs). These numerical simulations indicate that DNNs seem to have the fundamental flexibility to overcome the curse of dimensionality in the sense that the number of real parameters used to describe the DNN grows at most polynomially in both the reciprocal of the prescribed approximation accuracy $\varepsilon \gt 0$ and the dimension $d \in \mathbb{N}$ of the function which the DNN aims to approximate in such computational problems. There is also a large number of rigorous mathematical approximation results for artificial neural networks in the scientific literature but there are only a few special situations where results in the literature can rigorously justify the success of DNNs to approximate high-dimensional functions. The key contribution of this article is to reveal that DNNs do overcome the curse of dimensionality in the numerical approximation of Kolmogorov PDEs with constant diffusion and nonlinear drift coefficients. We prove that the number of parameters used to describe the employed DNN grows at most polynomially in both the reciprocal of the prescribed approximation accuracy $\varepsilon \gt 0$ and the PDE dimension $d \in \mathbb{N}$. A crucial ingredient in our proof is the fact that the artificial neural network used to approximate the solution of the PDE is indeed a deep artificial neural network with a large number of hidden layers.

Keywords

curse of dimensionality, partial differential equations, numerical approximation, Feynman–Kac, deep neural networks

2010 Mathematics Subject Classification

65C99, 65M75, 68T05

Received 26 March 2019

Accepted 28 November 2020

Published 11 November 2021