Please use this identifier to cite or link to this item:
http://hdl.handle.net/10397/106825
Title: | Deep nonparametric regression on approximate manifolds : nonasymptotic error bounds with polynomial prefactors | Authors: | Jiao, Y Shen, G Lin, Y Huang, J |
Issue Date: | Apr-2023 | Source: | Annals of statistics, Apr. 2023, v. 51, no. 2, p. 691-716 | Abstract: | We study the properties of nonparametric least squares regression using deep neural networks. We derive nonasymptotic upper bounds for the excess risk of the empirical risk minimizer of feedforward deep neural regression. Our error bounds achieve minimax optimal rate and improve over the existing ones in the sense that they depend polynomially on the dimension of the predictor, instead of exponentially on dimension. We show that the neural regression estimator can circumvent the curse of dimensionality under the assumption that the predictor is supported on an approximate low-dimensional manifold or a set with low Minkowski dimension. We also establish the optimal convergence rate under the exact manifold support assumption. We investigate how the prediction error of the neural regression estimator depends on the structure of neural networks and propose a notion of network relative efficiency between two types of neural networks, which provides a quantitative measure for evaluating the relative merits of different network structures. To establish these results, we derive a novel approximation error bound for the Hölder smooth functions using ReLU activated neural networks, which may be of independent interest. Our results are derived under weaker assumptions on the data distribution and the neural network structure than those in the existing literature. | Keywords: | Approximation error Curse of dimensionality Deep neural network Low-dimensional manifolds Network relative efficiency Nonasymptotic error bound |
Publisher: | Institute of Mathematical Statistics | Journal: | Annals of statistics | ISSN: | 0090-5364 | EISSN: | 2168-8966 | DOI: | 10.1214/23-AOS2266 | Rights: | © Institute of Mathematical Statistics, 2023 The following publication Yuling Jiao. Guohao Shen. Yuanyuan Lin. Jian Huang. "Deep nonparametric regression on approximate manifolds: Nonasymptotic error bounds with polynomial prefactors." Ann. Statist. 51 (2) 691 - 716, April 2023 is available at https://doi.org/10.1214/23-AOS2266. |
Appears in Collections: | Journal/Magazine Article |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
23-AOS2266.pdf | 404.93 kB | Adobe PDF | View/Open |
Page views
10
Citations as of Jun 30, 2024
Downloads
2
Citations as of Jun 30, 2024
SCOPUSTM
Citations
5
Citations as of Jun 21, 2024
WEB OF SCIENCETM
Citations
7
Citations as of Jun 27, 2024
![](/image/google_scholar.jpg)
Google ScholarTM
Check
Altmetric
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.