论文标题
深层网络近似平滑功能
Deep Network Approximation for Smooth Functions
论文作者
论文摘要
本文建立了深层线性单元（RELU）网络的（几乎）最佳近似误差表征，以同时宽度和深度来平滑功能。为此，我们首先证明可以通过宽度$ \ MATHCAL {O}（N）$和DEPTH $ \ MATHCAL {o}（l）$具有近似错误$ \ MATHCAL {o \ MATHCAL {O}（O}（n^{ - l}）$的Deep Relu网络可以近似多元多项式。通过当地的泰勒扩展及其深层relu网络近似，我们表明具有宽度$ \ Mathcal {o}（n \ ln n）$和depth $ \ mathcal {o}（l \ ln l）$的深度relu网络大约$ f \ in C^s（[0,1] d）$大约$ f \ $ \ MATHCAL {O}（\ | f \ | _ {C^s（[0,1]^d）} n^{ - 2s/d} l^{ - 2s/d}）$。我们的估计是非反应的，因为它对于$ n \ in \ mathbb {n}^+$和$ l \ in \ mathbb {n}^+$的任意宽度和深度有效。
This paper establishes the (nearly) optimal approximation error characterization of deep rectified linear unit (ReLU) networks for smooth functions in terms of both width and depth simultaneously. To that end, we first prove that multivariate polynomials can be approximated by deep ReLU networks of width $\mathcal{O}(N)$ and depth $\mathcal{O}(L)$ with an approximation error $\mathcal{O}(N^{-L})$. Through local Taylor expansions and their deep ReLU network approximations, we show that deep ReLU networks of width $\mathcal{O}(N\ln N)$ and depth $\mathcal{O}(L\ln L)$ can approximate $f\in C^s([0,1]^d)$ with a nearly optimal approximation error $\mathcal{O}(\|f\|_{C^s([0,1]^d)}N^{-2s/d}L^{-2s/d})$. Our estimate is non-asymptotic in the sense that it is valid for arbitrary width and depth specified by $N\in\mathbb{N}^+$ and $L\in\mathbb{N}^+$, respectively.
