论文标题
具有自适应漂移的随机梯度Langevin动力学算法
Stochastic Gradient Langevin Dynamics Algorithms with Adaptive Drifts
论文作者
论文摘要
贝叶斯深度学习提供了一种解决有关人工智能安全性（AI）的许多问题的原则方法，例如模型不确定性，模型可解释性和预测偏见。但是，由于缺乏有效的蒙特卡洛算法来从深神经网络（DNNS）的后部进行采样，因此，贝叶斯深度学习尚未为我们的AI系统提供动力。我们提出了一类自适应随机梯度马尔可夫链蒙特卡洛（SGMCMC）算法，其中漂移函数的偏置以增强从马鞍点逃脱，并且根据过去样品的梯度对偏见进行自适应调节。 We establish the convergence of the proposed algorithms under mild conditions, and demonstrate via numerical examples that the proposed algorithms can significantly outperform the existing SGMCMC algorithms, such as stochastic gradient Langevin dynamics (SGLD), stochastic gradient Hamiltonian Monte Carlo (SGHMC) and preconditioned SGLD, in both simulation and optimization tasks.
Bayesian deep learning offers a principled way to address many issues concerning safety of artificial intelligence (AI), such as model uncertainty,model interpretability, and prediction bias. However, due to the lack of efficient Monte Carlo algorithms for sampling from the posterior of deep neural networks (DNNs), Bayesian deep learning has not yet powered our AI system. We propose a class of adaptive stochastic gradient Markov chain Monte Carlo (SGMCMC) algorithms, where the drift function is biased to enhance escape from saddle points and the bias is adaptively adjusted according to the gradient of past samples. We establish the convergence of the proposed algorithms under mild conditions, and demonstrate via numerical examples that the proposed algorithms can significantly outperform the existing SGMCMC algorithms, such as stochastic gradient Langevin dynamics (SGLD), stochastic gradient Hamiltonian Monte Carlo (SGHMC) and preconditioned SGLD, in both simulation and optimization tasks.
