论文标题
汉密尔顿蒙特卡洛粒子群优化器
Hamiltonian Monte Carlo Particle Swarm Optimizer
论文作者
论文摘要
我们介绍了汉密尔顿蒙特卡洛粒子群优化器（HMC-PSO），这是一种优化算法，可获得指数平均的动量PSO和HMC采样的好处。在模拟中，每个粒子与哈密顿动力学的位置和速度的耦合可以为探索和开发搜索空间提供广泛的自由。它还提供了一种出色的技术，可以在确保有效采样的同时探索高度非凸功能。我们将方法扩展到以封闭形式的深度神经网络（DNN）设置的近似误差梯度。我们讨论了耦合和将其性能与戈洛姆统治者问题和分类任务的最先进优化者的性能进行比较的方法。
We introduce the Hamiltonian Monte Carlo Particle Swarm Optimizer (HMC-PSO), an optimization algorithm that reaps the benefits of both Exponentially Averaged Momentum PSO and HMC sampling. The coupling of the position and velocity of each particle with Hamiltonian dynamics in the simulation allows for extensive freedom for exploration and exploitation of the search space. It also provides an excellent technique to explore highly non-convex functions while ensuring efficient sampling. We extend the method to approximate error gradients in closed form for Deep Neural Network (DNN) settings. We discuss possible methods of coupling and compare its performance to that of state-of-the-art optimizers on the Golomb's Ruler problem and Classification tasks.
