论文标题
贝叶斯桥回归的变异推断
Variational Inference for Bayesian Bridge Regression
论文作者
论文摘要
我们研究贝叶斯对回归模型的贝叶斯推断的自动分化变异推理（ADVI）的实施。桥梁方法使用$ \ell_α$ norm，$α\ in（0， +\ infty）$ in定义了对回归系数的巨大值的罚款，其中包括套索（$α= 1 $）和ridge $（α= 2）$罚款为特殊情况。完整的贝叶斯推理无缝地提供了所有模型参数的关节不确定性估计。尽管MCMC径流可用于桥梁回归，但对于大型数据集而言，它可能会很慢，特别是在高尺寸上。 ADVI实现允许在每次迭代（由于基于随机梯度的算法）处使用小批次数据，因此与MCMC相比，加快了计算时间。我们说明了使用B-Splines的非参数回归模型的方法，尽管该方法可用于基础函数的其他选择。一项仿真研究显示了所提出方法的主要特性。
We study the implementation of Automatic Differentiation Variational inference (ADVI) for Bayesian inference on regression models with bridge penalization. The bridge approach uses $\ell_α$ norm, with $α\in (0, +\infty)$ to define a penalization on large values of the regression coefficients, which includes the Lasso ($α= 1$) and ridge $(α= 2)$ penalizations as special cases. Full Bayesian inference seamlessly provides joint uncertainty estimates for all model parameters. Although MCMC aproaches are available for bridge regression, it can be slow for large dataset, specially in high dimensions. The ADVI implementation allows the use of small batches of data at each iteration (due to stochastic gradient based algorithms), therefore speeding up computational time in comparison with MCMC. We illustrate the approach on non-parametric regression models with B-splines, although the method works seamlessly for other choices of basis functions. A simulation study shows the main properties of the proposed method.
