论文标题
高斯流程专家与SMC的混合物$^2 $
Mixtures of Gaussian Process Experts with SMC$^2$
论文作者
论文摘要
高斯流程是许多灵活的统计和机器学习模型的关键组成部分。但是，由于需要倒转和存储完整的协方差矩阵，它们表现出立方计算的复杂性和高内存约束。为了避免这种情况，已经考虑了高斯流程专家的混合物，其中数据点被分配给独立专家，从而通过允许基于较小的局部协方差矩阵来降低复杂性。此外，高斯流程专家的混合物大大培养了模型的灵活性，从而允许诸如非平稳性，异方差和不连续性等行为。在这项工作中，我们基于嵌套的顺序蒙特卡洛采样器构建了一种新的推理方法，以同时推断门控网络和高斯过程专家参数。与重要性采样相比，这大大改善了推断，尤其是在固定高斯过程不合适的情况下，同时仍然完全可行。
Gaussian processes are a key component of many flexible statistical and machine learning models. However, they exhibit cubic computational complexity and high memory constraints due to the need of inverting and storing a full covariance matrix. To circumvent this, mixtures of Gaussian process experts have been considered where data points are assigned to independent experts, reducing the complexity by allowing inference based on smaller, local covariance matrices. Moreover, mixtures of Gaussian process experts substantially enrich the model's flexibility, allowing for behaviors such as non-stationarity, heteroscedasticity, and discontinuities. In this work, we construct a novel inference approach based on nested sequential Monte Carlo samplers to simultaneously infer both the gating network and Gaussian process expert parameters. This greatly improves inference compared to importance sampling, particularly in settings when a stationary Gaussian process is inappropriate, while still being thoroughly parallelizable.
