论文标题
在模型不确定性下的非参数自适应贝叶斯随机控制
Nonparametric Adaptive Bayesian Stochastic Control Under Model Uncertainty
论文作者
论文摘要
在本文中，我们提出了一种新方法，用于在模型不确定性下解决离散时间随机的马尔可夫控制问题。通过利用Dirichlet过程，我们将基础随机过程的未知分布建模为随机概率度量，并以贝叶斯方式实现在线学习。我们的方法整合了优化和动态学习。在处理模型不确定性时，非参数框架使我们避免了通常在其他经典控制方法中发生的模型错误指定。然后，我们开发了一种数值算法来处理此设置中无限维状态空间，并利用高斯过程替代物来获得钟形递归中值函数的功能表示。我们还为最佳控制构建了单独的替代物，以消除样本外路径上的重复优化并带来计算加速。最后，我们证明了非参数贝叶斯框架的财务优势与诸如强大和时间一致的适应性之类的参数方法相比。
In this paper we propose a new methodology for solving a discrete time stochastic Markovian control problem under model uncertainty. By utilizing the Dirichlet process, we model the unknown distribution of the underlying stochastic process as a random probability measure and achieve online learning in a Bayesian manner. Our approach integrates optimizing and dynamic learning. When dealing with model uncertainty, the nonparametric framework allows us to avoid model misspecification that usually occurs in other classical control methods. Then, we develop a numerical algorithm to handle the infinitely dimensional state space in this setup and utilizes Gaussian process surrogates to obtain a functional representation of the value function in the Bellman recursion. We also build separate surrogates for optimal control to eliminate repeated optimizations on out-of-sample paths and bring computational speed-ups. Finally, we demonstrate the financial advantages of the nonparametric Bayesian framework compared to parametric approaches such as strong robust and time consistent adaptive.
