论文标题
在非组织不确定性下，对随机最佳控制的策略优化有限
Constrained Policy Optimization for Stochastic Optimal Control under Nonstationary Uncertainties
论文作者
论文摘要
本文为在非机构不确定性下对系统的最佳控制提供了一种有限的政策优化方法。我们介绍了一个假设，即我们称Markov嵌入式性，使我们能够将随机最佳控制问题作为策略优化问题在增强状态空间上。然后，通过应用函数近似，确定性采样和时间截断，无限维策略优化问题被近似为有限维非线性程序。通过使用自动分化和冷凝空间内点方法来解决近似问题。我们为近似问题的近似值和解决方案策略提出了几个概念性和实用的开放性问题。作为概念的证明，我们提供了一个数字示例，证明了通过所提出的方法获得的控制策略的性能。
This article presents a constrained policy optimization approach for the optimal control of systems under nonstationary uncertainties. We introduce an assumption that we call Markov embeddability that allows us to cast the stochastic optimal control problem as a policy optimization problem over the augmented state space. Then, the infinite-dimensional policy optimization problem is approximated as a finite-dimensional nonlinear program by applying function approximation, deterministic sampling, and temporal truncation. The approximated problem is solved by using automatic differentiation and condensed-space interior-point methods. We formulate several conceptual and practical open questions regarding the asymptotic exactness of the approximation and the solution strategies for the approximated problem. As a proof of concept, we provide a numerical example demonstrating the performance of the control policy obtained by the proposed method.
