论文标题
Black-box安全验证自主系统：多保化增强学习方法
Black-Box Safety Validation of Autonomous Systems: A Multi-Fidelity Reinforcement Learning Approach
论文作者
论文摘要
在社会中，自主和半自治的代理人的使用日益增加，因此对验证其安全至关重要。但是，使用它们的复杂场景可能使正式验证不可能。为了应对这一挑战，采用基于模拟的安全验证来测试复杂系统。使用强化学习的最新方法容易过度剥削已知的失败，并且在失败空间中缺乏覆盖范围。为了解决这一限制，已经定义了一种称为“知识MDP”的马尔可夫决策过程。这种方法考虑到通过“知道知识知道的”框架来估算系统知识的学习模型及其元数据（例如样本计数）。已经开发了一种将双向学习扩展到多个模拟器保真度的新型算法，以解决安全验证问题。通过案例研究证明了这种方法的有效性，在该案例研究中，对敌人进行了训练以在网格世界环境中拦截测试模型。 Monte Carlo试验比较了所提出的算法的样本效率与单性模拟器学习的样本效率，并展示了将有关学习模型的知识纳入决策过程的重要性。
The increasing use of autonomous and semi-autonomous agents in society has made it crucial to validate their safety. However, the complex scenarios in which they are used may make formal verification impossible. To address this challenge, simulation-based safety validation is employed to test the complex system. Recent approaches using reinforcement learning are prone to excessive exploitation of known failures and a lack of coverage in the space of failures. To address this limitation, a type of Markov decision process called the "knowledge MDP" has been defined. This approach takes into account both the learned model and its metadata, such as sample counts, in estimating the system's knowledge through the "knows what it knows" framework. A novel algorithm that extends bidirectional learning to multiple fidelities of simulators has been developed to solve the safety validation problem. The effectiveness of this approach is demonstrated through a case study in which an adversary is trained to intercept a test model in a grid-world environment. Monte Carlo trials compare the sample efficiency of the proposed algorithm to learning with a single-fidelity simulator and show the importance of incorporating knowledge about learned models into the decision-making process.
