论文标题
关于从复发性神经网络中有限状态机的可计算性，可学习性和可萃取性
On Computability, Learnability and Extractability of Finite State Machines from Recurrent Neural Networks
论文作者
论文摘要
这项工作旨在阐明有限状态机（FSM）和经常性神经网络（RNN）之间的连接。本主论文中检查的连接是三个方面：从复发性神经网络，可学习性方面和计算链接中的有限状态机器的可萃取性。关于前者，探索了RNN隐藏的状态空间的长期聚类假设，探索了识别普通语言的训练，并通过提供深度学习概括理论的最新进步的角度来探索了这一假设的新见解。至于可学习性，提出了更适合使用FSM近似RNN的问题的主动学习框架的扩展，以更好地将FSMS的RNN近似问题形式化。对该框架中的两种可能情况进行了理论分析。关于可计算性，给出了训练为语言模型的RNN与不同类型的加权有限状态机训练的RNN之间的新计算结果。
This work aims at shedding some light on connections between finite state machines (FSMs), and recurrent neural networks (RNNs). Examined connections in this master's thesis is threefold: the extractability of finite state machines from recurrent neural networks, learnability aspects and computationnal links. With respect to the former, the long-standing clustering hypothesis of RNN hidden state space when trained to recognize regular languages was explored, and new insights into this hypothesis through the lens of recent advances of the generalization theory of Deep Learning are provided. As for learnability, an extension of the active learning framework better suited to the problem of approximating RNNs with FSMs is proposed, with the aim of better formalizing the problem of RNN approximation by FSMs. Theoretical analysis of two possible scenarions in this framework were performed. With regard to computability, new computational results on the distance and the equivalence problem between RNNs trained as language models and different types of weighted finite state machines were given.
