论文标题
双词反向广告，有效
Dual-Numbers Reverse AD, Efficiently
论文作者
论文摘要
如果双数字向前模式自动分化（AD）将每个标量值配对其切线导数，双数字 /反向模式 / AD尝试使用类似简单的想法来实现反向AD：通过将每个标量值配对将每个标量值配对与后propagator函数。 Brunel，Mazza和Pagani已经分析了其在高阶输入语言上的正确性和效率，但是这种分析是在自定义操作语义上，目前尚不清楚是否可以有效地实现它。我们从使用 /线性保理 /从具有正确复杂性的算法中优化双词反向模式AD的灵感来源，并在具有资源线性类型的标准功能语言中享受有效的实现，例如Haskell。除线性保理成分外，我们的优化步骤还包括功能编程社区的知名思想。此外，我们观察到与基于经典命令的反向广告以及Kmett的“ AD” Haskell库的联系，Krawiec等人最近分析了。我们通过提供分化Haskell98的大部分的性能实现来证明我们技术的实际使用。
Where dual-numbers forward-mode automatic differentiation (AD) pairs each scalar value with its tangent derivative, dual-numbers /reverse-mode/ AD attempts to achieve reverse AD using a similarly simple idea: by pairing each scalar value with a backpropagator function. Its correctness and efficiency on higher-order input languages have been analysed by Brunel, Mazza and Pagani, but this analysis was on a custom operational semantics for which it is unclear whether it can be implemented efficiently. We take inspiration from their use of /linear factoring/ to optimise dual-numbers reverse-mode AD to an algorithm that has the correct complexity and enjoys an efficient implementation in a standard functional language with resource-linear types, such as Haskell. Aside from the linear factoring ingredient, our optimisation steps consist of well-known ideas from the functional programming community. Furthermore, we observe a connection with classical imperative taping-based reverse AD, as well as Kmett's 'ad' Haskell library, recently analysed by Krawiec et al. We demonstrate the practical use of our technique by providing a performant implementation that differentiates most of Haskell98.
