论文标题
剪辑调整：迈向无衍生的及时学习与奖励的混合
Clip-Tuning: Towards Derivative-free Prompt Learning with a Mixture of Rewards
论文作者
论文摘要
无衍生的提示学习已成为一种轻巧的替代方案，可以提示调整，这仅需要模型推断才能优化提示。但是，现有工作并未充分利用大型预训练语言模型（PLM）的过度参数化特征。在本文中，我们提出了一种简单而有效的方法，该方法采用了多种冷冻的“稀释” PLM网络，以获得奖励的混合，从而推进了无衍生的无衍生品及时学习。稀薄的网络由所有的隐藏单元组成，这些单元在固定的辍学策略中幸存下来，其推论预测反映了对促使培训样本的部分观点。我们的方法的表现优于以前的无梯度提示学习方法，并与基于梯度的七个语言理解基准的基准在几乎没有弹片设置的情况下实现了平等。
Derivative-free prompt learning has emerged as a lightweight alternative to prompt tuning, which only requires model inference to optimize the prompts. However, existing work did not take full advantage of the over-parameterized characteristics of large pre-trained language models (PLMs). In this paper, we propose Clip-Tuning, a simple yet effective method that adopts diverse frozen "thinned" networks of PLMs to obtain a mixture of rewards and thus advance the derivative-free prompt learning. The thinned networks consist of all the hidden units that survive a stationary dropout strategy, whose inference predictions reflect an ensemble of partial views over prompted training samples. Our method outperforms previous gradient-free prompt learning methods and achieves parity with gradient-based counterparts on seven language understanding benchmarks under few-shot settings.
