论文标题
神经体重搜索可扩展的任务增量学习
Neural Weight Search for Scalable Task Incremental Learning
论文作者
论文摘要
任务增量学习旨在使系统能够在学习新任务的同时保持其先前学习的任务的绩效，从而解决灾难性遗忘的问题。一种有希望的方法是建立一个单独的网络或子网络以实现未来的任务。但是，由于为新任务节省了额外的权重以及如何解决此问题，这导致了不断增长的内存，这在任务增量学习中仍然是一个开放的问题。在本文中，我们介绍了一种新型的神经搜索技术，该技术设计了一个固定的搜索空间，可以在其中搜索冷冻重量的最佳组合，以端到端的方式为新任务构建新的模型，从而实现可扩展和可控制的内存增长。在两个基准测试（即Split-CIFAR-100和Cub-Sketches）上进行了广泛的实验，表明我们的方法在平均推理准确性和总记忆成本方面都达到了最先进的性能。
Task incremental learning aims to enable a system to maintain its performance on previously learned tasks while learning new tasks, solving the problem of catastrophic forgetting. One promising approach is to build an individual network or sub-network for future tasks. However, this leads to an ever-growing memory due to saving extra weights for new tasks and how to address this issue has remained an open problem in task incremental learning. In this paper, we introduce a novel Neural Weight Search technique that designs a fixed search space where the optimal combinations of frozen weights can be searched to build new models for novel tasks in an end-to-end manner, resulting in scalable and controllable memory growth. Extensive experiments on two benchmarks, i.e., Split-CIFAR-100 and CUB-to-Sketches, show our method achieves state-of-the-art performance with respect to both average inference accuracy and total memory cost.
