论文标题
测试Hebbian Meta学习中的基因组瓶颈假说
Testing the Genomic Bottleneck Hypothesis in Hebbian Meta-Learning
论文作者
论文摘要
Hebbian Meta-Learning最近表现出了解决艰苦的加强学习问题的希望，使代理可以适应环境变化。但是，由于这些方法中的每个突触都可以学习一个非常具体的学习规则，因此将其推广到非常不同情况的能力可能会降低。我们假设，通过“基因组瓶颈”限制Hebbian学习规则的数量可以起到正常化的作用，从而使环境变化更好地概括。我们通过将Hebbian学习规则的数量从突触的数量中分离出来，并系统地改变Hebbian学习规则的数量来检验这一假设。本文的结果表明，同时学习Hebbian学习规则及其对突触的分配是一个困难的优化问题，导致在测试环境中的性能差。但是，对我们的并行研究发现，确实可以通过将类似规则聚集在一起来减少学习规则的数量。因此，如何最好地实施“基因组瓶颈”算法是一个重要的研究方向，需要进一步研究。
Hebbian meta-learning has recently shown promise to solve hard reinforcement learning problems, allowing agents to adapt to some degree to changes in the environment. However, because each synapse in these approaches can learn a very specific learning rule, the ability to generalize to very different situations is likely reduced. We hypothesize that limiting the number of Hebbian learning rules through a "genomic bottleneck" can act as a regularizer leading to better generalization across changes to the environment. We test this hypothesis by decoupling the number of Hebbian learning rules from the number of synapses and systematically varying the number of Hebbian learning rules. The results in this paper suggest that simultaneously learning the Hebbian learning rules and their assignment to synapses is a difficult optimization problem, leading to poor performance in the environments tested. However, parallel research to ours finds that it is indeed possible to reduce the number of learning rules by clustering similar rules together. How to best implement a "genomic bottleneck" algorithm is thus an important research direction that warrants further investigation.
