论文标题
自然梯度在过度散热系统中的不变性特性
Invariance Properties of the Natural Gradient in Overparametrised Systems
论文作者
论文摘要
自然梯度场是一个媒介场，该矢量场居住在配备有杰出的Riemannian指标的模型上，例如Fisher-Rao公制，代表模型在该度量方面的目标函数的最陡峭上升的方向。实际上，一个人试图通过将普通梯度乘以与度量的革兰氏矩阵的倒数来获得参数空间的相应方向。我们将此矢量称为参数空间，为自然参数梯度。在本文中，我们研究了自然参数梯度的推动力等于自然梯度。此外，我们研究了自然参数梯度的不变特性。这两个问题均在过度隔离的环境中解决。
The natural gradient field is a vector field that lives on a model equipped with a distinguished Riemannian metric, e.g. the Fisher-Rao metric, and represents the direction of steepest ascent of an objective function on the model with respect to this metric. In practice, one tries to obtain the corresponding direction on the parameter space by multiplying the ordinary gradient by the inverse of the Gram matrix associated with the metric. We refer to this vector on the parameter space as the natural parameter gradient. In this paper we study when the pushforward of the natural parameter gradient is equal to the natural gradient. Furthermore we investigate the invariance properties of the natural parameter gradient. Both questions are addressed in an overparametrised setting.
