View source on GitHub |
Returns a tff.learning.optimizers.Optimizer
for Adagrad.
tff.learning.optimizers.build_adagrad(
learning_rate: optimizer.Float,
initial_preconditioner_value: optimizer.Float = 0.1,
epsilon: optimizer.Float = 1e-07
) -> tff.learning.optimizers.Optimizer
The Adagrad optimizer is based on Adaptive Subgradient Methods for Online Learning and Stochastic Optimization
The update rule given learning rate lr
, epsilon eps
, preconditioner s
,
weights w
and gradients g
is:
s = s + g**2
w = w - lr * g / sqrt(s + eps)