layered.optimization module

class GradientDecent

Bases: object

Adapt the weights in the opposite direction of the gradient to reduce the error.

__call__(weights, gradient, learning_rate=0.1)

class Momentum

Bases: object

Slow down changes of direction in the gradient by aggregating previous values of the gradient and multiplying them in.

__call__(gradient, rate=0.9)

class WeightDecay

Bases: object

Slowly moves each weight closer to zero for regularization. This can help the model to find simpler solutions.

__call__(weights, rate=0.0001)

class WeightTying(*groups)

Bases: object

Constraint groups of slices of the gradient to have the same value by averaging them. Should be applied to the initial weights and each gradient.

__call__(matrices)