pytrio.AdamParams

class AdamParams(BaseModel):
    learning_rate: float = 0.0001
    beta1: float = 0.9
    beta2: float = 0.95
    eps: float = 1e-12
    weight_decay: float = 0.0
    grad_clip_norm: float = 0.0

AdamParams configures the Adam optimizer and is passed to TrainingClient.optim_step().

training_client.optim_step(AdamParams(learning_rate=1e-4)).result()

Parameters

Parameter	Type	Default	Description
`learning_rate`	`float`	`1e-4`	Learning rate
`beta1`	`float`	`0.9`	Decay factor for the first moment estimate
`beta2`	`float`	`0.95`	Decay factor for the second moment estimate
`eps`	`float`	`1e-12`	Numerical stability term that prevents division by zero
`weight_decay`	`float`	`0.0`	Weight decay coefficient
`grad_clip_norm`	`float`	`0.0`	Upper bound for gradient clipping norm

Parameters

On this page