Optimizers

class classy_vision.optim.Adam(lr: float = 0.1, betas: Tuple[float, float] = (0.9, 0.999), eps: float = 1e-08, weight_decay: float = 0.0, amsgrad: bool = False)
__init__(lr: float = 0.1, betas: Tuple[float, float] = (0.9, 0.999), eps: float = 1e-08, weight_decay: float = 0.0, amsgrad: bool = False) → None

Constructor for ClassyOptimizer.

classmethod from_config(config: Dict[str, Any]) → classy_vision.optim.adam.Adam

Instantiates a Adam from a configuration.

Parameters

config – A configuration for a Adam. See __init__() for parameters expected in the config.

Returns

A Adam instance.

init_pytorch_optimizer(model) → None

Initialize the underlying torch.optim.Optimizer instance.

Using the provided model, create param groups for the optimizer with a weight decay override for params which should be left unregularized.

Note

Deriving classes should initialize the underlying Pytorch optimizer in this call. The simplest way to do this after a call to

super().init_pytorch_optimizer()

Warning

This should called only after the model has been moved to the correct device.

class classy_vision.optim.ClassyOptimizer

Base class for classy optimizers.

This wraps a torch.optim.Optimizer instance, handles learning rate scheduling by using a param_scheduler.ClassyParamScheduler and supports specifying regularized and unregularized param groups. Specifying unregularized params is especially useful to avoid applying weight decay on batch norm. See classy_vision.models.ClassyModel.get_optimizer_params() for more information.

Deriving classes can extend functionality be overriding the appropriate functions.

__init__() → None

Constructor for ClassyOptimizer.

backward(loss: torch.Tensor) → None

Computer gradients with respect to the loss.

Calls zero_grad() and then computes the gradient using torch.Tensor.backward. See torch.autograd for more information.

classmethod from_config(config: Dict[str, Any]) → classy_vision.optim.classy_optimizer.ClassyOptimizer

Instantiates a ClassyOptimizer from a configuration.

Parameters

config – A configuration for the ClassyOptimizer.

Returns

A ClassyOptimizer instance.

get_classy_state() → Dict[str, Any]

Get the state of the ClassyOptimizer.

The returned state is used for checkpointing.

Returns

A state dictionary containing the state of the optimizer.

init_pytorch_optimizer(model: classy_vision.models.classy_model.ClassyModel) → None

Initialize the underlying torch.optim.Optimizer instance.

Using the provided model, create param groups for the optimizer with a weight decay override for params which should be left unregularized.

Note

Deriving classes should initialize the underlying Pytorch optimizer in this call. The simplest way to do this after a call to

super().init_pytorch_optimizer()

Warning

This should called only after the model has been moved to the correct device.

set_classy_state(state: Dict[str, Any]) → None

Set the state of the ClassyOptimizer.

Parameters

state_dict – The state dictionary. Must be the output of a call to get_classy_state().

This is used to load the state of the optimizer from a checkpoint.

set_param_schedulers(param_schedulers: Dict[str, classy_vision.optim.param_scheduler.classy_vision_param_scheduler.ClassyParamScheduler]) → classy_vision.optim.classy_optimizer.ClassyOptimizer

Set the param schedulers for the Classy Optimizer

Parameters

param_schedulers – A dictionary of :class:`ClassyParamScheduler`s containing the parameter scheduler to use for every parameter.

Returns

self

step(closure: Optional[Callable] = None)

Performs a single optimization step.

See `torch.optim.Optimizer.step <https://pytorch.org/docs/stable/ optim.html#torch.optim.Optimizer.step>`_for more information.

Parameters

closure – A closure that re-evaluates the model and returns the loss

update_schedule_on_epoch(where: float) → None

Update the param schedule at the end of an epoch.

This should be called by the task at the end of every epoch to update the schedule of epoch based param schedulers (See param_scheduler.ClassyParamScheduler for more information).

Parameters

where – where we are in terms of training progress (output of tasks.ClassyTask.where())

update_schedule_on_step(where: float) → None

Update the param schedule at the end of a train step.

This should be called by the task at the end of every train step ( tasks.ClassyTask.train_step()) to update the schedule of step based param schedulers (See param_scheduler.ClassyParamScheduler for more information).

Parameters

where – where we are in terms of training progress (output of :method:`ClassyTask.where`)

zero_grad()

Clears the gradients of all optimized parameters.

See torch.optim.Optimizer.zero_grad for more information.

class classy_vision.optim.RMSProp(lr: float = 0.1, momentum: float = 0, weight_decay: float = 0, alpha: float = 0.99, eps: float = 1e-08, centered: bool = False)
__init__(lr: float = 0.1, momentum: float = 0, weight_decay: float = 0, alpha: float = 0.99, eps: float = 1e-08, centered: bool = False) → None

Constructor for ClassyOptimizer.

classmethod from_config(config: Dict[str, Any]) → classy_vision.optim.rmsprop.RMSProp

Instantiates a RMSProp from a configuration.

Parameters

config – A configuration for a RMSProp. See __init__() for parameters expected in the config.

Returns

A RMSProp instance.

init_pytorch_optimizer(model)

Initialize the underlying torch.optim.Optimizer instance.

Using the provided model, create param groups for the optimizer with a weight decay override for params which should be left unregularized.

Note

Deriving classes should initialize the underlying Pytorch optimizer in this call. The simplest way to do this after a call to

super().init_pytorch_optimizer()

Warning

This should called only after the model has been moved to the correct device.

class classy_vision.optim.SGD(lr: float = 0.1, momentum: float = 0.0, weight_decay: float = 0.0, nesterov: bool = False)
__init__(lr: float = 0.1, momentum: float = 0.0, weight_decay: float = 0.0, nesterov: bool = False)

Constructor for ClassyOptimizer.

classmethod from_config(config: Dict[str, Any]) → classy_vision.optim.sgd.SGD

Instantiates a SGD from a configuration.

Parameters

config – A configuration for a SGD. See __init__() for parameters expected in the config.

Returns

A SGD instance.

init_pytorch_optimizer(model)

Initialize the underlying torch.optim.Optimizer instance.

Using the provided model, create param groups for the optimizer with a weight decay override for params which should be left unregularized.

Note

Deriving classes should initialize the underlying Pytorch optimizer in this call. The simplest way to do this after a call to

super().init_pytorch_optimizer()

Warning

This should called only after the model has been moved to the correct device.

classy_vision.optim.build_optimizer(config)

Builds a ClassyOptimizer from a config.

This assumes a ‘name’ key in the config which is used to determine what optimizer class to instantiate. For instance, a config {“name”: “my_optimizer”, “foo”: “bar”} will find a class that was registered as “my_optimizer” (see register_optimizer()) and call .from_config on it.

Also builds the param schedulers passed in the config and associates them with the optimizer. The config should contain an optional “param_schedulers” key containing a dictionary of param scheduler configs, keyed by the parameter they control. Adds “num_epochs” to each of the scheduler configs and then calls build_param_scheduler() on each config in the dictionary.

classy_vision.optim.register_optimizer(name)

Registers a ClassyOptimizer subclass.

This decorator allows Classy Vision to instantiate a subclass of ClassyOptimizer from a configuration file, even if the class itself is not part of the Classy Vision framework. To use it, apply this decorator to a ClassyOptimizer subclass, like this:

@register_optimizer('my_optimizer')
class MyOptimizer(ClassyOptimizer):
    ...

To instantiate an optimizer from a configuration file, see build_optimizer().