Param Schedulers¶

class classy_vision.optim.param_scheduler.ClassyParamScheduler(update_interval: classy_vision.optim.param_scheduler.classy_vision_param_scheduler.UpdateInterval)¶

Base class for Classy parameter schedulers.

update_interval¶: Specifies how often to update each parameter (before each epoch or each batch)

__init__(update_interval: classy_vision.optim.param_scheduler.classy_vision_param_scheduler.UpdateInterval)¶

Constructor for ClassyParamScheduler

Parameters: update_interval – Specifies the frequency of the param updates

classmethod from_config(config: Dict[str, Any]) → classy_vision.optim.param_scheduler.classy_vision_param_scheduler.ClassyParamScheduler¶

Instantiates a ClassyParamScheduler from a configuration.

Parameters: config – A configuration for the ClassyParamScheduler.
Returns: A ClassyParamScheduler instance.

class classy_vision.optim.param_scheduler.CompositeParamScheduler(schedulers: Sequence[fvcore.common.param_scheduler.ParamScheduler], lengths: Sequence[float], interval_scaling: Sequence[Union[classy_vision.optim.param_scheduler.composite_scheduler.IntervalScaling, str]], update_interval: classy_vision.optim.param_scheduler.classy_vision_param_scheduler.UpdateInterval = UpdateInterval.STEP)¶

Composite parameter scheduler composed of intermediate schedulers. Takes a list of schedulers and a list of lengths corresponding to percentage of training each scheduler should run for. Schedulers are run in order. All values in lengths should sum to 1.0.

Each scheduler also has a corresponding interval scale. If interval scale is ‘fixed’, the intermediate scheduler will be run without any rescaling of the time. If interval scale is ‘rescaled’, intermediate scheduler is run such that each scheduler will start and end at the same values as it would if it were the only scheduler. Default is ‘rescaled’ for all schedulers.

Example

schedulers = [
  ConstantParamScheduler(value=0.42),
  CosineParamScheduler(start_value=0.42, end_value=1e-4)
]
CompositeParamScheduler(
  schedulers=schedulers,
  interval_scaling=['rescaled', 'rescaled'],
  lengths=[0.3, 0.7])

The parameter value will be 0.42 for the first [0%, 30%) of steps, and then will cosine decay from 0.42 to 0.0001 for [30%, 100%) of training.

__init__(schedulers: Sequence[fvcore.common.param_scheduler.ParamScheduler], lengths: Sequence[float], interval_scaling: Sequence[Union[classy_vision.optim.param_scheduler.composite_scheduler.IntervalScaling, str]], update_interval: classy_vision.optim.param_scheduler.classy_vision_param_scheduler.UpdateInterval = UpdateInterval.STEP)¶

classmethod from_config(config: Dict[str, Any]) → classy_vision.optim.param_scheduler.composite_scheduler.CompositeParamScheduler¶

Instantiates a CompositeParamScheduler from a configuration.

Parameters: config – A configuration for a CompositeParamScheduler. See __init__() for parameters expected in the config.
Returns: A CompositeParamScheduler instance.

class classy_vision.optim.param_scheduler.ConstantParamScheduler(value: float)¶

Returns a constant value for a param.

classmethod from_config(config: Dict[str, Any]) → fvcore.common.param_scheduler.ParamScheduler¶

Instantiates a ClassyParamScheduler from a configuration.

Parameters: config – A configuration for the ClassyParamScheduler.
Returns: A ClassyParamScheduler instance.

class classy_vision.optim.param_scheduler.CosineParamScheduler(start_value: float, end_value: float)¶

Cosine decay or cosine warmup schedules based on start and end values. The schedule is updated based on the fraction of training progress. The schedule was proposed in ‘SGDR: Stochastic Gradient Descent with Warm Restarts’ (https://arxiv.org/abs/1608.03983). Note that this class only implements the cosine annealing part of SGDR, and not the restarts.

Example

CosineParamScheduler(start_value=0.1, end_value=0.0001)

classmethod from_config(config: Dict[str, Any]) → fvcore.common.param_scheduler.ParamScheduler¶

Instantiates a ClassyParamScheduler from a configuration.

Parameters: config – A configuration for the ClassyParamScheduler.
Returns: A ClassyParamScheduler instance.

class classy_vision.optim.param_scheduler.IntervalScaling(value)¶: An enumeration.

class classy_vision.optim.param_scheduler.LinearParamScheduler(start_value: float, end_value: float)¶

Linearly interpolates parameter between start_value and end_value. Can be used for either warmup or decay based on start and end values. The schedule is updated after every train step by default.

Example

LinearParamScheduler(start_value=0.0001, end_value=0.01)

Corresponds to a linear increasing schedule with values in [0.0001, 0.01)

classmethod from_config(config: Dict[str, Any]) → fvcore.common.param_scheduler.ParamScheduler¶

Instantiates a ClassyParamScheduler from a configuration.

Parameters: config – A configuration for the ClassyParamScheduler.
Returns: A ClassyParamScheduler instance.

class classy_vision.optim.param_scheduler.MultiStepParamScheduler(values: List[float], num_updates: Optional[int] = None, milestones: Optional[List[int]] = None)¶

Takes a predefined schedule for a param value, and a list of epochs or steps which stand for the upper boundary (excluded) of each range.

Example

MultiStepParamScheduler(
  values=[0.1, 0.01, 0.001, 0.0001],
  milestones=[30, 60, 80, 120]
)

Then the param value will be 0.1 for epochs 0-29, 0.01 for epochs 30-59, 0.001 for epochs 60-79, 0.0001 for epochs 80-120. Note that the length of values must be equal to the length of milestones plus one.

classmethod from_config(config: Dict[str, Any]) → fvcore.common.param_scheduler.ParamScheduler¶

Instantiates a ClassyParamScheduler from a configuration.

Parameters: config – A configuration for the ClassyParamScheduler.
Returns: A ClassyParamScheduler instance.

class classy_vision.optim.param_scheduler.ParamScheduler¶

Base class for parameter schedulers. A parameter scheduler defines a mapping from a progress value in [0, 1) to a number (e.g. learning rate).

__call__(where: float) → float¶

Get the value of the param for a given point at training.

We update params (such as learning rate) based on the percent progress of training completed. This allows a scheduler to be agnostic to the exact length of a particular run (e.g. 120 epochs vs 90 epochs), as long as the relative progress where params should be updated is the same. However, it assumes that the total length of training is known.

Parameters: where – A float in [0,1) that represents how far training has progressed

class classy_vision.optim.param_scheduler.PolynomialDecayParamScheduler(base_value: float, power: float)¶

Decays the param value after every epoch according to a polynomial function with a fixed power. The schedule is updated after every train step by default.

Example

PolynomialDecayParamScheduler(base_value=0.1, power=0.9)

Then the param value will be 0.1 for epoch 0, 0.099 for epoch 1, and so on.

classmethod from_config(config: Dict[str, Any]) → fvcore.common.param_scheduler.ParamScheduler¶

Instantiates a ClassyParamScheduler from a configuration.

Parameters: config – A configuration for the ClassyParamScheduler.
Returns: A ClassyParamScheduler instance.

class classy_vision.optim.param_scheduler.StepParamScheduler(num_updates: Union[int, float], values: List[float])¶

Takes a fixed schedule for a param value. If the length of the fixed schedule is less than the number of epochs, then the epochs are divided evenly among the param schedule. The schedule is updated after every train epoch by default.

Example

StepParamScheduler(values=[0.1, 0.01, 0.001, 0.0001], num_epochs=120)

Then the param value will be 0.1 for epochs 0-29, 0.01 for epochs 30-59, 0.001 for epoch 60-89, 0.0001 for epochs 90-119.

classmethod from_config(config: Dict[str, Any]) → fvcore.common.param_scheduler.ParamScheduler¶

Instantiates a ClassyParamScheduler from a configuration.

Parameters: config – A configuration for the ClassyParamScheduler.
Returns: A ClassyParamScheduler instance.

class classy_vision.optim.param_scheduler.StepWithFixedGammaParamScheduler(base_value: float, num_decays: int, gamma: float, num_updates: int)¶

Decays the param value by gamma at equal number of steps so as to have the specified total number of decays.

Example

StepWithFixedGammaParamScheduler(
  base_value=0.1, gamma=0.1, num_decays=3, num_epochs=120)

Then the param value will be 0.1 for epochs 0-29, 0.01 for epochs 30-59, 0.001 for epoch 60-89, 0.0001 for epochs 90-119.

classmethod from_config(config: Dict[str, Any]) → fvcore.common.param_scheduler.ParamScheduler¶

Instantiates a ClassyParamScheduler from a configuration.

Parameters: config – A configuration for the ClassyParamScheduler.
Returns: A ClassyParamScheduler instance.

class classy_vision.optim.param_scheduler.UpdateInterval(value)¶

Enum for specifying update frequency for scheduler.

EPOCH¶

Update param before each epoch

Type: str

STEP¶

Update param before each optimizer step

Type: str

classmethod from_config(config: Dict[str, Any], default: Optional[classy_vision.optim.param_scheduler.classy_vision_param_scheduler.UpdateInterval] = None) → classy_vision.optim.param_scheduler.classy_vision_param_scheduler.UpdateInterval¶

Fetches the update interval from a config

Parameters

config – The config for the parameter scheduler
default – The value to use if the config doesn’t specify an update interval. If not set, STEP is used.

classy_vision.optim.param_scheduler.build_param_scheduler(config: Dict[str, Any]) → fvcore.common.param_scheduler.ParamScheduler¶

Builds a ParamScheduler from a config.

This assumes a ‘name’ key in the config which is used to determine what param scheduler class to instantiate. For instance, a config {“name”: “my_scheduler”, “foo”: “bar”} will find a class that was registered as “my_scheduler” (see register_param_scheduler()) and call .from_config on it.

classy_vision.optim.param_scheduler.register_param_scheduler(name, bypass_checks=False)¶

Registers a ParamScheduler subclass.

This decorator allows Classy Vision to instantiate a subclass of ParamScheduler from a configuration file, even if the class itself is not part of the Classy Vision framework. To use it, apply this decorator to a ParamScheduler subclass that implements a from_config classmethod, like this:

@register_param_scheduler('my_scheduler')
class MyParamScheduler(ParamScheduler):
    ...

To instantiate a param scheduler from a configuration file, see build_param_scheduler().

Classy Vision

Param Schedulers¶

Classy Vision

Navigation

Related Topics