Meters¶

class classy_vision.meters.AccuracyMeter(topk)¶

Meter to calculate top-k accuracy for single label/ multi label image classification task.

__init__(topk)¶

Parameters: topk – list of int k values.

classmethod from_config(config: Dict[str, Any]) → classy_vision.meters.accuracy_meter.AccuracyMeter¶

Instantiates a AccuracyMeter from a configuration.

Parameters: config – A configuration for a AccuracyMeter. See __init__() for parameters expected in the config.
Returns: A AccuracyMeter instance.

get_classy_state()¶: Contains the states of the meter.

property name¶: The name of the meter.

reset()¶

Resets any internal meter state.

Should normally be called at the end of a phase.

set_classy_state(state)¶

Set the state of the ClassyMeter.

Parameters: state_dict – The state dictionary. Must be the output of a call to get_classy_state().

This is used to load the state of the meter from a checkpoint.

sync_state()¶

Syncs state with all other meters in distributed training.

If not provided by child class this does nothing by default and meter only provides the local process stats. If implemented then the meter provides the global stats at last sync + any local updates since the last sync.

Warning

Calls to sync_state could involve communications via torch.distributed which can result in a loss of performance or deadlocks if not coordinated among threads.

update(model_output, target, **kwargs)¶

Parameters

model_output – tensor of shape (B, C) where each value is either logit or class probability.
target – tensor of shape (B, C), which is one-hot / multi-label encoded, or tensor of shape (B) / (B, 1), integer encoded

validate(model_output_shape, target_shape)¶

Validate the meter.

Checks if the meter can be calculated on the given model_output_shape and target_shape.

property value¶: Value of meter based on local state, can be any python object.

Note

If there are multiple training processes then this represents the local state of the meter. If sync_state() is implemented, then value will return the global state since the last sync PLUS any local unsynced updates that have occurred in the local process.

class classy_vision.meters.ClassyMeter¶

Base class to measure various metrics during training and testing phases.

This can include meters like Accuracy, Precision and Recall, etc.

__init__()¶

classmethod from_config(config: Dict[str, Any]) → classy_vision.meters.classy_meter.ClassyMeter¶

Instantiates a ClassyMeter using a configuration.

Parameters: config – A configuration for a ClassyMeter.
Returns: A ClassyMeter instance.

get_classy_state() → Dict[str, Any]¶

Get the state of the ClassyMeter.

The returned state is used for checkpointing.

Returns: A state dictionary containing the state of the meter.

property name: str¶: The name of the meter.

reset()¶

Resets any internal meter state.

Should normally be called at the end of a phase.

set_classy_state(state: Dict[str, Any]) → None¶

Set the state of the ClassyMeter.

Parameters: state_dict – The state dictionary. Must be the output of a call to get_classy_state().

This is used to load the state of the meter from a checkpoint.

sync_state() → None¶

Syncs state with all other meters in distributed training.

If not provided by child class this does nothing by default and meter only provides the local process stats. If implemented then the meter provides the global stats at last sync + any local updates since the last sync.

Warning

Calls to sync_state could involve communications via torch.distributed which can result in a loss of performance or deadlocks if not coordinated among threads.

update(model_output: torch.Tensor, target: torch.Tensor, **kwargs) → None¶

Updates any internal state of meter.

Should be called after each batch processing of each phase.

Parameters

model_output – Output of a ClassyModel.
target – Target provided by a dataloader from ClassyDataset.

validate(model_output_shape: Tuple, target_shape: Tuple) → None¶

Validate the meter.

Checks if the meter can be calculated on the given model_output_shape and target_shape.

property value: Any¶: Value of meter based on local state, can be any python object.

Note

If there are multiple training processes then this represents the local state of the meter. If sync_state() is implemented, then value will return the global state since the last sync PLUS any local unsynced updates that have occurred in the local process.

class classy_vision.meters.PrecisionAtKMeter(topk)¶

Meter to calculate top-k precision for single-label or multi-label image classification task. Note, ties are resolved randomly.

__init__(topk)¶

Parameters: topk – list of int k values.

classmethod from_config(config: Dict[str, Any]) → classy_vision.meters.precision_meter.PrecisionAtKMeter¶

Instantiates a PrecisionAtKMeter from a configuration.

Parameters: config – A configuration for a PrecisionAtKMeter. See __init__() for parameters expected in the config.
Returns: A PrecisionAtKMeter instance.

get_classy_state()¶: Contains the states of the meter.

property name¶: The name of the meter.

reset()¶

Resets any internal meter state.

Should normally be called at the end of a phase.

set_classy_state(state)¶

Set the state of the ClassyMeter.

Parameters: state_dict – The state dictionary. Must be the output of a call to get_classy_state().

This is used to load the state of the meter from a checkpoint.

sync_state()¶

Syncs state with all other meters in distributed training.

If not provided by child class this does nothing by default and meter only provides the local process stats. If implemented then the meter provides the global stats at last sync + any local updates since the last sync.

Warning

Calls to sync_state could involve communications via torch.distributed which can result in a loss of performance or deadlocks if not coordinated among threads.

update(model_output, target, **kwargs)¶

Parameters

model_output – tensor of shape (B, C) where each value is either logit or class probability.
target – tensor of shape (B, C), which is one-hot / multi-label encoded, or tensor of shape (B) / (B, 1), integer encoded

validate(model_output_shape, target_shape)¶

Validate the meter.

Checks if the meter can be calculated on the given model_output_shape and target_shape.

property value¶: Value of meter based on local state, can be any python object.

Note

If there are multiple training processes then this represents the local state of the meter. If sync_state() is implemented, then value will return the global state since the last sync PLUS any local unsynced updates that have occurred in the local process.

class classy_vision.meters.RecallAtKMeter(topk, target_is_one_hot=True, num_classes=None)¶

Meter to calculate top-k recall for single-label or multi-label image classification task.

__init__(topk, target_is_one_hot=True, num_classes=None)¶

Parameters: topk – list of int k values.

classmethod from_config(config: Dict[str, Any]) → classy_vision.meters.recall_meter.RecallAtKMeter¶

Instantiates a RecallAtKMeter from a configuration.

Parameters: config – A configuration for a RecallAtKMeter. See __init__() for parameters expected in the config.
Returns: A RecallAtKMeter instance.

get_classy_state()¶: Contains the states of the meter.

property name¶: The name of the meter.

reset()¶

Resets any internal meter state.

Should normally be called at the end of a phase.

set_classy_state(state)¶

Set the state of the ClassyMeter.

Parameters: state_dict – The state dictionary. Must be the output of a call to get_classy_state().

This is used to load the state of the meter from a checkpoint.

sync_state()¶

Syncs state with all other meters in distributed training.

If not provided by child class this does nothing by default and meter only provides the local process stats. If implemented then the meter provides the global stats at last sync + any local updates since the last sync.

Warning

Calls to sync_state could involve communications via torch.distributed which can result in a loss of performance or deadlocks if not coordinated among threads.

update(model_output, target, **kwargs)¶

Parameters

model_output – tensor of shape (B, C) where each value is either logit or class probability.
target – tensor of shape (B, C), which is one-hot / multi-label encoded, or tensor of shape (B) / (B, 1), integer encoded

validate(model_output_shape, target_shape)¶

Validate the meter.

Checks if the meter can be calculated on the given model_output_shape and target_shape.

property value¶: Value of meter based on local state, can be any python object.

Note

If there are multiple training processes then this represents the local state of the meter. If sync_state() is implemented, then value will return the global state since the last sync PLUS any local unsynced updates that have occurred in the local process.

class classy_vision.meters.VideoAccuracyMeter(topk, clips_per_video_train, clips_per_video_test)¶

Meter to calculate top-k video-level accuracy for single/multi label: video classification task.

Video-level accuarcy is computed by averaging clip-level predictions and compare the reslt with video-level groundtruth label.

__init__(topk, clips_per_video_train, clips_per_video_test)¶

Parameters

topk – list of int k values.
clips_per_video_train – No. of clips sampled per video at train time
clips_per_video_test – No. of clips sampled per video at test time

classmethod from_config(config: Dict[str, Any]) → classy_vision.meters.video_accuracy_meter.VideoAccuracyMeter¶

Instantiates a VideoAccuracyMeter from a configuration.

Parameters: config – A configuration for a VideoAccuracyMeter. See __init__() for parameters expected in the config.
Returns: A VideoAccuracyMeter instance.

property meter¶

Every video meter should implement to have its own internal meter.

It consumes the video level predictions and ground truth label, and compute the actual metrics.

Returns: An instance of ClassyMeter.

property name¶: The name of the meter.

classy_vision.meters.build_meter(config)¶

Builds a ClassyMeter from a config.

This assumes a ‘name’ key in the config which is used to determine what meter class to instantiate. For instance, a config {“name”: “my_meter”, “foo”: “bar”} will find a class that was registered as “my_meter” (see register_meter()) and call .from_config on it.

classy_vision.meters.register_meter(name, bypass_checks=False)¶

Registers a ClassyMeter subclass.

This decorator allows Classy Vision to instantiate a subclass of ClassyMeter from a configuration file, even if the class itself is not part of the Classy Vision framework. To use it, apply this decorator to a ClassyMeter subclass, like this:

@register_meter('accuracy')
class AccuracyMeter(ClassyMeter):
    ...

To instantiate a meter from a configuration file, see build_meter().

Classy Vision

Meters¶

Classy Vision

Navigation

Related Topics