tfma.metrics.MetricComputation

MetricComputation represents one or more metric computations.

tfma.metrics.MetricComputation(
    keys: List[tfma.metrics.MetricKey],
    preprocessors: Optional[List[Preprocessor]],
    combiner: beam.CombineFn
)

The preprocessors are called with a PCollection of extracts (or list of extracts if query_key is used) to compute the initial combiner input state which is then passed to the combiner. This needs to be done in two steps because slicing happens between the call to the preprocessors and the combiner and this state may end up in multiple slices so we want the representation to be as efficient as possible. If the preprocessors are None, then StandardMetricInputs will be passed.

A MetricComputation is uniquely identified by the combination of the combiner's name and the keys. Duplicate computations will be removed automatically.

keys List of metric keys associated with computation. If the keys are defined as part of the computation then this may be empty in which case only the combiner name will be used for identifying computation uniqueness.
preprocessors Takes a extracts (or a list of extracts) as input (which typically will contain labels, predictions, example weights, and optionally features) and should return the initial state that the combiner will use as input. The output of a processor should only contain information needed by the combiner.
combiner Takes preprocessor output as input and outputs a tuple: (slice, metric results). The metric results should be a dict from MetricKey to value (float, int, distribution, ...).