tfma.EvalResult

The result of a single model analysis run.

tfma.EvalResult(
    slicing_metrics,
    plots,
    attributions,
    config,
    data_location,
    file_format,
    model_location
)

slicing_metrics a list of tfma.SlicedMetrics, containing metric values for each slice.
plots List of slice-plot pairs.
attributions List of SlicedAttributions containing attribution values for each slice.
config The config containing slicing and metrics specification.
data_location Optional location for data used with config.
file_format Optional format for data used with config.
model_location Optional location(s) for model(s) used with config.

Methods

get_attributions_for_all_slices

View source

get_attributions_for_all_slices(
    metric_name: str = '',
    output_name: str = '',
    class_id: Optional[int] = None,
    k: Optional[int] = None,
    top_k: Optional[int] = None
) -> Dict[str, AttributionsByFeatureKey]

Get attribution feature keys and values for every slice.

Args
metric_name Name of metric to get attributions for. Optional if only one metric used.
output_name The name of the output (optional, only used for multi-output models).
class_id Used with multi-class metrics to identify a specific class ID.
k Used with multi-class metrics to identify the kth predicted value.
top_k Used with multi-class and ranking metrics to identify top-k predicted values.

Returns
Dictionary mapping slices to attribution feature keys and values.

get_attributions_for_slice

View source

get_attributions_for_slice(
    slice_name: slicer.SliceKeyType = (),
    metric_name: str = '',
    output_name: str = '',
    class_id: Optional[int] = None,
    k: Optional[int] = None,
    top_k: Optional[int] = None
) -> Union[AttributionsByFeatureKey, None]

Get attribution features names and values for a slice.

Args
slice_name A tuple of the form (column, value), indicating which slice to get attributions from. Optional; if excluded, use overall slice.
metric_name Name of metric to get attributions for. Optional if only one metric used.
output_name The name of the output. Optional, only used for multi-output models.
class_id Used with multi-class models to identify a specific class ID.
k Used with multi-class models to identify the kth predicted value.
top_k Used with multi-class models to identify top-k attribution values.

Returns
Dictionary containing feature keys and values for the specified slice.

Raises
ValueError If metric_name is required.

get_metric_names

View source

get_metric_names() -> Sequence[str]

Get names of metrics.

Returns
List of metric names.

get_metrics_for_all_slices

View source

get_metrics_for_all_slices(
    output_name: str = '',
    class_id: Optional[int] = None,
    k: Optional[int] = None,
    top_k: Optional[int] = None
) -> Dict[str, MetricsByTextKey]

Get metric names and values for every slice.

Args
output_name The name of the output (optional, only used for multi-output models).
class_id Used with multi-class metrics to identify a specific class ID.
k Used with multi-class metrics to identify the kth predicted value.
top_k Used with multi-class and ranking metrics to identify top-k predicted values.

Returns
Dictionary mapping slices to metric names and values.

get_metrics_for_slice

View source

get_metrics_for_slice(
    slice_name: slicer.SliceKeyType = (),
    output_name: str = '',
    class_id: Optional[int] = None,
    k: Optional[int] = None,
    top_k: Optional[int] = None
) -> Union[MetricsByTextKey, None]

Get metric names and values for a slice.

Args
slice_name A tuple of the form (column, value), indicating which slice to get metrics from. Optional; if excluded, return overall metrics.
output_name The name of the output. Optional, only used for multi-output models.
class_id Used with multi-class metrics to identify a specific class ID.
k Used with multi-class metrics to identify the kth predicted value.
top_k Used with multi-class and ranking metrics to identify top-k predicted values.

Returns
Dictionary containing metric names and values for the specified slice.

get_slice_names

View source

get_slice_names() -> Sequence[str]

Get names of slices.

Returns
List of slice names.