View source on GitHub |
FlipRate is the rate at which predictions between models switch.
Inherits From: Metric
tfma.metrics.BooleanFlipRates(
threshold: float = _DEFAULT_FLIP_RATE_THRESHOLD,
flip_rate_name: str = FLIP_RATE_NAME,
neg_to_neg_flip_rate_name: str = NEG_TO_NEG_FLIP_RATE_NAME,
neg_to_pos_flip_rate_name: str = NEG_TO_POS_FLIP_RATE_NAME,
pos_to_neg_flip_rate_name: str = POS_TO_NEG_FLIP_RATE_NAME,
pos_to_pos_flip_rate_name: str = POS_TO_POS_FLIP_RATE_NAME
)
Given a pair of models and a threshold for converting continuous model outputs into boolean predictions, this metric will produce three numbers (keyed by separate MetricKeys):
- (symmetric) flip rate: The number of times the boolean predictions don't match, regardless of the direction of the flip.
- negative-to-positive flip rate: The rate at which the baseline model's boolean prediction is negative but the candidate model's is positive.
- positive-to-negative flip rate: The rate at which the baseline model's boolean prediction is positive but the candidate model's is negative.
Methods
computations
computations(
eval_config: Optional[tfma.EvalConfig
] = None,
schema: Optional[schema_pb2.Schema] = None,
model_names: Optional[List[str]] = None,
output_names: Optional[List[str]] = None,
sub_keys: Optional[List[Optional[SubKey]]] = None,
aggregation_type: Optional[AggregationType] = None,
class_weights: Optional[Dict[int, float]] = None,
example_weighted: bool = False,
query_key: Optional[str] = None
) -> tfma.metrics.MetricComputations
Creates computations associated with metric.
from_config
@classmethod
from_config( config: Dict[str, Any] ) -> 'Metric'
get_config
get_config() -> Dict[str, Any]
Returns serializable config.