View source on GitHub |
Computes focal cross-entropy loss between true labels and predictions.
Inherits From: Loss
tf.keras.losses.BinaryFocalCrossentropy(
apply_class_balancing=False,
alpha=0.25,
gamma=2.0,
from_logits=False,
label_smoothing=0.0,
axis=-1,
reduction=losses_utils.ReductionV2.AUTO,
name='binary_focal_crossentropy'
)
Binary cross-entropy loss is often used for binary (0 or 1) classification tasks. The loss function requires the following inputs:
y_true
(true label): This is either 0 or 1.y_pred
(predicted value): This is the model's prediction, i.e, a single floating-point value which either represents a logit, (i.e, value in [-inf, inf] whenfrom_logits=True
) or a probability (i.e, value in[0., 1.]
whenfrom_logits=False
).
According to Lin et al., 2018, it helps to apply a "focal factor" to down-weight easy examples and focus more on hard examples. By default, the focal tensor is computed as follows:
focal_factor = (1 - output) ** gamma
for class 1
focal_factor = output ** gamma
for class 0
where gamma
is a focusing parameter. When gamma=0
, this function is
equivalent to the binary crossentropy loss.
With the compile()
API:
model.compile(
loss=tf.keras.losses.BinaryFocalCrossentropy(gamma=2.0, from_logits=True),
....
)
As a standalone function:
# Example 1: (batch_size = 1, number of samples = 4)
y_true = [0, 1, 0, 0]
y_pred = [-18.6, 0.51, 2.94, -12.8]
loss = tf.keras.losses.BinaryFocalCrossentropy(gamma=2,
from_logits=True)
loss(y_true, y_pred).numpy()
0.691
# Apply class weight
loss = tf.keras.losses.BinaryFocalCrossentropy(
apply_class_balancing=True, gamma=2, from_logits=True)
loss(y_true, y_pred).numpy()
0.51
# Example 2: (batch_size = 2, number of samples = 4)
y_true = [[0, 1], [0, 0]]
y_pred = [[-18.6, 0.51], [2.94, -12.8]]
# Using default 'auto'/'sum_over_batch_size' reduction type.
loss = tf.keras.losses.BinaryFocalCrossentropy(gamma=3,
from_logits=True)
loss(y_true, y_pred).numpy()
0.647
# Apply class weight
loss = tf.keras.losses.BinaryFocalCrossentropy(
apply_class_balancing=True, gamma=3, from_logits=True)
loss(y_true, y_pred).numpy()
0.482
# Using 'sample_weight' attribute with focal effect
loss = tf.keras.losses.BinaryFocalCrossentropy(gamma=3,
from_logits=True)
loss(y_true, y_pred, sample_weight=[0.8, 0.2]).numpy()
0.133
# Apply class weight
loss = tf.keras.losses.BinaryFocalCrossentropy(
apply_class_balancing=True, gamma=3, from_logits=True)
loss(y_true, y_pred, sample_weight=[0.8, 0.2]).numpy()
0.097
# Using 'sum' reduction` type.
loss = tf.keras.losses.BinaryFocalCrossentropy(gamma=4,
from_logits=True,
reduction=tf.keras.losses.Reduction.SUM)
loss(y_true, y_pred).numpy()
1.222
# Apply class weight
loss = tf.keras.losses.BinaryFocalCrossentropy(
apply_class_balancing=True, gamma=4, from_logits=True,
reduction=tf.keras.losses.Reduction.SUM)
loss(y_true, y_pred).numpy()
0.914
# Using 'none' reduction type.
loss = tf.keras.losses.BinaryFocalCrossentropy(
gamma=5, from_logits=True,
reduction=tf.keras.losses.Reduction.NONE)
loss(y_true, y_pred).numpy()
array([0.0017 1.1561], dtype=float32)
# Apply class weight
loss = tf.keras.losses.BinaryFocalCrossentropy(
apply_class_balancing=True, gamma=5, from_logits=True,
reduction=tf.keras.losses.Reduction.NONE)
loss(y_true, y_pred).numpy()
array([0.0004 0.8670], dtype=float32)
Args | |
---|---|
apply_class_balancing
|
A bool, whether to apply weight balancing on the binary classes 0 and 1. |
alpha
|
A weight balancing factor for class 1, default is 0.25 as
mentioned in reference Lin et al., 2018. The weight for class 0 is
1.0 - alpha .
|
gamma
|
A focusing parameter used to compute the focal factor, default is
2.0 as mentioned in the reference
Lin et al., 2018.
|
from_logits
|
Whether to interpret y_pred as a tensor of
logit values. By default, we
assume that y_pred are probabilities (i.e., values in [0, 1] ).
|
label_smoothing
|
Float in [0, 1] . When 0 , no smoothing occurs. When >
0 , we compute the loss between the predicted labels and a smoothed
version of the true labels, where the smoothing squeezes the labels
towards 0.5 . Larger values of label_smoothing correspond to heavier
smoothing.
|
axis
|
The axis along which to compute crossentropy (the features axis).
Defaults to -1 .
|
reduction
|
Type of tf.keras.losses.Reduction to apply to
loss. Default value is AUTO . AUTO indicates that the reduction
option will be determined by the usage context. For almost all cases
this defaults to SUM_OVER_BATCH_SIZE . When used under a
tf.distribute.Strategy , except via Model.compile() and
Model.fit() , using AUTO or SUM_OVER_BATCH_SIZE
will raise an error. Please see this custom training tutorial
for more details.
|
name
|
Name for the op. Defaults to 'binary_focal_crossentropy'. |
Methods
from_config
@classmethod
from_config( config )
Instantiates a Loss
from its config (output of get_config()
).
Args | |
---|---|
config
|
Output of get_config() .
|
Returns | |
---|---|
A keras.losses.Loss instance.
|
get_config
get_config()
Returns the config dictionary for a Loss
instance.
__call__
__call__(
y_true, y_pred, sample_weight=None
)
Invokes the Loss
instance.
Args | |
---|---|
y_true
|
Ground truth values. shape = [batch_size, d0, .. dN] , except
sparse loss functions such as sparse categorical crossentropy where
shape = [batch_size, d0, .. dN-1]
|
y_pred
|
The predicted values. shape = [batch_size, d0, .. dN]
|
sample_weight
|
Optional sample_weight acts as a coefficient for the
loss. If a scalar is provided, then the loss is simply scaled by the
given value. If sample_weight is a tensor of size [batch_size] ,
then the total loss for each sample of the batch is rescaled by the
corresponding element in the sample_weight vector. If the shape of
sample_weight is [batch_size, d0, .. dN-1] (or can be
broadcasted to this shape), then each loss element of y_pred is
scaled by the corresponding value of sample_weight . (Note
ondN-1 : all loss functions reduce by 1 dimension, usually
axis=-1.)
|
Returns | |
---|---|
Weighted loss float Tensor . If reduction is NONE , this has
shape [batch_size, d0, .. dN-1] ; otherwise, it is scalar. (Note
dN-1 because all loss functions reduce by 1 dimension, usually
axis=-1.)
|
Raises | |
---|---|
ValueError
|
If the shape of sample_weight is invalid.
|