IntraPairVarianceLoss¶
Deep Metric Learning with Tuplet Margin Loss
losses.IntraPairVarianceLoss(pos_eps=0.01, neg_eps=0.01, **options)
Equation:
Here \(\theta_{ap}\) and \(\theta_{an}\) are anchor-positive and anchor-negative angles, \(\mu_{ap}\) and \(\mu_{an}\) are the batch means of \(\cos\theta_{ap}\) and \(\cos\theta_{an}\), and \([\cdot]_+ = \max(0, \cdot)\).
Implementation notes: The means run over the positive pairs and over the
negative pairs of the batch, or of the supplied indices_tuple. Gradients flow
through \(\mu_{ap}\) and \(\mu_{an}\). With a distance where smaller means closer,
the inequalities are reversed.
The paper combines this loss with TupletMarginLoss through MultipleLosses.
Example:
import torch
from pytorch_metric_learning import losses
loss_func = losses.MultipleLosses(
{"tuplet": losses.TupletMarginLoss(), "variance": losses.IntraPairVarianceLoss()},
weights={"tuplet": 1.0, "variance": 0.5},
)
embeddings = torch.randn(32, 128, requires_grad=True)
labels = torch.randint(0, 8, (32,))
loss = loss_func(embeddings, labels)
loss.backward()
Parameters:
- pos_eps: \(\epsilon\) in \(\mathcal{L}_{pos}\). The paper uses 0.01.
- neg_eps: \(\epsilon\) in \(\mathcal{L}_{neg}\). The paper uses 0.01.
Default distance:
Default reducer:
Reducer input:
- pos_loss: One loss per (anchor, positive) pair, indexed by
(anchors, positives). - neg_loss: One loss per (anchor, negative) pair, indexed by
(anchors, negatives).
Float16/bfloat16 selected pair values are widened to float32 before the mean, epsilon adjustment, and squared hinge. Terms stay in float32 through the configured reducer, preserving small nonzero penalties and avoiding premature square overflow. Float32 and float64 pair values retain their dtype.