CircleLoss¶
Circle Loss: A Unified Perspective of Pair Similarity Optimization
losses.CircleLoss(margin=0.4, gamma=80, **options)
Equation:
For each anchor, \(s_p^i\) and \(s_n^j\) are its cosine similarities to its \(K\) positives and \(L\) negatives.
where
Implementation notes: \(\alpha_p\) and \(\alpha_n\) are treated as constants for gradients. The two sums are computed per anchor with log-sum-exp. An anchor without a positive or without a negative has no loss term, so any reducer averages only anchors that have both. The default reducer also skips anchors whose loss is 0.
Example:
import torch
from pytorch_metric_learning import losses
loss_func = losses.CircleLoss(margin=0.25, gamma=256)
embeddings = torch.randn(32, 128, requires_grad=True)
labels = torch.randint(0, 8, (32,))
loss = loss_func(embeddings, labels)
loss.backward()
Parameters:
- margin: The relaxation factor \(m\), which controls the radius of the decision boundary. The paper uses 0.25 for face recognition and 0.4 for fine-grained image retrieval (birds, cars, and online products).
- gamma: The scale factor \(\gamma\). The paper uses 256 for face recognition and 80 for fine-grained image retrieval.
Default distance:
CosineSimilarity(). This is the only compatible distance; another distance class raisesTypeError.
Default reducer:
Reducer input:
- loss: One loss per anchor that has a positive and a negative, indexed by
anchor position in
embeddings.
Float16/bfloat16 pair scores are widened before adaptive-weight multiplication
and log-sum-exp. This prevents an intermediate gamma * alpha overflow from
turning finite pair logits into infinities or NaNs. Per-anchor losses retain the accumulation dtype through the reducer, including
the nonempty scalar result; adaptive weights remain detached for gradients.
Each selected logit partition is centered before log-sum-exp, preserving its softmax gradients when gamma makes the absolute logits large. The two detached offsets are combined before the centered partitions, retaining cancellation between attraction and repulsion without erasing the smaller partition terms.