RankedListLoss¶
Ranked List Loss for Deep Metric Learning
losses.RankedListLoss(
margin, neg_temperature, imbalance=0.5, alpha=None, pos_temperature=0, **options
)
Equation:
Here \(d\) is the Euclidean distance between normalized embeddings, \(P^*_i\) holds the positives of \(\mathbf{x}_i\) with \(d_{ij} > \alpha - m\), \(N^*_i\) holds the negatives with \(d_{ik} < \alpha\), \(\lambda\) is imbalance, \(m\) is margin, and \(T_p\) and \(T_n\) are pos_temperature and neg_temperature.
Implementation notes: \(L_P\) and \(L_N\) are exact weighted averages,
computed as softmax-weighted sums over the violating pairs. Scores are
centered before temperature multiplication, with the appropriate direction for
positive and negative temperatures. Float16 and bfloat16 matrices use float32
for violations, weights, and the returned loss, preserving finite gradients at
large temperatures. An anchor without violating positives or without
violating negatives contributes 0 for that part. Each embedding is an anchor
against the rest of the batch. Labels are required; indices_tuple and
ref_emb raise ValueError.
Paper settings: The defaults pos_temperature=0 and alpha=None
correspond to the paper's RLL-Simpler variant, which it recommends for
exploratory analysis: positives are weighted uniformly and
\(\alpha = 1 + m/2\).
Example:
import torch
from pytorch_metric_learning import losses
loss_func = losses.RankedListLoss(margin=0.4, neg_temperature=10)
embeddings = torch.randn(32, 128, requires_grad=True)
labels = torch.randint(0, 8, (32,))
loss = loss_func(embeddings, labels)
loss.backward()
Parameters:
- margin: The margin \(m\) between the positive and negative boundaries.
- neg_temperature: The temperature \(T_n\) that weights negative pairs.
- imbalance: The tradeoff \(\lambda\) between the positive and negative
parts, between 0 and 1; other values raise
ValueError. - alpha: The negative boundary \(\alpha\).
Nonesets \(\alpha = 1 + m/2\). - pos_temperature: The temperature \(T_p\) that weights positive pairs.
Default distance:
Default reducer:
Reducer input:
- loss: One loss per anchor, indexed by anchor position in
embeddings.