SignalToNoiseRatioContrastiveLoss¶
Signal-to-Noise Ratio: A Robust Distance Metric for Deep Metric Learning
losses.SignalToNoiseRatioContrastiveLoss(pos_margin=0.0, neg_margin=1.0, **options)
Equation:
Implementation notes: This is ContrastiveLoss with
the SNRDistance: the noise-to-signal ratio
replaces the Euclidean distance. The paper pairs it with
ZeroMeanRegularizer as the
embedding_regularizer.
For low-precision embeddings, the distance computes its variances and ratio in float32 before returning its usual result dtype. Small nonzero variances therefore remain usable instead of producing NaN distances that the default reducer would discard. Rows with truly zero variance remain undefined.
Paper settings: The paper adds the zero-mean regularizer, which is not
enabled by default. Pass
embedding_regularizer=regularizers.ZeroMeanRegularizer() for the paper's
formulation.
Example:
import torch
from pytorch_metric_learning import losses, regularizers
loss_func = losses.SignalToNoiseRatioContrastiveLoss(
embedding_regularizer=regularizers.ZeroMeanRegularizer()
)
embeddings = torch.randn(32, 128, requires_grad=True)
labels = torch.randint(0, 8, (32,))
loss = loss_func(embeddings, labels)
loss.backward()
Parameters:
- pos_margin: The noise-to-signal ratio over which positive pairs contribute to the loss.
- neg_margin: The noise-to-signal ratio under which negative pairs contribute to the loss.
Default distance:
SNRDistance(). This is the only compatible distance; another distance class raisesTypeError.
Default reducer:
Reducer input:
- pos_loss: One loss per (anchor, positive) pair, indexed by
(anchors, positives). - neg_loss: One loss per (anchor, negative) pair, indexed by
(anchors, negatives).