Skip to content

SignalToNoiseRatioContrastiveLoss

Signal-to-Noise Ratio: A Robust Distance Metric for Deep Metric Learning

losses.SignalToNoiseRatioContrastiveLoss(pos_margin=0.0, neg_margin=1.0, **options)

Equation:

\[ L = \left[\operatorname{SNR}(\mathbf{x}_a, \mathbf{x}_p) - m_{pos}\right]_+ + \left[m_{neg} - \operatorname{SNR}(\mathbf{x}_a, \mathbf{x}_n)\right]_+, \qquad \operatorname{SNR}(\mathbf{x}_a, \mathbf{x}) = \frac{\operatorname{Var}(\mathbf{x}_a - \mathbf{x})}{\operatorname{Var}(\mathbf{x}_a)} \]

Implementation notes: This is ContrastiveLoss with the SNRDistance: the noise-to-signal ratio replaces the Euclidean distance. The paper pairs it with ZeroMeanRegularizer as the embedding_regularizer.

For low-precision embeddings, the distance computes its variances and ratio in float32 before returning its usual result dtype. Small nonzero variances therefore remain usable instead of producing NaN distances that the default reducer would discard. Rows with truly zero variance remain undefined.

Paper settings: The paper adds the zero-mean regularizer, which is not enabled by default. Pass embedding_regularizer=regularizers.ZeroMeanRegularizer() for the paper's formulation.

Example:

import torch
from pytorch_metric_learning import losses, regularizers

loss_func = losses.SignalToNoiseRatioContrastiveLoss(
    embedding_regularizer=regularizers.ZeroMeanRegularizer()
)
embeddings = torch.randn(32, 128, requires_grad=True)
labels = torch.randint(0, 8, (32,))
loss = loss_func(embeddings, labels)
loss.backward()

Parameters:

  • pos_margin: The noise-to-signal ratio over which positive pairs contribute to the loss.
  • neg_margin: The noise-to-signal ratio under which negative pairs contribute to the loss.

Default distance:

  • SNRDistance(). This is the only compatible distance; another distance class raises TypeError.

Default reducer:

Reducer input:

  • pos_loss: One loss per (anchor, positive) pair, indexed by (anchors, positives).
  • neg_loss: One loss per (anchor, negative) pair, indexed by (anchors, negatives).