P2SGradLoss¶
P2SGrad: Refined Gradients for Optimizing Deep Face Models
losses.P2SGradLoss(
num_classes,
embedding_size,
*,
embedding_regularizer=None,
embedding_reg_weight=1.0,
)
Equation:
Here \(\cos\theta_{i,j}\) is the cosine between embedding \(i\) and the unit-norm weight vector of class \(j\), and \(C\) is num_classes.
Implementation notes: The paper specifies the gradient
\(\partial L / \partial \mathbf{x}_i\) directly; this mean squared error between
the cosines and the one-hot target produces it. The cosines are computed
directly from L2-normalized embeddings and weights. The weight rows are
renormalized to unit length in place, under torch.no_grad(), at every forward
pass. Labels are required; indices_tuple and ref_emb raise ValueError. An
empty batch gives a differentiable zero.
The class weights are the parameter weight, shaped
(num_classes, embedding_size) and initialized uniformly in \([-1, 1]\), then
normalized. Include loss_func.parameters() in an optimizer. The
placement rule of learnable weights
applies; this loss does not derive from BaseProxyLoss, has no weight
regularization arguments, and has no get_logits.
Example:
import torch
from pytorch_metric_learning import losses
loss_func = losses.P2SGradLoss(num_classes=10, embedding_size=128)
optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
embeddings = torch.randn(32, 128, requires_grad=True)
labels = torch.randint(0, 10, (32,))
loss = loss_func(embeddings, labels)
loss.backward()
optimizer.step()
Parameters:
- num_classes: The number of classes in the training set. Labels must be in
0..num_classes - 1. - embedding_size: The size of the embeddings passed to the loss.
- embedding_regularizer, embedding_reg_weight: See loss options.
Distance:
- None. The loss computes cosines directly, so it has no
distanceoption; passingdistanceraisesTypeError.
Reducer:
- None. The loss reduces its own value, so it has no
reduceroption; passingreducerraisesTypeError.
Loss terms:
- loss: The value of \(L\), with
indices=None.
Low-precision embedding normalization, cosine multiplication, and squared residuals are evaluated in float32 before returning the original scalar dtype. Autocast does not round the cosine matrix back to low precision, and its float32 mean-squared-error output is retained. This preserves nonzero learning signals near a class center. The original dtype-specific normalization epsilon and in-place, no-grad weight normalization remain unchanged.