Version 3.0.4¶
Version 3.0.4 fixes numerical stability, training gradients, mining boundaries, and retrieval edge cases. Public signatures and minimum supported Python, PyTorch, and NumPy versions are unchanged.
Numerical fixes¶
SNRDistancecomputes low-precision differences and variances in float32, preventing small nonzero variances from becoming NaN distances and corruptingSignalToNoiseRatioContrastiveLossgradients (#790).LpRegularizer,ZeroMeanRegularizer, andSphericalEmbeddingRegularizerretain float32 statistics and nonempty scalars for low-precision inputs, preventing overflow before reduction or later regularization weighting (#791, #795).LargeMarginSoftmaxLossandSphereFaceLosswiden magnitude calculations before scaling logits, preserving representable values and gradients (#792).ManifoldLossevaluates ranking terms in log space to keep large finite margins from overflowing (#793).- ArcFace losses retain target-cosine derivatives in the linear continuation
and at zero margin.
DynamicArcFaceLossalso preserves power-law margins for extreme powers and closely spaced integer class counts (#794, #805). MultiSupConLosspreserves caller-supplied matrices with greater precision than the embeddings (#797).- NCA, ProxyNCA, DCL, and DCLW retain loss differences beside large common score offsets. Centered scaling also handles opposite finite extremes whose raw difference overflows (#799, #807).
RankedListLoss,SoftTripleLoss, andNormalizedSoftmaxLosspreserve representable results through extreme temperature scaling. SoftTriple supports both temperature signs; NormalizedSoftmax training continues to use the publicget_logitsextension hook (#800, #803, #804).- PNP variants retain very small penalties and gradients instead of losing them to cancellation (#801).
IntraPairVarianceLosswidens pair statistics,P2SGradLossforms cosine residuals before low-precision rounding, andCircleLosswidens adaptive weights and centers its log partitions (#808, #810, #811).- Shared normalization preserves directions and gradients for finite large embeddings. Dtype and shape bounds skip rescaling when the norm cannot overflow; P2SGrad uses the same normalization (#812).
Mining and retrieval¶
- Weighted triplet sampling rescales eligible weights before converting to float32, preserving relative probabilities for tiny or huge finite weights (#796).
- BatchEasyHardMiner and BatchHardMiner keep finite dtype-limit scores inside
the candidate set. BatchEasyHardMiner applies inclusive distance ranges to
the
"all"strategies as well (#798, #813). DistanceWeightedMinerwidens density arithmetic so high-dimensional, low-precision inputs do not silently lose eligible anchors (#806).- Empty retrieval queries work consistently with self-search, custom label comparisons, NDCG, and per-class reductions (#802).
- NDCG uses shared gain scaling to preserve ranking ratios for large and subnormal relevance gains (#809).
Observable changes¶
- Nonempty low-precision losses from the regularizers listed above, RankedListLoss, IntraPairVarianceLoss, NormalizedSoftmaxLoss, and CircleLoss retain float32 through reduction. Gradients still accumulate in the input and parameter dtypes; empty reductions retain their existing behavior.
- Exact seeded weighted triplet draws can change. Relative sampling probabilities are preserved.
- NormalizedSoftmax normalizes class weights once. Tiny class weights therefore follow the distance's documented epsilon without a second normalization.
The integration measurements compare loss-only forward/backward workloads on an A100. The largest measured float32 NT-Xent case (batch 1,024, dimension 512) costs about 0.34 ms and 2 MiB more for overflow-safe normalization. Proxy classification improved; results depend on the workload and do not measure a complete training pipeline.
Development dependencies¶
The development lock updates certifi to 2026.10.18, removing compromised emSign roots. The expired certifi applicability and uv release-age assessments are retired; uv 0.13.4 now meets the ordinary admission window.
Contributors¶
Thanks to Afloat16 for 23 fixes (#790–#812), and AryanRG for the all-strategy mining range fix (#813). All original contribution commits are preserved in this release.