Skip to content

Version 3.0.4

Version 3.0.4 fixes numerical stability, training gradients, mining boundaries, and retrieval edge cases. Public signatures and minimum supported Python, PyTorch, and NumPy versions are unchanged.

Numerical fixes

  • SNRDistance computes low-precision differences and variances in float32, preventing small nonzero variances from becoming NaN distances and corrupting SignalToNoiseRatioContrastiveLoss gradients (#790).
  • LpRegularizer, ZeroMeanRegularizer, and SphericalEmbeddingRegularizer retain float32 statistics and nonempty scalars for low-precision inputs, preventing overflow before reduction or later regularization weighting (#791, #795).
  • LargeMarginSoftmaxLoss and SphereFaceLoss widen magnitude calculations before scaling logits, preserving representable values and gradients (#792).
  • ManifoldLoss evaluates ranking terms in log space to keep large finite margins from overflowing (#793).
  • ArcFace losses retain target-cosine derivatives in the linear continuation and at zero margin. DynamicArcFaceLoss also preserves power-law margins for extreme powers and closely spaced integer class counts (#794, #805).
  • MultiSupConLoss preserves caller-supplied matrices with greater precision than the embeddings (#797).
  • NCA, ProxyNCA, DCL, and DCLW retain loss differences beside large common score offsets. Centered scaling also handles opposite finite extremes whose raw difference overflows (#799, #807).
  • RankedListLoss, SoftTripleLoss, and NormalizedSoftmaxLoss preserve representable results through extreme temperature scaling. SoftTriple supports both temperature signs; NormalizedSoftmax training continues to use the public get_logits extension hook (#800, #803, #804).
  • PNP variants retain very small penalties and gradients instead of losing them to cancellation (#801).
  • IntraPairVarianceLoss widens pair statistics, P2SGradLoss forms cosine residuals before low-precision rounding, and CircleLoss widens adaptive weights and centers its log partitions (#808, #810, #811).
  • Shared normalization preserves directions and gradients for finite large embeddings. Dtype and shape bounds skip rescaling when the norm cannot overflow; P2SGrad uses the same normalization (#812).

Mining and retrieval

  • Weighted triplet sampling rescales eligible weights before converting to float32, preserving relative probabilities for tiny or huge finite weights (#796).
  • BatchEasyHardMiner and BatchHardMiner keep finite dtype-limit scores inside the candidate set. BatchEasyHardMiner applies inclusive distance ranges to the "all" strategies as well (#798, #813).
  • DistanceWeightedMiner widens density arithmetic so high-dimensional, low-precision inputs do not silently lose eligible anchors (#806).
  • Empty retrieval queries work consistently with self-search, custom label comparisons, NDCG, and per-class reductions (#802).
  • NDCG uses shared gain scaling to preserve ranking ratios for large and subnormal relevance gains (#809).

Observable changes

  • Nonempty low-precision losses from the regularizers listed above, RankedListLoss, IntraPairVarianceLoss, NormalizedSoftmaxLoss, and CircleLoss retain float32 through reduction. Gradients still accumulate in the input and parameter dtypes; empty reductions retain their existing behavior.
  • Exact seeded weighted triplet draws can change. Relative sampling probabilities are preserved.
  • NormalizedSoftmax normalizes class weights once. Tiny class weights therefore follow the distance's documented epsilon without a second normalization.

The integration measurements compare loss-only forward/backward workloads on an A100. The largest measured float32 NT-Xent case (batch 1,024, dimension 512) costs about 0.34 ms and 2 MiB more for overflow-safe normalization. Proxy classification improved; results depend on the workload and do not measure a complete training pipeline.

Development dependencies

The development lock updates certifi to 2026.10.18, removing compromised emSign roots. The expired certifi applicability and uv release-age assessments are retired; uv 0.13.4 now meets the ordinary admission window.

Contributors

Thanks to Afloat16 for 23 fixes (#790–#812), and AryanRG for the all-strategy mining range fix (#813). All original contribution commits are preserved in this release.