[BUG] Contrastive pretraining pairs anchors with the wrong positives/negatives

[ChrisW09]

Describe the bug
ContrastivePretrainer.contrastive_loss pairs anchors with the wrong positives/negatives (deeptab/training/pretraining.py:227-231 unpooled branch, 257-261 pooled branch).

The pair tensor is built as torch.cat(pairs, dim=0) = [positives of a0..aN | negatives of a0..aN], but the anchor tensor is built with embeddings.repeat_interleave(k_neighbors * len(pairs), dim=0) = [a0 × 2k, a1 × 2k, ...]. The two layouts don't line up: with defaults (use_positive=True, use_negative=True, so len(pairs) == 2), row j of the pair tensor holds the positives of anchor j // k, while the anchor at row j is e_{j // (2k)}.

Verified numerically: for N=4 anchors, k=2 neighbors, 12 of 16 (anchor, pair, label) triples reference the wrong anchor — e.g. CosineEmbeddingLoss pushes anchor e2 away from e3 (label −1) when e3 was actually a positive neighbor of a different anchor. The contrastive pretraining objective is effectively scrambled noise; it only aligns when exactly one of use_positive/use_negative is enabled.

The correct construction is embeddings.repeat_interleave(k_neighbors, dim=0).repeat(len(pairs), 1).

To Reproduce

import torch
N, k, n_pairs = 4, 2, 2
# row j of cat(pairs) belongs to anchor (j % (N*k)) // k
pair_anchor = torch.arange(N * k * n_pairs) % (N * k) // k
# row j of repeat_interleave(k * n_pairs) is anchor j // (k * n_pairs)
actual_anchor = torch.arange(N * k * n_pairs) // (k * n_pairs)
print((pair_anchor != actual_anchor).sum().item(), "of", N * k * n_pairs) # 12 of 16

Expected behavior
Each (anchor, pair, label) triple handed to CosineEmbeddingLoss must reference the anchor whose kNN produced that pair.

Screenshots
n/a

Desktop (please complete the following information):

• OS: macOS (Darwin 25.5.0, arm64)
• Python version: 3.11.15
• deeptab Version: 2.0.0 (main @ 4e6a359)

Additional context
Related bug in the same file: for classification pretraining (regression=False), get_knn (pretraining.py:182-193) stacks indices only for the valid subset (samples that have same-class and different-class neighbors), shape (N', k), while contrastive_loss still builds anchors and labels from the full batch size N — any batch containing a class-singleton row crashes CosineEmbeddingLoss with a shape mismatch, and the surviving indices no longer correspond to anchor positions.