Robust Contrastive Learning against Noisy Views

This repository provides a PyTorch implementation of the Robust InfoNCE loss proposed in paper Robust Contrastive Learning against Noisy Views.

Requirements:

• PyTorch >= 1.5.0

Pseudo Code

The implementation only requires a small modification to the InfoNCE code.

# bsz : batch size (number of positive pairs)# pos : exponent for positive example, shape=[bsz]# neg : sum of exponents for negative examples, shape=[bsz]# q, lam : hyperparameters of RINCEinfo_nce_loss=-log(pos/ (pos+neg))
rince_loss=-pos**q/q+ (lam* (pos+neg))**q/q

ImagNet Experiments

The code can be found in ImageNet/moco-v3 and ImageNet/simclr.

Citation

If you find this repo useful for your research, please consider citing the paper

@article{chuang2022robust,
title={Robust Contrastive Learning against Noisy Views},
author={Chuang, Ching-Yao and Hjelm, R Devon and Wang, Xin and Vineet, Vibhav and Joshi, Neel and Torralba, Antonio and Jegelka, Stefanie and Song, Yale},
journal={arXiv preprint arXiv:2201.04309},
year={2022}
}

For any questions, please contact Ching-Yao Chuang (cychuang@mit.edu).