Hello. Thank you for your great work and sharing your code.
I run your code several times, but i cannot reproduce accuracy in paper.
I get 74.3% final valdiation accuracy in the setting(teacher : resnet32x4 / student : resnet8x4), but paper says 75.25% (ICKD without Hinton KD).
I also run ICKD-C(w/o KL) on the other settings, is it hyper parameter issue on diverse settings?(such as beta2 which multipied on CC Loss)
And I found some different procedure between paper and code, in calculating ICC matrices and L_CC.
In your code, row-wise L2 normalization has been applied on ICC matrices and Loss is divided by (channel * batch size) not (channel * channel * batch size)
Please let me know, if there are some implementation detail or error.
Thank you
Hello. Thank you for your great work and sharing your code.
I run your code several times, but i cannot reproduce accuracy in paper.
I get 74.3% final valdiation accuracy in the setting(teacher : resnet32x4 / student : resnet8x4), but paper says 75.25% (ICKD without Hinton KD).
I also run ICKD-C(w/o KL) on the other settings, is it hyper parameter issue on diverse settings?(such as beta2 which multipied on CC Loss)
And I found some different procedure between paper and code, in calculating ICC matrices and L_CC.
In your code, row-wise L2 normalization has been applied on ICC matrices and Loss is divided by (channel * batch size) not (channel * channel * batch size)
Please let me know, if there are some implementation detail or error.
Thank you