Size mismatch

[hchoi71]

Hello Author,

Thanks for sharing this great work.

There's been an issue with size mismatch when I run your code (ickd) using different arch type (e.g. teacher: resnet50, student: MobileNetV2). With same arch type (e.g. teacher: resnet110, student: resnet20), it works fine.

I'd appreciate it if you could check this out.

Thanks,

[hchoi71]

Also, in ICKD.py it seems this embed_t has not been used, but why did you define this line? https://github.com/ADLab-AutoDrive/ICKD/blob/107935f3abce903f09e7c2dbd6202be5b1e3346b/Cifar100/distiller_zoo/ICKD.py#L35

self.embed_t = Embed(opt.t_dim, opt.feat_dim)

[ADLab-AutoDrive]

Hi, @hchoi71 , thanks for your attention. We have fixed the bug.

[hchoi71]

Thanks for fixing the bug.

I have one more question about the result reported in the paper.

I tried to get the same result using resnet110(teacher) and resnet20(student) with beta 2.5 as the following command.

python train_student.py --path_t ./save/models/resnet110_vanilla/ckpt_epoch_240.pth --distill ickd --model_s resnet20 -a 0 -b 2.5 --trial 1

But I couldn't reach up to 71.68 (w/o KL). I got only 70.20%. Could you please guide me on how to get this correctly? I haven't changed anything in your code.

Thanks

[ADLab-AutoDrive]

Thanks for fixing the bug.

I have one more question about the result reported in the paper.

I tried to get the same result using resnet110(teacher) and resnet20(student) with beta 2.5 as the following command.

python train_student.py --path_t ./save/models/resnet110_vanilla/ckpt_epoch_240.pth --distill ickd --model_s resnet20 -a 0 -b 2.5 --trial 1

But I couldn't reach up to 71.68 (w/o KL). I got only 70.20%. Could you please guide me on how to get this correctly? I haven't changed anything in your code.

Thanks

The random seed in trials has influence the result, and we have fixed the seed. Also the hyper parameter like beta1 and beta2 should be adjusted to the result.

[hchoi71]

Thanks for fixing the bug.
I have one more question about the result reported in the paper.
I tried to get the same result using resnet110(teacher) and resnet20(student) with beta 2.5 as the following command.
python train_student.py --path_t ./save/models/resnet110_vanilla/ckpt_epoch_240.pth --distill ickd --model_s resnet20 -a 0 -b 2.5 --trial 1
But I couldn't reach up to 71.68 (w/o KL). I got only 70.20%. Could you please guide me on how to get this correctly? I haven't changed anything in your code.
Thanks

The random seed in trials has influence the result, and we have fixed the seed. Also the hyper parameter like beta1 and beta2 should be adjusted to the result.

Why do we set the random seed to get reported results? Are all reported results the best accuracy for one trial?