shuffling of the samples in the mk_loader funtiona

[sidulova]

Is there any chance we can have shuffle in the mk_loader function as an arg.parser argument?
For some of the models that we are implementing in DomId, we would like to have non randomized batching.

DomainLab/domainlab/tasks/utils_task.py

Line 49 in 592f084

def mk_loader(dset, bsize,

Thank you!

[smilesun]

@sidulova , at the moment not, but you can implement it like this

DomainLab/domainlab/arg_parser.py

Line 196 in 592f084

arg_group_jigen = parser.add_argument_group('jigen')

or simply add such an argument in the major args @agisga, but in which scenario you do not want to shuffle? From a distributed optimization point of view, no shuffle does not make sense.

Alternatively, the test domain, as well as training domain validation daaset, are not shuffled by default. See

DomainLab/domainlab/tasks/b_task.py

Line 51 in 592f084

self._loader_val = mk_loader(ddset_mix_val, args.bs,

@sidulova

[smilesun]

@sidulova , maybe you can also take a look at the API coding:

https://github.com/marrlab/DomainLab/blob/master/examples/api/jigen_dann_transformer.py

[agisga]

but in which scenario you do not want to shuffle? From a distributed optimization point of view, no shuffle does not make sense.

It's for a specific use case in digital pathology, where the h and w dimensions of whole slide images (WSI) are about 100,000 pixels, so the WSI is typically cut up into many small image patches, and we don't want to shuffle patches from different WSIs together in our case. Essentially, we are applying a graph neural network based method, where we need each batch to include only patches from the same WSI due to the type of graph construction used.

[sidulova]

@smilesun, we have implemented the necessary changes to address the following issue.
Would you mind looking at the pull request?
#792
Thank you~!