master: Fixed hyperparameter collisions

[MatteoWohlrapp]

Introduced a unique id for every object, which is added to the parameter name.

[smilesun]

model JiGen is still missing?

[smilesun]

Trainers like mldg dial also have gamma, that can be done also with id?

[MatteoWohlrapp]

From what I can see, jigen has not hyper_init and hyper_update functions, so I assumed hyperparameters are not updated.

To the other question: In your initial issue, you split it into two parts. The first part, the gamma collision when combining trainers, is not addressed with this. The second part, where the naming of model parameters collides, is. Will need to think about the first part, as this won't work with ids

[smilesun]

From what I can see, jigen has not hyper_init and hyper_update functions, so I assumed hyperparameters are not updated.

To the other question: In your initial issue, you split it into two parts. The first part, the gamma collision when combining trainers, is not addressed with this. The second part, where the naming of model parameters collides, is. Will need to think about the first part, as this won't work with ids

you are right, i forgt, sorry, jigen is a subclass of dann

[smilesun]

I read the pr several times since it first launched, the key point i want to make sure is that this design will be consistent with extensions, for example, if we have hyperscheduler_dial_mldg trainer, training diva_hduva model.

[MatteoWohlrapp]

Solved gamma_reg naming collision by introducing functionality to pass different gamma values for different trainers and models based on the naming of the class. With this comes a slight change in how the command line or yaml arguments are passed, however, the previous format is still supported. Also added parameter printing for models and trainers.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.23%. Comparing base (faf7589) to head (0d62919).

❗ Current head 0d62919 differs from pull request most recent head bdf84c2

Please upload reports for the commit bdf84c2 to get more accurate results.

Additional details and impacted files

@@ Coverage Diff @@## master #806 +/- ##
==========================================
+ Coverage 95.17% 95.23% +0.05% 
==========================================
Files 128 129 +1 Lines 5080 5138 +58 ==========================================
+ Hits 4835 4893 +58 
Misses 245 245 

Flag	Coverage Δ
unittests	95.23% <100.00%> (+0.05%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[smilesun]

@MatteoWohlrapp , could you fix the yaml according to my review?

[MatteoWohlrapp]

Can't see a review

[smilesun]

see the screenshot, the same scene not on your side?

[smilesun]

@MatteoWohlrapp, the test coverage must be above 95%

currently

Attention: Patch coverage is 88.05970% with 8 lines in your changes are missing coverage. Please review.

[smilesun]

Codecov Report

Attention: Patch coverage is 88.05970% with 8 lines in your changes are missing coverage. Please review.

Project coverage is 95.00%. Comparing base (af8a238) to head (cc650ad).
Report is 4 commits behind head on master.

Files Patch % Lines
domainlab/models/a_model.py 78.57% 3 Missing ⚠️
domainlab/algos/trainers/a_trainer.py 33.33% 2 Missing ⚠️
domainlab/arg_parser.py 83.33% 2 Missing ⚠️
domainlab/utils/hyperparameter_retrieval.py 88.88% 1 Missing ⚠️
Additional details and impacted files
☔ View full report in Codecov by Sentry. 📢 Have feedback on the report? Share it here.

@MatteoWohlrapp see detailed coverage report in quotes.

[smilesun]

So Fbopt can run with changes from this branch? If that is the case I will merge this branch into master

[smilesun]

@MatteoWohlrapp

[smilesun]

actually, due to the specific implementation of DIVA, instead of $$\ell(\cdot) + \mu R(\cdot)$$ type of loss, for DIVA, it is $$\mu \ell(\cdot) + R(\cdot)$$, here $\mu$ is gamma_reg in the code

[MatteoWohlrapp]

But where is this specified for model diva. I can only find this initialization of diva which does not have a regularization :
model = mk_diva(list_str_y=task.list_str_y)(
node,
zd_dim=args.zd_dim,
zy_dim=args.zy_dim,
zx_dim=args.zx_dim,
list_d_tr=task.list_domain_tr,
gamma_d=args.gamma_d,
gamma_y=args.gamma_y,
beta_x=args.beta_x,
beta_y=args.beta_y,
beta_d=args.beta_d,
)
It's here

[smilesun]

it is specified here:

DomainLab/domainlab/models/model_diva.py

Line 152 in 5ff5fd3

return [

and

DomainLab/domainlab/models/model_vae_xyd_classif.py

Line 31 in faf7589

defmultiplier4task_loss(self):

[smilesun]

@MatteoWohlrapp , how about mldg?

[MatteoWohlrapp]

Will get the default values assigned

[smilesun]

is this behavior tested somewhere? in which test, for instance, now the gamma_reg value for mldg should be 1.0? @MatteoWohlrapp

[smilesun]

i made a pull request to test his, still under work @MatteoWohlrapp

#847

[smilesun]

@MatteoWohlrapp

Run poetry run python examples/api/jigen_dann_transformer.py
Traceback (most recent call last):
File "/home/runner/work/DomainLab/DomainLab/examples/api/jigen_dann_transformer.py", line 9, in <module>
from domainlab.mk_exp import mk_exp
File "/home/runner/work/DomainLab/DomainLab/domainlab/mk_exp.py", line 4, in <module>
from domainlab.arg_parser import mk_parser_main
File "/home/runner/work/DomainLab/DomainLab/domainlab/arg_parser.py", line 33
my_dict = {} if "=" in values:
^
SyntaxError: invalid syntax
Error: Process completed with exit code 1.

[smilesun]

@MatteoWohlrapp

Run poetry run python examples/api/jigen_dann_transformer.py
Traceback (most recent call last):
File "/home/runner/work/DomainLab/DomainLab/examples/api/jigen_dann_transformer.py", line 9, in <module>
from domainlab.mk_exp import mk_exp
File "/home/runner/work/DomainLab/DomainLab/domainlab/mk_exp.py", line 4, in <module>
from domainlab.arg_parser import mk_parser_main
File "/home/runner/work/DomainLab/DomainLab/domainlab/arg_parser.py", line 33
my_dict = {} if "=" in values:
^
SyntaxError: invalid syntax
Error: Process completed with exit code 1.

lookes like error introduced by my commit

[smilesun]

@MatteoWohlrapp

Run poetry run python examples/api/jigen_dann_transformer.py
Traceback (most recent call last):
File "/home/runner/work/DomainLab/DomainLab/examples/api/jigen_dann_transformer.py", line 9, in <module>
from domainlab.mk_exp import mk_exp
File "/home/runner/work/DomainLab/DomainLab/domainlab/mk_exp.py", line 4, in <module>
from domainlab.arg_parser import mk_parser_main
File "/home/runner/work/DomainLab/DomainLab/domainlab/arg_parser.py", line 33
my_dict = {} if "=" in values:
^
SyntaxError: invalid syntax
Error: Process completed with exit code 1.

lookes like error introduced by my commit

you corrected that already in the last two commits.