Document configuration options used for analysis runtime

[pwolfram]

A config file that fully can reproduce an analysis should be output with MPAS-Analysis results to provide a reproducible-capability, e.g., MPAS-Analysis can be run with this config file to regenerate results.

This is needed because there are already a few config options that are generated in the code that may not be present in the config file. For instance, default values may change between MPAS-Analysis versions and this ensures that the precise parameters used for an analysis run are recorded.

[xylar]

Thanks for creating this issue @pwolfram. While I think it's related to #23 and #35 in that all 3 involve provenance, I don't think it would be a good idea to try to store the provenance from a run (other than things that are clearly intended as config options) in the output config file so #23 probably still requires additional files or metadata elsewhere that stores the required provenance and #35 seems to imply that the provenance data is stored in the resulting netcdf file, so not obviously related to config file provenance.

[xylar]

@pwolfram, I believe it should be sufficient to write out the config object to the log directory. Does that sound like an adequate solution?

[pwolfram]

@xylar, I think that is good for now. We may want something more complete later but this should do at least in the short term.

[xylar]

@pwolfram, what would be missing with my suggested solution? The config file stored in the log directory should contain exactly the configuration needed to re-run the analysis. Would you want to add additional config options that aren't used directly by the analysis but where provenance is stored?

[pwolfram]

Do we have the config file output the hash of the version of MPAS-Analysis used? This is probably something we are missing to meet the full requirement.

More broadly, the config file is intimately tied to source and is a key part of the configuration. Optimally we could run the "config file" as its own script that uses a particular version of MPAS-Analysis but this is out of scope.

[xylar]

Do we have the config file output the hash of the version of MPAS-Analysis used? This is probably something we are missing to meet the full requirement.

Yes, I think that's right, that the hash tag should be added as a config option for provenance. But I also don't think config files should become the primary place for storing the full provenance information. I would think the HTML files (once we are producing them) would be the clearer place to include this information.

More broadly, the config file is intimately tied to source and is a key part of the configuration. Optimally we could run the "config file" as its own script that uses a particular version of MPAS-Analysis but this is out of scope.

I do not think this will be possible without going far beyond the specs of config files as they currently exist. They are definitely not intended to be scripts and they do not support execution. I do not agree that this would be an optimal direction to move in. It also sounds very complicated and perhaps too ambitious for this project, given the time we have to give it.

[xylar]

Addressed (a long time ago) by #129