Add support for short-term archiving

[xylar]

This requires new input subdirectories for run, ocean history, and sea ice history. It also requires a separate StreamsFile object for output in run and in history subdirectories in each analysis task.

A new helper function has been added under shared/analysis_task.py to set up the common functionality of all tasks. Additional setup common to all sea ice tasks has been added in sea_ice/utility.py

The buildConfigFullPath function has been changed to build_config_full_path everywhere.

A make_directories utility function has been added to make all directories in a path if they don't already exist.

[xylar]

This branch is has #151 as its base so shouldn't be merged until after that PR. A review will be easier after that PR has been merged and this one has been rebased.

[xylar]

This PR is intended to address #143.

[xylar]

Testing

I tested this PR by simulating short-term archiving of the QU240 test case. Here is the input section of my config file:

[input]
baseDirectory = /media/xylar/extra_data/data_overflow/analysis/GMPAS-QU240/sim_archiving
runSubdirectory = run
oceanHistorySubdirectory = ocn/hist
seaIceHistorySubdirectory = ice/hist
oceanNamelistFileName = run/mpas-o_in
oceanStreamsFileName = run/streams.ocean
seaIceNamelistFileName = run/mpas-cice_in
seaIceStreamsFileName = run/streams.cice
mpasMeshName = QU240

Analysis ran as normal.

Note, as this run doesn't support streamfunctionMOC, this task was not tested (yet).

[xylar]

@milenaveneziani, this has been rebased and should be ready to test out. Until we actually have a run with short-term archiving, you can fake it with something like the following in bash shell. (If you use csh, just temporarily run bash):

mkdir fake_short_term_archiving
cd fake_short_term_archiving
mkdir run
mkdir -p ocn/hist
mkdir -p ice/hist
cd run
forfilein<real_run_dir>/mpas*.rst*;doecho$file; ln -s $file;doneforfilein<real_run_dir>/mpas*_in;doecho$file; ln -s $file;doneforfilein<real_run_dir>/streams.*;doecho$file; ln -s $file;donecd ../ocn/hist
forfilein<real_run_dir>/mpaso.hist.timeSeriesStats*;doecho$file; ln -s $file;donecd ../../ice/hist
forfilein<real_run_dir>/mpascice.hist.timeSeriesStats*;doecho$file; ln -s $file;done

Then, you should be able to use an input section similar to what I showed above.

[milenaveneziani]

OK, I will try this later tonight. Thanks.

[milenaveneziani]

I set up a test on edison as suggested, using the beta1 run. I am testing all diagnostics, so it will take a bit to finish. For now things seem to be working. I will report back tomorrow.
Do you want to add this (and perhaps solve #149) in v0.2?

[xylar]

Do you want to add this (and perhaps solve #149) in v0.2?

Yes, I think this and #157 should be merged as part of v0.2.

[milenaveneziani]

All tests went well on edison.