MaCh3-PythonUtils

Very simple tool for analysing MCMC. Currently only accepts chains where all variables are saved within a ROOT TTree.

Setup

Setup is relatively simple. The recommended way of running this is using a virtual environment

virtualenv .env

Then to setup,

source .env/bin/activate

The module can then be installed through pip with

pip install .

Running

Running the package is also simple. pip adds mach3_python_utils as an executable so you simply need to run

mach3_python_utils -c /path/to/config.yml

This command can be accessed anywhere

Some example configs can be found in the configs folder.

Configs

Configs are in YAML format

For all packages the initial setup is very similar:

FileSettings:
FileName : "/path/to/file"# Name of FileChainName : "/tree/name"# Name of Tree in file containing chain# Do you want it to be verbose?Verbose: False# What plots do we want?MakeMLModel: True # ML stuffMakeDiagnostics: True # Make MCMC diagnostic plotsMakePosteriors: True # Make posteriors # Settings for parameter optionsParameterSettings:
# Names of parameters to fit, finds all parameters containing names in this string as sub-stringParameterNames : ["sin2th", "sin2th","delm2_12", "delta", "xsec"]# Name of variable you're fitting inLabelName : "LogL"# Parameters you don't want to include in the modelIgnoredParameters : ["LogL_systematic_xsec_cov", "Log", "LogL_systematic_nddet_cov", ]# Any cuts, for MaCh3 I'd recommend capping LogLParameterCuts : ["LogL<12345678", "step>10000"]CircularParameters: [] # Do any parameters loop?

Plotting Settings

This package contains various plotting tools required for analysing markov chains which are stored in the Plotting library!

PlottingSettings:
DiagnosticsSettings:
# Where are we prioting?DiagnosticsOutputFile: "diagnostics_output.pdf"# Make Trace/AC Plot?MakeTraceAC: True# Make Violin Plot?MakeViolin: False#Make ESS Plot?MakeESS: False# Make MCSE PlotMakeMCSE: False# Make suboptimality PlotMakeSuboptimality: False# Steps/calculation for subopt.SuboptimalitySteps: 10000# Print summary statsPrintSummary: TruePosteriorSettings:
# Output PDFPosteriorOutputFile: "posteriors.pdf"# Do you want 1D CIs?Make1DPosteriors: True# Plotted CIsCredibleIntervals: [0.6, 0.90, 0.95]# Do you want 2D CIs?Make2DPosteriors: False# Do you want a triangle plot?MakeTrianglePlot: False# variables to put in the triangle

ML Settings

For scikit learn based packages the settings are then set in the following way (where FitterKwargs directly sets the keyword arguments for the scikit fitting tool being used):

MLSettings:
# Package model is included inFitterPackage : "SciKit"# Fitter NameFitterName : "HistBoost"# Size of test set (range is 0-1)TestSize : 0.8# Set fitter Hyper Parameters, these are found in the fitter's readmeFitterKwargs:
# n_jobs = 32verbose: Truemax_iter: 10000# n_estimators = 200

For TensorFlow based packages the settings are more complex

FitterSettings:
FitterPackage : "TensorFlow"FitterName : "Sequential"TestSize : 0.9FitterKwargs:
BuildSettings:
optimizer: 'adam'loss: 'mse'FitSettings:
epochs: 20batch_size: 20Layers:
- dense:
units: 128
- dense:
units: 64
- dropout:
rate: 0.5
- dense: units: 16
- dense:
units: 1

Here FitterKwargs is now split into sub-settings with BuildSettings being passed to the model compile method, FitSettings setting up training information, and Layers which defines the types + kwargs of each layer in the model. New layers can be implemented in the __TF_LAYER_IMPLEMENTATIONS object which lives in machine_learning/tf_interface

Implementing a New Fitter

Implementing a new fitter is relatively simple. Mostly this is done in machine_learining/ml_factory/MLFactory. For Scikit-Learn based models, the new method just needs to imported and added to the scikit entry in __IMPLEMENTED_ALGORITHMS.

For non-scikit/tf based algorithms currently no implementation exists. For such cases a new interface class (which inherits from FMLInterface) needs to be implemented. Hopefully in future this is easy to do!

TO DO LIST

• Better diagnostic plotting (particularly for the NNs)
• Smart hyper parameter tuning (Just a random grid search will do!)
• Better sampling methods that "MCMC output go brrrrrr"