Skip to content

Repository files navigation

CaloDiffusion unofficial repository (WIP) - 2.0

Implemented with Pytorch 2.0. Dependencies listed in the pyproject.yaml

Install with

git clone https://github.com/[fork]/CaloDiffusion.git
pip install -e .

Installation can be tested with

pip install pytest pytest-dependency
python3 -m pytest tests/test_execution.py

Locations of data used for testing can be set with the --data-dir option during execution, and there are additional options for setting the location of the HGCalShowers and CaloChallenge directories with the options --hgcalshowers and --calochallenge options respectively.

HGCal Tests are run using a mocked dataset using random distributions, and can be run independently using python3 -m pytest tests/test_execution.py -m "hgcal", or excluded with -m "not hgcal"

Data

Results are presented using the Fast Calorimeter Data Challenge dataset and are available for download on zenodo:

Run the training scripts with

calodif-train -d DATA-DIR \
-c CONFIG \
--checkpoint SAVE-DIR \
MODEL-TYPE
  • Example configs in [config_dataset1.json/config_dataset2.json/config_dataset3.json]
  • Additional options can be seen with calodif-train --help

Sampling with the learned model

calodif-inference
--n-events N \
-c CONFIG \
sample MODEL-TYPE
  • Additional options can be found with calodif-inference --help

Creating the plots shown in the paper

calodif-inference
--n-events N \
-c CONFIG \
plot \
--generated RESULTS-H5F

Repository Structure and Contributing

This repository is broken into 4 main parts:

1. Scripts

calodiffusion/inference.py and calodiffusion/train.py allow for CLI based inference and training, consider them the client for the rest of the repository. Functionality can be seen using --help menus.

2. Train

The base calodiffusion/train/train.py class is an abstract class that can load all the necessary functions and data for training a model. It also contains saving methods. It is a "driver" class, only providing minimal instructions on how to iterate through batches of data during training or inference. Subclasses of Train have two necessary functions - init_model and training_loop. Init model returns a specific initialized calodiffusion/model/diffusion object that will be used in training, and training_loop defines how a single batch of data is processed. calodiffusion/train/evaluation.py contains extra metrics to quantify the success of training.

3. Models

Diffusion Models

The base model class in calodiffusion is calodiffusion/models/diffusion.py. This abstract class contains methods for loading a specific sampler for inference, which loss is used, and how training or inference forward passes are performed. It can also define specific ways .pt trained weights are loaded in the case of models with different moving pieces. Diffusion is meant to have a (or several) pytorch.nn.module attributes to be used, not be a subclass of pytorch.nn.module itself.

A subclass of Diffusion has 4 required functions:

  • init_model - provide a pytorch.nn.module object to be assigned to self.model
  • forward - define how that model takes data and provides a prediction. Can be as simple as calling model.forward(). Called during training.
  • __call__ - Define how denoising is done for a specific model. Called during inference.
  • noise_generation - Generate noise for each inference step. Provides a generic "default_noise" option, but each subclass must confirm that they are using this default.

Samplers

Functions used in the denoising process to condition input for each step in the process. Additional settings for each sampler can be set using the SAMPLER_OPTIONS of the configuration. The selected sampler is using in diffusion.sample.

Loss

Loss is calculated in 2 stages - the loss metric, and then the loss calculation. Metrics can be mixed and matched with calculations. These are both set in the config.json file. The metric defines how the prediction is processed to be compared with ground truth values, and the calculation defines how they are numerically compared (using an L1 loss, MSE, etc).

4. Utils

A catch-all category for small utility functions used across training, inference, and evaluation.

About

Diffusion models for calorimeter simulation

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
GitHub - voetberg/CaloDiffusion: Diffusion models for calorimeter simulation · GitHub
Skip to content

Repository files navigation

CaloDiffusion unofficial repository (WIP) - 2.0

Implemented with Pytorch 2.0. Dependencies listed in the pyproject.yaml

Install with

git clone https://github.com/[fork]/CaloDiffusion.git
pip install -e .

Installation can be tested with

pip install pytest pytest-dependency
python3 -m pytest tests/test_execution.py

Locations of data used for testing can be set with the --data-dir option during execution, and there are additional options for setting the location of the HGCalShowers and CaloChallenge directories with the options --hgcalshowers and --calochallenge options respectively.

HGCal Tests are run using a mocked dataset using random distributions, and can be run independently using python3 -m pytest tests/test_execution.py -m "hgcal", or excluded with -m "not hgcal"

Data

Results are presented using the Fast Calorimeter Data Challenge dataset and are available for download on zenodo:

Run the training scripts with

calodif-train -d DATA-DIR \
-c CONFIG \
--checkpoint SAVE-DIR \
MODEL-TYPE
  • Example configs in [config_dataset1.json/config_dataset2.json/config_dataset3.json]
  • Additional options can be seen with calodif-train --help

Sampling with the learned model

calodif-inference
--n-events N \
-c CONFIG \
sample MODEL-TYPE
  • Additional options can be found with calodif-inference --help

Creating the plots shown in the paper

calodif-inference
--n-events N \
-c CONFIG \
plot \
--generated RESULTS-H5F

Repository Structure and Contributing

This repository is broken into 4 main parts:

1. Scripts

calodiffusion/inference.py and calodiffusion/train.py allow for CLI based inference and training, consider them the client for the rest of the repository. Functionality can be seen using --help menus.

2. Train

The base calodiffusion/train/train.py class is an abstract class that can load all the necessary functions and data for training a model. It also contains saving methods. It is a "driver" class, only providing minimal instructions on how to iterate through batches of data during training or inference. Subclasses of Train have two necessary functions - init_model and training_loop. Init model returns a specific initialized calodiffusion/model/diffusion object that will be used in training, and training_loop defines how a single batch of data is processed. calodiffusion/train/evaluation.py contains extra metrics to quantify the success of training.

3. Models

Diffusion Models

The base model class in calodiffusion is calodiffusion/models/diffusion.py. This abstract class contains methods for loading a specific sampler for inference, which loss is used, and how training or inference forward passes are performed. It can also define specific ways .pt trained weights are loaded in the case of models with different moving pieces. Diffusion is meant to have a (or several) pytorch.nn.module attributes to be used, not be a subclass of pytorch.nn.module itself.

A subclass of Diffusion has 4 required functions:

  • init_model - provide a pytorch.nn.module object to be assigned to self.model
  • forward - define how that model takes data and provides a prediction. Can be as simple as calling model.forward(). Called during training.
  • __call__ - Define how denoising is done for a specific model. Called during inference.
  • noise_generation - Generate noise for each inference step. Provides a generic "default_noise" option, but each subclass must confirm that they are using this default.

Samplers

Functions used in the denoising process to condition input for each step in the process. Additional settings for each sampler can be set using the SAMPLER_OPTIONS of the configuration. The selected sampler is using in diffusion.sample.

Loss

Loss is calculated in 2 stages - the loss metric, and then the loss calculation. Metrics can be mixed and matched with calculations. These are both set in the config.json file. The metric defines how the prediction is processed to be compared with ground truth values, and the calculation defines how they are numerically compared (using an L1 loss, MSE, etc).

4. Utils

A catch-all category for small utility functions used across training, inference, and evaluation.

About

Diffusion models for calorimeter simulation

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' GitHub - voetberg/CaloDiffusion: Diffusion models for calorimeter simulation · GitHub
Skip to content

Repository files navigation

CaloDiffusion unofficial repository (WIP) - 2.0

Implemented with Pytorch 2.0. Dependencies listed in the pyproject.yaml

Install with

git clone https://github.com/[fork]/CaloDiffusion.git
pip install -e .

Installation can be tested with

pip install pytest pytest-dependency
python3 -m pytest tests/test_execution.py

Locations of data used for testing can be set with the --data-dir option during execution, and there are additional options for setting the location of the HGCalShowers and CaloChallenge directories with the options --hgcalshowers and --calochallenge options respectively.

HGCal Tests are run using a mocked dataset using random distributions, and can be run independently using python3 -m pytest tests/test_execution.py -m "hgcal", or excluded with -m "not hgcal"

Data

Results are presented using the Fast Calorimeter Data Challenge dataset and are available for download on zenodo:

Run the training scripts with

calodif-train -d DATA-DIR \
-c CONFIG \
--checkpoint SAVE-DIR \
MODEL-TYPE
  • Example configs in [config_dataset1.json/config_dataset2.json/config_dataset3.json]
  • Additional options can be seen with calodif-train --help

Sampling with the learned model

calodif-inference
--n-events N \
-c CONFIG \
sample MODEL-TYPE
  • Additional options can be found with calodif-inference --help

Creating the plots shown in the paper

calodif-inference
--n-events N \
-c CONFIG \
plot \
--generated RESULTS-H5F

Repository Structure and Contributing

This repository is broken into 4 main parts:

1. Scripts

calodiffusion/inference.py and calodiffusion/train.py allow for CLI based inference and training, consider them the client for the rest of the repository. Functionality can be seen using --help menus.

2. Train

The base calodiffusion/train/train.py class is an abstract class that can load all the necessary functions and data for training a model. It also contains saving methods. It is a "driver" class, only providing minimal instructions on how to iterate through batches of data during training or inference. Subclasses of Train have two necessary functions - init_model and training_loop. Init model returns a specific initialized calodiffusion/model/diffusion object that will be used in training, and training_loop defines how a single batch of data is processed. calodiffusion/train/evaluation.py contains extra metrics to quantify the success of training.

3. Models

Diffusion Models

The base model class in calodiffusion is calodiffusion/models/diffusion.py. This abstract class contains methods for loading a specific sampler for inference, which loss is used, and how training or inference forward passes are performed. It can also define specific ways .pt trained weights are loaded in the case of models with different moving pieces. Diffusion is meant to have a (or several) pytorch.nn.module attributes to be used, not be a subclass of pytorch.nn.module itself.

A subclass of Diffusion has 4 required functions:

  • init_model - provide a pytorch.nn.module object to be assigned to self.model
  • forward - define how that model takes data and provides a prediction. Can be as simple as calling model.forward(). Called during training.
  • __call__ - Define how denoising is done for a specific model. Called during inference.
  • noise_generation - Generate noise for each inference step. Provides a generic "default_noise" option, but each subclass must confirm that they are using this default.

Samplers

Functions used in the denoising process to condition input for each step in the process. Additional settings for each sampler can be set using the SAMPLER_OPTIONS of the configuration. The selected sampler is using in diffusion.sample.

Loss

Loss is calculated in 2 stages - the loss metric, and then the loss calculation. Metrics can be mixed and matched with calculations. These are both set in the config.json file. The metric defines how the prediction is processed to be compared with ground truth values, and the calculation defines how they are numerically compared (using an L1 loss, MSE, etc).

4. Utils

A catch-all category for small utility functions used across training, inference, and evaluation.

About

Diffusion models for calorimeter simulation

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' GitHub - voetberg/CaloDiffusion: Diffusion models for calorimeter simulation · GitHub
Skip to content

Repository files navigation

CaloDiffusion unofficial repository (WIP) - 2.0

Implemented with Pytorch 2.0. Dependencies listed in the pyproject.yaml

Install with

git clone https://github.com/[fork]/CaloDiffusion.git
pip install -e .

Installation can be tested with

pip install pytest pytest-dependency
python3 -m pytest tests/test_execution.py

Locations of data used for testing can be set with the --data-dir option during execution, and there are additional options for setting the location of the HGCalShowers and CaloChallenge directories with the options --hgcalshowers and --calochallenge options respectively.

HGCal Tests are run using a mocked dataset using random distributions, and can be run independently using python3 -m pytest tests/test_execution.py -m "hgcal", or excluded with -m "not hgcal"

Data

Results are presented using the Fast Calorimeter Data Challenge dataset and are available for download on zenodo:

Run the training scripts with

calodif-train -d DATA-DIR \
-c CONFIG \
--checkpoint SAVE-DIR \
MODEL-TYPE
  • Example configs in [config_dataset1.json/config_dataset2.json/config_dataset3.json]
  • Additional options can be seen with calodif-train --help

Sampling with the learned model

calodif-inference
--n-events N \
-c CONFIG \
sample MODEL-TYPE
  • Additional options can be found with calodif-inference --help

Creating the plots shown in the paper

calodif-inference
--n-events N \
-c CONFIG \
plot \
--generated RESULTS-H5F

Repository Structure and Contributing

This repository is broken into 4 main parts:

1. Scripts

calodiffusion/inference.py and calodiffusion/train.py allow for CLI based inference and training, consider them the client for the rest of the repository. Functionality can be seen using --help menus.

2. Train

The base calodiffusion/train/train.py class is an abstract class that can load all the necessary functions and data for training a model. It also contains saving methods. It is a "driver" class, only providing minimal instructions on how to iterate through batches of data during training or inference. Subclasses of Train have two necessary functions - init_model and training_loop. Init model returns a specific initialized calodiffusion/model/diffusion object that will be used in training, and training_loop defines how a single batch of data is processed. calodiffusion/train/evaluation.py contains extra metrics to quantify the success of training.

3. Models

Diffusion Models

The base model class in calodiffusion is calodiffusion/models/diffusion.py. This abstract class contains methods for loading a specific sampler for inference, which loss is used, and how training or inference forward passes are performed. It can also define specific ways .pt trained weights are loaded in the case of models with different moving pieces. Diffusion is meant to have a (or several) pytorch.nn.module attributes to be used, not be a subclass of pytorch.nn.module itself.

A subclass of Diffusion has 4 required functions:

  • init_model - provide a pytorch.nn.module object to be assigned to self.model
  • forward - define how that model takes data and provides a prediction. Can be as simple as calling model.forward(). Called during training.
  • __call__ - Define how denoising is done for a specific model. Called during inference.
  • noise_generation - Generate noise for each inference step. Provides a generic "default_noise" option, but each subclass must confirm that they are using this default.

Samplers

Functions used in the denoising process to condition input for each step in the process. Additional settings for each sampler can be set using the SAMPLER_OPTIONS of the configuration. The selected sampler is using in diffusion.sample.

Loss

Loss is calculated in 2 stages - the loss metric, and then the loss calculation. Metrics can be mixed and matched with calculations. These are both set in the config.json file. The metric defines how the prediction is processed to be compared with ground truth values, and the calculation defines how they are numerically compared (using an L1 loss, MSE, etc).

4. Utils

A catch-all category for small utility functions used across training, inference, and evaluation.

About

Diffusion models for calorimeter simulation

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' GitHub - voetberg/CaloDiffusion: Diffusion models for calorimeter simulation · GitHub
Skip to content

Repository files navigation

CaloDiffusion unofficial repository (WIP) - 2.0

Implemented with Pytorch 2.0. Dependencies listed in the pyproject.yaml

Install with

git clone https://github.com/[fork]/CaloDiffusion.git
pip install -e .

Installation can be tested with

pip install pytest pytest-dependency
python3 -m pytest tests/test_execution.py

Locations of data used for testing can be set with the --data-dir option during execution, and there are additional options for setting the location of the HGCalShowers and CaloChallenge directories with the options --hgcalshowers and --calochallenge options respectively.

HGCal Tests are run using a mocked dataset using random distributions, and can be run independently using python3 -m pytest tests/test_execution.py -m "hgcal", or excluded with -m "not hgcal"

Data

Results are presented using the Fast Calorimeter Data Challenge dataset and are available for download on zenodo:

Run the training scripts with

calodif-train -d DATA-DIR \
-c CONFIG \
--checkpoint SAVE-DIR \
MODEL-TYPE
  • Example configs in [config_dataset1.json/config_dataset2.json/config_dataset3.json]
  • Additional options can be seen with calodif-train --help

Sampling with the learned model

calodif-inference
--n-events N \
-c CONFIG \
sample MODEL-TYPE
  • Additional options can be found with calodif-inference --help

Creating the plots shown in the paper

calodif-inference
--n-events N \
-c CONFIG \
plot \
--generated RESULTS-H5F

Repository Structure and Contributing

This repository is broken into 4 main parts:

1. Scripts

calodiffusion/inference.py and calodiffusion/train.py allow for CLI based inference and training, consider them the client for the rest of the repository. Functionality can be seen using --help menus.

2. Train

The base calodiffusion/train/train.py class is an abstract class that can load all the necessary functions and data for training a model. It also contains saving methods. It is a "driver" class, only providing minimal instructions on how to iterate through batches of data during training or inference. Subclasses of Train have two necessary functions - init_model and training_loop. Init model returns a specific initialized calodiffusion/model/diffusion object that will be used in training, and training_loop defines how a single batch of data is processed. calodiffusion/train/evaluation.py contains extra metrics to quantify the success of training.

3. Models

Diffusion Models

The base model class in calodiffusion is calodiffusion/models/diffusion.py. This abstract class contains methods for loading a specific sampler for inference, which loss is used, and how training or inference forward passes are performed. It can also define specific ways .pt trained weights are loaded in the case of models with different moving pieces. Diffusion is meant to have a (or several) pytorch.nn.module attributes to be used, not be a subclass of pytorch.nn.module itself.

A subclass of Diffusion has 4 required functions:

  • init_model - provide a pytorch.nn.module object to be assigned to self.model
  • forward - define how that model takes data and provides a prediction. Can be as simple as calling model.forward(). Called during training.
  • __call__ - Define how denoising is done for a specific model. Called during inference.
  • noise_generation - Generate noise for each inference step. Provides a generic "default_noise" option, but each subclass must confirm that they are using this default.

Samplers

Functions used in the denoising process to condition input for each step in the process. Additional settings for each sampler can be set using the SAMPLER_OPTIONS of the configuration. The selected sampler is using in diffusion.sample.

Loss

Loss is calculated in 2 stages - the loss metric, and then the loss calculation. Metrics can be mixed and matched with calculations. These are both set in the config.json file. The metric defines how the prediction is processed to be compared with ground truth values, and the calculation defines how they are numerically compared (using an L1 loss, MSE, etc).

4. Utils

A catch-all category for small utility functions used across training, inference, and evaluation.

About

Diffusion models for calorimeter simulation

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' GitHub - voetberg/CaloDiffusion: Diffusion models for calorimeter simulation · GitHub
Skip to content

Repository files navigation

CaloDiffusion unofficial repository (WIP) - 2.0

Implemented with Pytorch 2.0. Dependencies listed in the pyproject.yaml

Install with

git clone https://github.com/[fork]/CaloDiffusion.git
pip install -e .

Installation can be tested with

pip install pytest pytest-dependency
python3 -m pytest tests/test_execution.py

Locations of data used for testing can be set with the --data-dir option during execution, and there are additional options for setting the location of the HGCalShowers and CaloChallenge directories with the options --hgcalshowers and --calochallenge options respectively.

HGCal Tests are run using a mocked dataset using random distributions, and can be run independently using python3 -m pytest tests/test_execution.py -m "hgcal", or excluded with -m "not hgcal"

Data

Results are presented using the Fast Calorimeter Data Challenge dataset and are available for download on zenodo:

Run the training scripts with

calodif-train -d DATA-DIR \
-c CONFIG \
--checkpoint SAVE-DIR \
MODEL-TYPE
  • Example configs in [config_dataset1.json/config_dataset2.json/config_dataset3.json]
  • Additional options can be seen with calodif-train --help

Sampling with the learned model

calodif-inference
--n-events N \
-c CONFIG \
sample MODEL-TYPE
  • Additional options can be found with calodif-inference --help

Creating the plots shown in the paper

calodif-inference
--n-events N \
-c CONFIG \
plot \
--generated RESULTS-H5F

Repository Structure and Contributing

This repository is broken into 4 main parts:

1. Scripts

calodiffusion/inference.py and calodiffusion/train.py allow for CLI based inference and training, consider them the client for the rest of the repository. Functionality can be seen using --help menus.

2. Train

The base calodiffusion/train/train.py class is an abstract class that can load all the necessary functions and data for training a model. It also contains saving methods. It is a "driver" class, only providing minimal instructions on how to iterate through batches of data during training or inference. Subclasses of Train have two necessary functions - init_model and training_loop. Init model returns a specific initialized calodiffusion/model/diffusion object that will be used in training, and training_loop defines how a single batch of data is processed. calodiffusion/train/evaluation.py contains extra metrics to quantify the success of training.

3. Models

Diffusion Models

The base model class in calodiffusion is calodiffusion/models/diffusion.py. This abstract class contains methods for loading a specific sampler for inference, which loss is used, and how training or inference forward passes are performed. It can also define specific ways .pt trained weights are loaded in the case of models with different moving pieces. Diffusion is meant to have a (or several) pytorch.nn.module attributes to be used, not be a subclass of pytorch.nn.module itself.

A subclass of Diffusion has 4 required functions:

  • init_model - provide a pytorch.nn.module object to be assigned to self.model
  • forward - define how that model takes data and provides a prediction. Can be as simple as calling model.forward(). Called during training.
  • __call__ - Define how denoising is done for a specific model. Called during inference.
  • noise_generation - Generate noise for each inference step. Provides a generic "default_noise" option, but each subclass must confirm that they are using this default.

Samplers

Functions used in the denoising process to condition input for each step in the process. Additional settings for each sampler can be set using the SAMPLER_OPTIONS of the configuration. The selected sampler is using in diffusion.sample.

Loss

Loss is calculated in 2 stages - the loss metric, and then the loss calculation. Metrics can be mixed and matched with calculations. These are both set in the config.json file. The metric defines how the prediction is processed to be compared with ground truth values, and the calculation defines how they are numerically compared (using an L1 loss, MSE, etc).

4. Utils

A catch-all category for small utility functions used across training, inference, and evaluation.

About

Diffusion models for calorimeter simulation

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' GitHub - voetberg/CaloDiffusion: Diffusion models for calorimeter simulation · GitHub
Skip to content

Repository files navigation

CaloDiffusion unofficial repository (WIP) - 2.0

Implemented with Pytorch 2.0. Dependencies listed in the pyproject.yaml

Install with

git clone https://github.com/[fork]/CaloDiffusion.git
pip install -e .

Installation can be tested with

pip install pytest pytest-dependency
python3 -m pytest tests/test_execution.py

Locations of data used for testing can be set with the --data-dir option during execution, and there are additional options for setting the location of the HGCalShowers and CaloChallenge directories with the options --hgcalshowers and --calochallenge options respectively.

HGCal Tests are run using a mocked dataset using random distributions, and can be run independently using python3 -m pytest tests/test_execution.py -m "hgcal", or excluded with -m "not hgcal"

Data

Results are presented using the Fast Calorimeter Data Challenge dataset and are available for download on zenodo:

Run the training scripts with

calodif-train -d DATA-DIR \
-c CONFIG \
--checkpoint SAVE-DIR \
MODEL-TYPE
  • Example configs in [config_dataset1.json/config_dataset2.json/config_dataset3.json]
  • Additional options can be seen with calodif-train --help

Sampling with the learned model

calodif-inference
--n-events N \
-c CONFIG \
sample MODEL-TYPE
  • Additional options can be found with calodif-inference --help

Creating the plots shown in the paper

calodif-inference
--n-events N \
-c CONFIG \
plot \
--generated RESULTS-H5F

Repository Structure and Contributing

This repository is broken into 4 main parts:

1. Scripts

calodiffusion/inference.py and calodiffusion/train.py allow for CLI based inference and training, consider them the client for the rest of the repository. Functionality can be seen using --help menus.

2. Train

The base calodiffusion/train/train.py class is an abstract class that can load all the necessary functions and data for training a model. It also contains saving methods. It is a "driver" class, only providing minimal instructions on how to iterate through batches of data during training or inference. Subclasses of Train have two necessary functions - init_model and training_loop. Init model returns a specific initialized calodiffusion/model/diffusion object that will be used in training, and training_loop defines how a single batch of data is processed. calodiffusion/train/evaluation.py contains extra metrics to quantify the success of training.

3. Models

Diffusion Models

The base model class in calodiffusion is calodiffusion/models/diffusion.py. This abstract class contains methods for loading a specific sampler for inference, which loss is used, and how training or inference forward passes are performed. It can also define specific ways .pt trained weights are loaded in the case of models with different moving pieces. Diffusion is meant to have a (or several) pytorch.nn.module attributes to be used, not be a subclass of pytorch.nn.module itself.

A subclass of Diffusion has 4 required functions:

  • init_model - provide a pytorch.nn.module object to be assigned to self.model
  • forward - define how that model takes data and provides a prediction. Can be as simple as calling model.forward(). Called during training.
  • __call__ - Define how denoising is done for a specific model. Called during inference.
  • noise_generation - Generate noise for each inference step. Provides a generic "default_noise" option, but each subclass must confirm that they are using this default.

Samplers

Functions used in the denoising process to condition input for each step in the process. Additional settings for each sampler can be set using the SAMPLER_OPTIONS of the configuration. The selected sampler is using in diffusion.sample.

Loss

Loss is calculated in 2 stages - the loss metric, and then the loss calculation. Metrics can be mixed and matched with calculations. These are both set in the config.json file. The metric defines how the prediction is processed to be compared with ground truth values, and the calculation defines how they are numerically compared (using an L1 loss, MSE, etc).

4. Utils

A catch-all category for small utility functions used across training, inference, and evaluation.

About

Diffusion models for calorimeter simulation

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); GitHub - voetberg/CaloDiffusion: Diffusion models for calorimeter simulation · GitHub
Skip to content

Repository files navigation

CaloDiffusion unofficial repository (WIP) - 2.0

Implemented with Pytorch 2.0. Dependencies listed in the pyproject.yaml

Install with

git clone https://github.com/[fork]/CaloDiffusion.git
pip install -e .

Installation can be tested with

pip install pytest pytest-dependency
python3 -m pytest tests/test_execution.py

Locations of data used for testing can be set with the --data-dir option during execution, and there are additional options for setting the location of the HGCalShowers and CaloChallenge directories with the options --hgcalshowers and --calochallenge options respectively.

HGCal Tests are run using a mocked dataset using random distributions, and can be run independently using python3 -m pytest tests/test_execution.py -m "hgcal", or excluded with -m "not hgcal"

Data

Results are presented using the Fast Calorimeter Data Challenge dataset and are available for download on zenodo:

Run the training scripts with

calodif-train -d DATA-DIR \
-c CONFIG \
--checkpoint SAVE-DIR \
MODEL-TYPE
  • Example configs in [config_dataset1.json/config_dataset2.json/config_dataset3.json]
  • Additional options can be seen with calodif-train --help

Sampling with the learned model

calodif-inference
--n-events N \
-c CONFIG \
sample MODEL-TYPE
  • Additional options can be found with calodif-inference --help

Creating the plots shown in the paper

calodif-inference
--n-events N \
-c CONFIG \
plot \
--generated RESULTS-H5F

Repository Structure and Contributing

This repository is broken into 4 main parts:

1. Scripts

calodiffusion/inference.py and calodiffusion/train.py allow for CLI based inference and training, consider them the client for the rest of the repository. Functionality can be seen using --help menus.

2. Train

The base calodiffusion/train/train.py class is an abstract class that can load all the necessary functions and data for training a model. It also contains saving methods. It is a "driver" class, only providing minimal instructions on how to iterate through batches of data during training or inference. Subclasses of Train have two necessary functions - init_model and training_loop. Init model returns a specific initialized calodiffusion/model/diffusion object that will be used in training, and training_loop defines how a single batch of data is processed. calodiffusion/train/evaluation.py contains extra metrics to quantify the success of training.

3. Models

Diffusion Models

The base model class in calodiffusion is calodiffusion/models/diffusion.py. This abstract class contains methods for loading a specific sampler for inference, which loss is used, and how training or inference forward passes are performed. It can also define specific ways .pt trained weights are loaded in the case of models with different moving pieces. Diffusion is meant to have a (or several) pytorch.nn.module attributes to be used, not be a subclass of pytorch.nn.module itself.

A subclass of Diffusion has 4 required functions:

  • init_model - provide a pytorch.nn.module object to be assigned to self.model
  • forward - define how that model takes data and provides a prediction. Can be as simple as calling model.forward(). Called during training.
  • __call__ - Define how denoising is done for a specific model. Called during inference.
  • noise_generation - Generate noise for each inference step. Provides a generic "default_noise" option, but each subclass must confirm that they are using this default.

Samplers

Functions used in the denoising process to condition input for each step in the process. Additional settings for each sampler can be set using the SAMPLER_OPTIONS of the configuration. The selected sampler is using in diffusion.sample.

Loss

Loss is calculated in 2 stages - the loss metric, and then the loss calculation. Metrics can be mixed and matched with calculations. These are both set in the config.json file. The metric defines how the prediction is processed to be compared with ground truth values, and the calculation defines how they are numerically compared (using an L1 loss, MSE, etc).

4. Utils

A catch-all category for small utility functions used across training, inference, and evaluation.

About

Diffusion models for calorimeter simulation

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages