Skip to content
BranchesTags

Latest commit

History

614 Commits

Folders and files

NameName
Last commit message
Last commit date
.gitignore
.gitignore
Activate_globus.sh
Activate_globus.sh
Create_CSVs.sh
Create_CSVs.sh
Create_readset_transfer_BAMS.sh
Create_readset_transfer_BAMS.sh
DB_OPS.py
DB_OPS.py
GenPipes_wrapper.sh
GenPipes_wrapper.sh
Generate_pairs_readset.py
Generate_pairs_readset.py
Generate_pairs_readset_topup.py
Generate_pairs_readset_topup.py
MOH_Check_Progress.py
MOH_Check_Progress.py
MOH_ln_output.py
MOH_ln_output.py
Metrics_Update.py
Metrics_Update.py
ProjectTracking_RunProcessing.sh
ProjectTracking_RunProcessing.sh
README.md
README.md
add_mosdepth.py
add_mosdepth.py
bucket_delivery.py
bucket_delivery.py
bucket_delivery.sh
bucket_delivery.sh
check_GenPipes.sh
check_GenPipes.sh
create_readset_all_samples.py
create_readset_all_samples.py
csv2json.py
csv2json.py
fix_bam_paths.sh
fix_bam_paths.sh
fms_ProjectTracking_RunProcessing.sh
fms_ProjectTracking_RunProcessing.sh
fms_run_processing2json.py
fms_run_processing2json.py
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing_topup_same_name.sh
fms_transfer_RunProcessing_topup_same_name.sh
genpipes_deliverables_metrics_tagging.py
genpipes_deliverables_metrics_tagging.py
globus_acl.sh
globus_acl.sh
globus_collections.json
globus_collections.json
ingest_sample_to_old_db.py
ingest_sample_to_old_db.py
jenkins_wrapper.sh
jenkins_wrapper.sh
jsonify_db.py
jsonify_db.py
merge_GenPipes_jsons.py
merge_GenPipes_jsons.py
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_run_processing2json.py
mgc_mock_run_processing2json.py
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
parse_check_GenPipes_log.sh
parse_check_GenPipes_log.sh
pt_check_sample.sh
pt_check_sample.sh
rename_sample_in_bams.sh
rename_sample_in_bams.sh
run_processing2json.py
run_processing2json.py
transfer2json.py
transfer2json.py
transfer_GenPipes.sh
transfer_GenPipes.sh
transfer_RunProcessing_topup_same_name.sh
transfer_RunProcessing_topup_same_name.sh

Repository files navigation

Marathon of Hope Automation

This repository contains all of the scripts for the automation of the MOH projects and integration of genpipes

Abacus Scripts

Activate_globus.sh

Activates the globus endpoints for Create_readset_transfer_BAMS.sh
Must be updated for individual users
Usage: Activate_globus.sh

Create_readset_transfer_BAMS.sh

Takes the full path of the read folder as an input and transfers the BAMs/fastqs with the MoH prefix.In addition it tranfers over the key run processing metrics and generates a log file forlater usage with the database
Globus fields must be updated for individual users
Usage: Create_readset_transfer_BAMS.sh PATH_TO_RUN_FOLDER

Beluga Scripts

Metrics_Update.py

This script takes a single file and parses it to update the database with metrics from run processing and from both DNA and RNA Genpipes. It compares the extracted values to known acceptable values and adds them to the KEY_METRICS table within the MoH database.
Usage: Metrics_Update.py

MOH_Check_Progress.py

Parses the file structure of MAIN and looks for the output files produced from samples in the Sample table. It then updates STATUS table with any progress. It queries for all files, so any deliverables that are removed will result in an incomplete listing. In addition this script populates/updates the Timestamps and File_Locations tables.
Usage: MOH_Check_Progress.py

DB_OPS.py

This contains all of the functions necessary for interacting with a file. Required for other scripts.

MOH_ln_output.py

Takes a single sample input and hard links all of the deliverable files within a structured directory with their final names. In addition it constructs the custom readme and log files. All data is taken from the database.
Usage: MOH_ln_output.py Sample
Currently has a coverage cutoff implemented

Create_CSVs.sh

Dumps the tables within the database as csv's within the CSV folder of DATABASE.
Usage: Create_CSVs.sh

Generate_pairs_readset.py

Searches the temporary raw_reads folder for matching DNA pairs or RNA. It moves the fastqs/BAMs and creates the readset and pairs files in preparation for Setup_run_XXX.sh. In addition, it populates the Samples table. It will not move process any files that are outside the naming convention and it will exit if it finds files with the same name at the destination.
Usage Generate_pairs_readset.py DNA or Generate_pairs_readset.py RNA

About

Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation.

Resources

Readme
Activity
Custom properties

Stars

1 star

Watchers

1 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
GitHub - c3g/MoH_Automation: Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation. · GitHub
Skip to content
BranchesTags
Latest commit
History
614 Commits
Folders and files
NameName
Last commit message
Last commit date
.gitignore
.gitignore
Activate_globus.sh
Activate_globus.sh
Create_CSVs.sh
Create_CSVs.sh
Create_readset_transfer_BAMS.sh
Create_readset_transfer_BAMS.sh
DB_OPS.py
DB_OPS.py
GenPipes_wrapper.sh
GenPipes_wrapper.sh
Generate_pairs_readset.py
Generate_pairs_readset.py
Generate_pairs_readset_topup.py
Generate_pairs_readset_topup.py
MOH_Check_Progress.py
MOH_Check_Progress.py
MOH_ln_output.py
MOH_ln_output.py
Metrics_Update.py
Metrics_Update.py
ProjectTracking_RunProcessing.sh
ProjectTracking_RunProcessing.sh
README.md
README.md
add_mosdepth.py
add_mosdepth.py
bucket_delivery.py
bucket_delivery.py
bucket_delivery.sh
bucket_delivery.sh
check_GenPipes.sh
check_GenPipes.sh
create_readset_all_samples.py
create_readset_all_samples.py
csv2json.py
csv2json.py
fix_bam_paths.sh
fix_bam_paths.sh
fms_ProjectTracking_RunProcessing.sh
fms_ProjectTracking_RunProcessing.sh
fms_run_processing2json.py
fms_run_processing2json.py
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing_topup_same_name.sh
fms_transfer_RunProcessing_topup_same_name.sh
genpipes_deliverables_metrics_tagging.py
genpipes_deliverables_metrics_tagging.py
globus_acl.sh
globus_acl.sh
globus_collections.json
globus_collections.json
ingest_sample_to_old_db.py
ingest_sample_to_old_db.py
jenkins_wrapper.sh
jenkins_wrapper.sh
jsonify_db.py
jsonify_db.py
merge_GenPipes_jsons.py
merge_GenPipes_jsons.py
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_run_processing2json.py
mgc_mock_run_processing2json.py
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
parse_check_GenPipes_log.sh
parse_check_GenPipes_log.sh
pt_check_sample.sh
pt_check_sample.sh
rename_sample_in_bams.sh
rename_sample_in_bams.sh
run_processing2json.py
run_processing2json.py
transfer2json.py
transfer2json.py
transfer_GenPipes.sh
transfer_GenPipes.sh
transfer_RunProcessing_topup_same_name.sh
transfer_RunProcessing_topup_same_name.sh
Repository files navigation
Marathon of Hope Automation
This repository contains all of the scripts for the automation of the MOH projects
and integration of genpipes
Abacus Scripts
Activate_globus.sh
Activates the globus endpoints for Create_readset_transfer_BAMS.sh
Must be updated for individual users

Usage: Activate_globus.sh
Create_readset_transfer_BAMS.sh
Takes the full path of the read folder as an input and transfers the BAMs/fastqs with the
MoH prefix.In addition it tranfers over the key run processing metrics and generates
a log file forlater usage with the database
Globus fields must be updated for individual users

Usage: Create_readset_transfer_BAMS.sh PATH_TO_RUN_FOLDER
Beluga Scripts
Metrics_Update.py
This script takes a single file and parses it to update the database with metrics from
run processing and from both DNA and RNA Genpipes. It compares the extracted values to
known acceptable values and adds them to the KEY_METRICS table within the MoH database.

Usage: Metrics_Update.py
MOH_Check_Progress.py
Parses the file structure of MAIN and looks for the output files produced from samples in
the Sample table. It then updates STATUS table with any progress. It queries for all files,
so any deliverables that are removed will result in an incomplete listing. In addition this
script populates/updates the Timestamps and File_Locations tables.

Usage: MOH_Check_Progress.py
DB_OPS.py
This contains all of the functions necessary for interacting with a file. Required for
other scripts.
MOH_ln_output.py
Takes a single sample input and hard links all of the deliverable files within a structured
directory with their final names. In addition it constructs the custom readme and log files.
All data is taken from the database.

Usage: MOH_ln_output.py Sample
Currently has a coverage cutoff implemented
Create_CSVs.sh
Dumps the tables within the database as csv's within the CSV folder of DATABASE.

Usage: Create_CSVs.sh
Generate_pairs_readset.py
Searches the temporary raw_reads folder for matching DNA pairs or RNA. It moves the
fastqs/BAMs and creates the readset and pairs files in preparation for Setup_run_XXX.sh.
In addition, it populates the Samples table. It will not move process any files that are
outside the naming convention and it will exit if it finds files with the same name at the
destination.

Usage Generate_pairs_readset.py DNA or Generate_pairs_readset.py RNA
About
Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation.
Resources
Readme
Activity
Custom properties
Stars
1 star
Watchers
1 watching
Forks
0 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - c3g/MoH_Automation: Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation. · GitHub
Skip to content
BranchesTags
Latest commit
History
614 Commits
Folders and files
NameName
Last commit message
Last commit date
.gitignore
.gitignore
Activate_globus.sh
Activate_globus.sh
Create_CSVs.sh
Create_CSVs.sh
Create_readset_transfer_BAMS.sh
Create_readset_transfer_BAMS.sh
DB_OPS.py
DB_OPS.py
GenPipes_wrapper.sh
GenPipes_wrapper.sh
Generate_pairs_readset.py
Generate_pairs_readset.py
Generate_pairs_readset_topup.py
Generate_pairs_readset_topup.py
MOH_Check_Progress.py
MOH_Check_Progress.py
MOH_ln_output.py
MOH_ln_output.py
Metrics_Update.py
Metrics_Update.py
ProjectTracking_RunProcessing.sh
ProjectTracking_RunProcessing.sh
README.md
README.md
add_mosdepth.py
add_mosdepth.py
bucket_delivery.py
bucket_delivery.py
bucket_delivery.sh
bucket_delivery.sh
check_GenPipes.sh
check_GenPipes.sh
create_readset_all_samples.py
create_readset_all_samples.py
csv2json.py
csv2json.py
fix_bam_paths.sh
fix_bam_paths.sh
fms_ProjectTracking_RunProcessing.sh
fms_ProjectTracking_RunProcessing.sh
fms_run_processing2json.py
fms_run_processing2json.py
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing_topup_same_name.sh
fms_transfer_RunProcessing_topup_same_name.sh
genpipes_deliverables_metrics_tagging.py
genpipes_deliverables_metrics_tagging.py
globus_acl.sh
globus_acl.sh
globus_collections.json
globus_collections.json
ingest_sample_to_old_db.py
ingest_sample_to_old_db.py
jenkins_wrapper.sh
jenkins_wrapper.sh
jsonify_db.py
jsonify_db.py
merge_GenPipes_jsons.py
merge_GenPipes_jsons.py
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_run_processing2json.py
mgc_mock_run_processing2json.py
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
parse_check_GenPipes_log.sh
parse_check_GenPipes_log.sh
pt_check_sample.sh
pt_check_sample.sh
rename_sample_in_bams.sh
rename_sample_in_bams.sh
run_processing2json.py
run_processing2json.py
transfer2json.py
transfer2json.py
transfer_GenPipes.sh
transfer_GenPipes.sh
transfer_RunProcessing_topup_same_name.sh
transfer_RunProcessing_topup_same_name.sh
Repository files navigation
Marathon of Hope Automation
This repository contains all of the scripts for the automation of the MOH projects
and integration of genpipes
Abacus Scripts
Activate_globus.sh
Activates the globus endpoints for Create_readset_transfer_BAMS.sh
Must be updated for individual users

Usage: Activate_globus.sh
Create_readset_transfer_BAMS.sh
Takes the full path of the read folder as an input and transfers the BAMs/fastqs with the
MoH prefix.In addition it tranfers over the key run processing metrics and generates
a log file forlater usage with the database
Globus fields must be updated for individual users

Usage: Create_readset_transfer_BAMS.sh PATH_TO_RUN_FOLDER
Beluga Scripts
Metrics_Update.py
This script takes a single file and parses it to update the database with metrics from
run processing and from both DNA and RNA Genpipes. It compares the extracted values to
known acceptable values and adds them to the KEY_METRICS table within the MoH database.

Usage: Metrics_Update.py
MOH_Check_Progress.py
Parses the file structure of MAIN and looks for the output files produced from samples in
the Sample table. It then updates STATUS table with any progress. It queries for all files,
so any deliverables that are removed will result in an incomplete listing. In addition this
script populates/updates the Timestamps and File_Locations tables.

Usage: MOH_Check_Progress.py
DB_OPS.py
This contains all of the functions necessary for interacting with a file. Required for
other scripts.
MOH_ln_output.py
Takes a single sample input and hard links all of the deliverable files within a structured
directory with their final names. In addition it constructs the custom readme and log files.
All data is taken from the database.

Usage: MOH_ln_output.py Sample
Currently has a coverage cutoff implemented
Create_CSVs.sh
Dumps the tables within the database as csv's within the CSV folder of DATABASE.

Usage: Create_CSVs.sh
Generate_pairs_readset.py
Searches the temporary raw_reads folder for matching DNA pairs or RNA. It moves the
fastqs/BAMs and creates the readset and pairs files in preparation for Setup_run_XXX.sh.
In addition, it populates the Samples table. It will not move process any files that are
outside the naming convention and it will exit if it finds files with the same name at the
destination.

Usage Generate_pairs_readset.py DNA or Generate_pairs_readset.py RNA
About
Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation.
Resources
Readme
Activity
Custom properties
Stars
1 star
Watchers
1 watching
Forks
0 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - c3g/MoH_Automation: Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation. · GitHub
Skip to content
BranchesTags
Latest commit
History
614 Commits
Folders and files
NameName
Last commit message
Last commit date
.gitignore
.gitignore
Activate_globus.sh
Activate_globus.sh
Create_CSVs.sh
Create_CSVs.sh
Create_readset_transfer_BAMS.sh
Create_readset_transfer_BAMS.sh
DB_OPS.py
DB_OPS.py
GenPipes_wrapper.sh
GenPipes_wrapper.sh
Generate_pairs_readset.py
Generate_pairs_readset.py
Generate_pairs_readset_topup.py
Generate_pairs_readset_topup.py
MOH_Check_Progress.py
MOH_Check_Progress.py
MOH_ln_output.py
MOH_ln_output.py
Metrics_Update.py
Metrics_Update.py
ProjectTracking_RunProcessing.sh
ProjectTracking_RunProcessing.sh
README.md
README.md
add_mosdepth.py
add_mosdepth.py
bucket_delivery.py
bucket_delivery.py
bucket_delivery.sh
bucket_delivery.sh
check_GenPipes.sh
check_GenPipes.sh
create_readset_all_samples.py
create_readset_all_samples.py
csv2json.py
csv2json.py
fix_bam_paths.sh
fix_bam_paths.sh
fms_ProjectTracking_RunProcessing.sh
fms_ProjectTracking_RunProcessing.sh
fms_run_processing2json.py
fms_run_processing2json.py
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing_topup_same_name.sh
fms_transfer_RunProcessing_topup_same_name.sh
genpipes_deliverables_metrics_tagging.py
genpipes_deliverables_metrics_tagging.py
globus_acl.sh
globus_acl.sh
globus_collections.json
globus_collections.json
ingest_sample_to_old_db.py
ingest_sample_to_old_db.py
jenkins_wrapper.sh
jenkins_wrapper.sh
jsonify_db.py
jsonify_db.py
merge_GenPipes_jsons.py
merge_GenPipes_jsons.py
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_run_processing2json.py
mgc_mock_run_processing2json.py
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
parse_check_GenPipes_log.sh
parse_check_GenPipes_log.sh
pt_check_sample.sh
pt_check_sample.sh
rename_sample_in_bams.sh
rename_sample_in_bams.sh
run_processing2json.py
run_processing2json.py
transfer2json.py
transfer2json.py
transfer_GenPipes.sh
transfer_GenPipes.sh
transfer_RunProcessing_topup_same_name.sh
transfer_RunProcessing_topup_same_name.sh
Repository files navigation
Marathon of Hope Automation
This repository contains all of the scripts for the automation of the MOH projects
and integration of genpipes
Abacus Scripts
Activate_globus.sh
Activates the globus endpoints for Create_readset_transfer_BAMS.sh
Must be updated for individual users

Usage: Activate_globus.sh
Create_readset_transfer_BAMS.sh
Takes the full path of the read folder as an input and transfers the BAMs/fastqs with the
MoH prefix.In addition it tranfers over the key run processing metrics and generates
a log file forlater usage with the database
Globus fields must be updated for individual users

Usage: Create_readset_transfer_BAMS.sh PATH_TO_RUN_FOLDER
Beluga Scripts
Metrics_Update.py
This script takes a single file and parses it to update the database with metrics from
run processing and from both DNA and RNA Genpipes. It compares the extracted values to
known acceptable values and adds them to the KEY_METRICS table within the MoH database.

Usage: Metrics_Update.py
MOH_Check_Progress.py
Parses the file structure of MAIN and looks for the output files produced from samples in
the Sample table. It then updates STATUS table with any progress. It queries for all files,
so any deliverables that are removed will result in an incomplete listing. In addition this
script populates/updates the Timestamps and File_Locations tables.

Usage: MOH_Check_Progress.py
DB_OPS.py
This contains all of the functions necessary for interacting with a file. Required for
other scripts.
MOH_ln_output.py
Takes a single sample input and hard links all of the deliverable files within a structured
directory with their final names. In addition it constructs the custom readme and log files.
All data is taken from the database.

Usage: MOH_ln_output.py Sample
Currently has a coverage cutoff implemented
Create_CSVs.sh
Dumps the tables within the database as csv's within the CSV folder of DATABASE.

Usage: Create_CSVs.sh
Generate_pairs_readset.py
Searches the temporary raw_reads folder for matching DNA pairs or RNA. It moves the
fastqs/BAMs and creates the readset and pairs files in preparation for Setup_run_XXX.sh.
In addition, it populates the Samples table. It will not move process any files that are
outside the naming convention and it will exit if it finds files with the same name at the
destination.

Usage Generate_pairs_readset.py DNA or Generate_pairs_readset.py RNA
About
Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation.
Resources
Readme
Activity
Custom properties
Stars
1 star
Watchers
1 watching
Forks
0 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
GitHub - c3g/MoH_Automation: Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation. · GitHub
Skip to content
BranchesTags
Latest commit
History
614 Commits
Folders and files
NameName
Last commit message
Last commit date
.gitignore
.gitignore
Activate_globus.sh
Activate_globus.sh
Create_CSVs.sh
Create_CSVs.sh
Create_readset_transfer_BAMS.sh
Create_readset_transfer_BAMS.sh
DB_OPS.py
DB_OPS.py
GenPipes_wrapper.sh
GenPipes_wrapper.sh
Generate_pairs_readset.py
Generate_pairs_readset.py
Generate_pairs_readset_topup.py
Generate_pairs_readset_topup.py
MOH_Check_Progress.py
MOH_Check_Progress.py
MOH_ln_output.py
MOH_ln_output.py
Metrics_Update.py
Metrics_Update.py
ProjectTracking_RunProcessing.sh
ProjectTracking_RunProcessing.sh
README.md
README.md
add_mosdepth.py
add_mosdepth.py
bucket_delivery.py
bucket_delivery.py
bucket_delivery.sh
bucket_delivery.sh
check_GenPipes.sh
check_GenPipes.sh
create_readset_all_samples.py
create_readset_all_samples.py
csv2json.py
csv2json.py
fix_bam_paths.sh
fix_bam_paths.sh
fms_ProjectTracking_RunProcessing.sh
fms_ProjectTracking_RunProcessing.sh
fms_run_processing2json.py
fms_run_processing2json.py
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing_topup_same_name.sh
fms_transfer_RunProcessing_topup_same_name.sh
genpipes_deliverables_metrics_tagging.py
genpipes_deliverables_metrics_tagging.py
globus_acl.sh
globus_acl.sh
globus_collections.json
globus_collections.json
ingest_sample_to_old_db.py
ingest_sample_to_old_db.py
jenkins_wrapper.sh
jenkins_wrapper.sh
jsonify_db.py
jsonify_db.py
merge_GenPipes_jsons.py
merge_GenPipes_jsons.py
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_run_processing2json.py
mgc_mock_run_processing2json.py
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
parse_check_GenPipes_log.sh
parse_check_GenPipes_log.sh
pt_check_sample.sh
pt_check_sample.sh
rename_sample_in_bams.sh
rename_sample_in_bams.sh
run_processing2json.py
run_processing2json.py
transfer2json.py
transfer2json.py
transfer_GenPipes.sh
transfer_GenPipes.sh
transfer_RunProcessing_topup_same_name.sh
transfer_RunProcessing_topup_same_name.sh
Repository files navigation
Marathon of Hope Automation
This repository contains all of the scripts for the automation of the MOH projects
and integration of genpipes
Abacus Scripts
Activate_globus.sh
Activates the globus endpoints for Create_readset_transfer_BAMS.sh
Must be updated for individual users

Usage: Activate_globus.sh
Create_readset_transfer_BAMS.sh
Takes the full path of the read folder as an input and transfers the BAMs/fastqs with the
MoH prefix.In addition it tranfers over the key run processing metrics and generates
a log file forlater usage with the database
Globus fields must be updated for individual users

Usage: Create_readset_transfer_BAMS.sh PATH_TO_RUN_FOLDER
Beluga Scripts
Metrics_Update.py
This script takes a single file and parses it to update the database with metrics from
run processing and from both DNA and RNA Genpipes. It compares the extracted values to
known acceptable values and adds them to the KEY_METRICS table within the MoH database.

Usage: Metrics_Update.py
MOH_Check_Progress.py
Parses the file structure of MAIN and looks for the output files produced from samples in
the Sample table. It then updates STATUS table with any progress. It queries for all files,
so any deliverables that are removed will result in an incomplete listing. In addition this
script populates/updates the Timestamps and File_Locations tables.

Usage: MOH_Check_Progress.py
DB_OPS.py
This contains all of the functions necessary for interacting with a file. Required for
other scripts.
MOH_ln_output.py
Takes a single sample input and hard links all of the deliverable files within a structured
directory with their final names. In addition it constructs the custom readme and log files.
All data is taken from the database.

Usage: MOH_ln_output.py Sample
Currently has a coverage cutoff implemented
Create_CSVs.sh
Dumps the tables within the database as csv's within the CSV folder of DATABASE.

Usage: Create_CSVs.sh
Generate_pairs_readset.py
Searches the temporary raw_reads folder for matching DNA pairs or RNA. It moves the
fastqs/BAMs and creates the readset and pairs files in preparation for Setup_run_XXX.sh.
In addition, it populates the Samples table. It will not move process any files that are
outside the naming convention and it will exit if it finds files with the same name at the
destination.

Usage Generate_pairs_readset.py DNA or Generate_pairs_readset.py RNA
About
Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation.
Resources
Readme
Activity
Custom properties
Stars
1 star
Watchers
1 watching
Forks
0 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - c3g/MoH_Automation: Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation. · GitHub
Skip to content
BranchesTags
Latest commit
History
614 Commits
Folders and files
NameName
Last commit message
Last commit date
.gitignore
.gitignore
Activate_globus.sh
Activate_globus.sh
Create_CSVs.sh
Create_CSVs.sh
Create_readset_transfer_BAMS.sh
Create_readset_transfer_BAMS.sh
DB_OPS.py
DB_OPS.py
GenPipes_wrapper.sh
GenPipes_wrapper.sh
Generate_pairs_readset.py
Generate_pairs_readset.py
Generate_pairs_readset_topup.py
Generate_pairs_readset_topup.py
MOH_Check_Progress.py
MOH_Check_Progress.py
MOH_ln_output.py
MOH_ln_output.py
Metrics_Update.py
Metrics_Update.py
ProjectTracking_RunProcessing.sh
ProjectTracking_RunProcessing.sh
README.md
README.md
add_mosdepth.py
add_mosdepth.py
bucket_delivery.py
bucket_delivery.py
bucket_delivery.sh
bucket_delivery.sh
check_GenPipes.sh
check_GenPipes.sh
create_readset_all_samples.py
create_readset_all_samples.py
csv2json.py
csv2json.py
fix_bam_paths.sh
fix_bam_paths.sh
fms_ProjectTracking_RunProcessing.sh
fms_ProjectTracking_RunProcessing.sh
fms_run_processing2json.py
fms_run_processing2json.py
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing_topup_same_name.sh
fms_transfer_RunProcessing_topup_same_name.sh
genpipes_deliverables_metrics_tagging.py
genpipes_deliverables_metrics_tagging.py
globus_acl.sh
globus_acl.sh
globus_collections.json
globus_collections.json
ingest_sample_to_old_db.py
ingest_sample_to_old_db.py
jenkins_wrapper.sh
jenkins_wrapper.sh
jsonify_db.py
jsonify_db.py
merge_GenPipes_jsons.py
merge_GenPipes_jsons.py
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_run_processing2json.py
mgc_mock_run_processing2json.py
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
parse_check_GenPipes_log.sh
parse_check_GenPipes_log.sh
pt_check_sample.sh
pt_check_sample.sh
rename_sample_in_bams.sh
rename_sample_in_bams.sh
run_processing2json.py
run_processing2json.py
transfer2json.py
transfer2json.py
transfer_GenPipes.sh
transfer_GenPipes.sh
transfer_RunProcessing_topup_same_name.sh
transfer_RunProcessing_topup_same_name.sh
Repository files navigation
Marathon of Hope Automation
This repository contains all of the scripts for the automation of the MOH projects
and integration of genpipes
Abacus Scripts
Activate_globus.sh
Activates the globus endpoints for Create_readset_transfer_BAMS.sh
Must be updated for individual users

Usage: Activate_globus.sh
Create_readset_transfer_BAMS.sh
Takes the full path of the read folder as an input and transfers the BAMs/fastqs with the
MoH prefix.In addition it tranfers over the key run processing metrics and generates
a log file forlater usage with the database
Globus fields must be updated for individual users

Usage: Create_readset_transfer_BAMS.sh PATH_TO_RUN_FOLDER
Beluga Scripts
Metrics_Update.py
This script takes a single file and parses it to update the database with metrics from
run processing and from both DNA and RNA Genpipes. It compares the extracted values to
known acceptable values and adds them to the KEY_METRICS table within the MoH database.

Usage: Metrics_Update.py
MOH_Check_Progress.py
Parses the file structure of MAIN and looks for the output files produced from samples in
the Sample table. It then updates STATUS table with any progress. It queries for all files,
so any deliverables that are removed will result in an incomplete listing. In addition this
script populates/updates the Timestamps and File_Locations tables.

Usage: MOH_Check_Progress.py
DB_OPS.py
This contains all of the functions necessary for interacting with a file. Required for
other scripts.
MOH_ln_output.py
Takes a single sample input and hard links all of the deliverable files within a structured
directory with their final names. In addition it constructs the custom readme and log files.
All data is taken from the database.

Usage: MOH_ln_output.py Sample
Currently has a coverage cutoff implemented
Create_CSVs.sh
Dumps the tables within the database as csv's within the CSV folder of DATABASE.

Usage: Create_CSVs.sh
Generate_pairs_readset.py
Searches the temporary raw_reads folder for matching DNA pairs or RNA. It moves the
fastqs/BAMs and creates the readset and pairs files in preparation for Setup_run_XXX.sh.
In addition, it populates the Samples table. It will not move process any files that are
outside the naming convention and it will exit if it finds files with the same name at the
destination.

Usage Generate_pairs_readset.py DNA or Generate_pairs_readset.py RNA
About
Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation.
Resources
Readme
Activity
Custom properties
Stars
1 star
Watchers
1 watching
Forks
0 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
GitHub - c3g/MoH_Automation: Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation. · GitHub
Skip to content
BranchesTags
Latest commit
History
614 Commits
Folders and files
NameName
Last commit message
Last commit date
.gitignore
.gitignore
Activate_globus.sh
Activate_globus.sh
Create_CSVs.sh
Create_CSVs.sh
Create_readset_transfer_BAMS.sh
Create_readset_transfer_BAMS.sh
DB_OPS.py
DB_OPS.py
GenPipes_wrapper.sh
GenPipes_wrapper.sh
Generate_pairs_readset.py
Generate_pairs_readset.py
Generate_pairs_readset_topup.py
Generate_pairs_readset_topup.py
MOH_Check_Progress.py
MOH_Check_Progress.py
MOH_ln_output.py
MOH_ln_output.py
Metrics_Update.py
Metrics_Update.py
ProjectTracking_RunProcessing.sh
ProjectTracking_RunProcessing.sh
README.md
README.md
add_mosdepth.py
add_mosdepth.py
bucket_delivery.py
bucket_delivery.py
bucket_delivery.sh
bucket_delivery.sh
check_GenPipes.sh
check_GenPipes.sh
create_readset_all_samples.py
create_readset_all_samples.py
csv2json.py
csv2json.py
fix_bam_paths.sh
fix_bam_paths.sh
fms_ProjectTracking_RunProcessing.sh
fms_ProjectTracking_RunProcessing.sh
fms_run_processing2json.py
fms_run_processing2json.py
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing_topup_same_name.sh
fms_transfer_RunProcessing_topup_same_name.sh
genpipes_deliverables_metrics_tagging.py
genpipes_deliverables_metrics_tagging.py
globus_acl.sh
globus_acl.sh
globus_collections.json
globus_collections.json
ingest_sample_to_old_db.py
ingest_sample_to_old_db.py
jenkins_wrapper.sh
jenkins_wrapper.sh
jsonify_db.py
jsonify_db.py
merge_GenPipes_jsons.py
merge_GenPipes_jsons.py
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_run_processing2json.py
mgc_mock_run_processing2json.py
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
parse_check_GenPipes_log.sh
parse_check_GenPipes_log.sh
pt_check_sample.sh
pt_check_sample.sh
rename_sample_in_bams.sh
rename_sample_in_bams.sh
run_processing2json.py
run_processing2json.py
transfer2json.py
transfer2json.py
transfer_GenPipes.sh
transfer_GenPipes.sh
transfer_RunProcessing_topup_same_name.sh
transfer_RunProcessing_topup_same_name.sh
Repository files navigation
Marathon of Hope Automation
This repository contains all of the scripts for the automation of the MOH projects
and integration of genpipes
Abacus Scripts
Activate_globus.sh
Activates the globus endpoints for Create_readset_transfer_BAMS.sh
Must be updated for individual users

Usage: Activate_globus.sh
Create_readset_transfer_BAMS.sh
Takes the full path of the read folder as an input and transfers the BAMs/fastqs with the
MoH prefix.In addition it tranfers over the key run processing metrics and generates
a log file forlater usage with the database
Globus fields must be updated for individual users

Usage: Create_readset_transfer_BAMS.sh PATH_TO_RUN_FOLDER
Beluga Scripts
Metrics_Update.py
This script takes a single file and parses it to update the database with metrics from
run processing and from both DNA and RNA Genpipes. It compares the extracted values to
known acceptable values and adds them to the KEY_METRICS table within the MoH database.

Usage: Metrics_Update.py
MOH_Check_Progress.py
Parses the file structure of MAIN and looks for the output files produced from samples in
the Sample table. It then updates STATUS table with any progress. It queries for all files,
so any deliverables that are removed will result in an incomplete listing. In addition this
script populates/updates the Timestamps and File_Locations tables.

Usage: MOH_Check_Progress.py
DB_OPS.py
This contains all of the functions necessary for interacting with a file. Required for
other scripts.
MOH_ln_output.py
Takes a single sample input and hard links all of the deliverable files within a structured
directory with their final names. In addition it constructs the custom readme and log files.
All data is taken from the database.

Usage: MOH_ln_output.py Sample
Currently has a coverage cutoff implemented
Create_CSVs.sh
Dumps the tables within the database as csv's within the CSV folder of DATABASE.

Usage: Create_CSVs.sh
Generate_pairs_readset.py
Searches the temporary raw_reads folder for matching DNA pairs or RNA. It moves the
fastqs/BAMs and creates the readset and pairs files in preparation for Setup_run_XXX.sh.
In addition, it populates the Samples table. It will not move process any files that are
outside the naming convention and it will exit if it finds files with the same name at the
destination.

Usage Generate_pairs_readset.py DNA or Generate_pairs_readset.py RNA
About
Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation.
Resources
Readme
Activity
Custom properties
Stars
1 star
Watchers
1 watching
Forks
0 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
GitHub - c3g/MoH_Automation: Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation. · GitHub
Skip to content
BranchesTags
Latest commit
History
614 Commits
Folders and files
NameName
Last commit message
Last commit date
.gitignore
.gitignore
Activate_globus.sh
Activate_globus.sh
Create_CSVs.sh
Create_CSVs.sh
Create_readset_transfer_BAMS.sh
Create_readset_transfer_BAMS.sh
DB_OPS.py
DB_OPS.py
GenPipes_wrapper.sh
GenPipes_wrapper.sh
Generate_pairs_readset.py
Generate_pairs_readset.py
Generate_pairs_readset_topup.py
Generate_pairs_readset_topup.py
MOH_Check_Progress.py
MOH_Check_Progress.py
MOH_ln_output.py
MOH_ln_output.py
Metrics_Update.py
Metrics_Update.py
ProjectTracking_RunProcessing.sh
ProjectTracking_RunProcessing.sh
README.md
README.md
add_mosdepth.py
add_mosdepth.py
bucket_delivery.py
bucket_delivery.py
bucket_delivery.sh
bucket_delivery.sh
check_GenPipes.sh
check_GenPipes.sh
create_readset_all_samples.py
create_readset_all_samples.py
csv2json.py
csv2json.py
fix_bam_paths.sh
fix_bam_paths.sh
fms_ProjectTracking_RunProcessing.sh
fms_ProjectTracking_RunProcessing.sh
fms_run_processing2json.py
fms_run_processing2json.py
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing.sh
fms_transfer_RunProcessing_topup_same_name.sh
fms_transfer_RunProcessing_topup_same_name.sh
genpipes_deliverables_metrics_tagging.py
genpipes_deliverables_metrics_tagging.py
globus_acl.sh
globus_acl.sh
globus_collections.json
globus_collections.json
ingest_sample_to_old_db.py
ingest_sample_to_old_db.py
jenkins_wrapper.sh
jenkins_wrapper.sh
jsonify_db.py
jsonify_db.py
merge_GenPipes_jsons.py
merge_GenPipes_jsons.py
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_ProjectTracking_RunProcessing.sh
mgc_mock_run_processing2json.py
mgc_mock_run_processing2json.py
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
mgc_mock_transfer_RunProcessing_topup_same_name.sh
parse_check_GenPipes_log.sh
parse_check_GenPipes_log.sh
pt_check_sample.sh
pt_check_sample.sh
rename_sample_in_bams.sh
rename_sample_in_bams.sh
run_processing2json.py
run_processing2json.py
transfer2json.py
transfer2json.py
transfer_GenPipes.sh
transfer_GenPipes.sh
transfer_RunProcessing_topup_same_name.sh
transfer_RunProcessing_topup_same_name.sh
Repository files navigation
Marathon of Hope Automation
This repository contains all of the scripts for the automation of the MOH projects
and integration of genpipes
Abacus Scripts
Activate_globus.sh
Activates the globus endpoints for Create_readset_transfer_BAMS.sh
Must be updated for individual users

Usage: Activate_globus.sh
Create_readset_transfer_BAMS.sh
Takes the full path of the read folder as an input and transfers the BAMs/fastqs with the
MoH prefix.In addition it tranfers over the key run processing metrics and generates
a log file forlater usage with the database
Globus fields must be updated for individual users

Usage: Create_readset_transfer_BAMS.sh PATH_TO_RUN_FOLDER
Beluga Scripts
Metrics_Update.py
This script takes a single file and parses it to update the database with metrics from
run processing and from both DNA and RNA Genpipes. It compares the extracted values to
known acceptable values and adds them to the KEY_METRICS table within the MoH database.

Usage: Metrics_Update.py
MOH_Check_Progress.py
Parses the file structure of MAIN and looks for the output files produced from samples in
the Sample table. It then updates STATUS table with any progress. It queries for all files,
so any deliverables that are removed will result in an incomplete listing. In addition this
script populates/updates the Timestamps and File_Locations tables.

Usage: MOH_Check_Progress.py
DB_OPS.py
This contains all of the functions necessary for interacting with a file. Required for
other scripts.
MOH_ln_output.py
Takes a single sample input and hard links all of the deliverable files within a structured
directory with their final names. In addition it constructs the custom readme and log files.
All data is taken from the database.

Usage: MOH_ln_output.py Sample
Currently has a coverage cutoff implemented
Create_CSVs.sh
Dumps the tables within the database as csv's within the CSV folder of DATABASE.

Usage: Create_CSVs.sh
Generate_pairs_readset.py
Searches the temporary raw_reads folder for matching DNA pairs or RNA. It moves the
fastqs/BAMs and creates the readset and pairs files in preparation for Setup_run_XXX.sh.
In addition, it populates the Samples table. It will not move process any files that are
outside the naming convention and it will exit if it finds files with the same name at the
destination.

Usage Generate_pairs_readset.py DNA or Generate_pairs_readset.py RNA
About
Automation for MoH. It includes capabilities for file transfer, tracking, and genpipes, initiation.
Resources
Readme
Activity
Custom properties
Stars
1 star
Watchers
1 watching
Forks
0 forks
Report repository
Releases
Packages
Contributors
Languages