BranchesTags

Repository files navigation

AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder

Generic badgemade-with-pythonWebsite shields.ioPyPI version fury.io

VQ-VAE

This repository contains the code associated with the following publication:

AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
Samir Sadok, Simon Leglaive, Laurent Girin, Gaël Richard, Xavier Alameda-Pineda
International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2025.

If you use this code for your research, please cite the above paper.

Useful links:

Setup

  • Pypi:
    • pip install -i https://test.pypi.org/simple/ ancogen --no-deps
  • Install the package locally (for use on your system):
    • In the current directory: pip install -e .
  • Virtual Environment:
    • conda create -n ancogen python=3.9
    • conda activate ancogen
    • In the current directory: pip install -r requirements.txt

Usage

Pretrained models

After loading the weights of the pre-trained models: speechVQVAE, HIFIGAN and AnCoGen, put them all in the NestAnCoGen class.

fromsrcimportNestAnCoGenancogen=NestAnCoGen(ancogen=model, hifigan=generator, vqvae=vqvae, improved=False)
ModelLink
Speech-VQVAElink
HiFi-GANlink
AnCoGenlink / link (Improved)

Analysis

To do analysis with AnCoGen (link, test_analyse), which correspond to the estimation of the speech attributes from a Mel-spectrogram. Please see the paper for a complete description of the attributes.

"""Test the analyse function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Pitch estimation + plottingaudio, attributes=ancogen.analyse(audio, apply_max=True, attribute_name="pitch", plot_bool=True)

Generation

To do speech analysis-resynthesis mapping wih AnCoGen (link, test_generation) which are simply obtained by using AnCoGen to map a Mel-spectrogram to the corresponding speech attributes (analysis stage) and then back to the Mel-spectrogram (generation stage).

"""Test the generation function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Generate outputgenerated=ancogen.generate(path=PATH_AUDIO, from_attributes=None, save_dir="wavs", return_metrics=True)

Control

To do analysis, transformation, and synthesis with AnCoGen, where the speech attributes are controlled between the analysis and generation stages in order to perform speech denoising (by increasing the SNR attribute), pitch shifting, dereverberation (by increasing the C50 attribute) or voice conversion (by controlling the speaker identity attribute).

"""Test the control function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Control with AnCoGenancogen.pitch_control(attributes, target_pitch, **kwargs) ancogen.content_control(attributes, target_content_index, **kwargs)
ancogen.snr_control(attributes, target_snr, **kwargs)
ancogen.c50_control(attributes, target_c50, **kwargs)
ancogen.voice_conversion(target_identity: str, source_signal: str, save_dir: str='')

License

GNU Affero General Public License (version 3), see LICENSE.txt.

About

[ICASSP 2025] AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder

samsad35.github.io/site-ancogen/

Topics

audio-processinggenerative-modelmasked-autoencoderpitch-estimationpitch-shiftspeech-analysisspeech-synthesisvoice-conversion

Resources

Readme
AGPL-3.0 license
Activity

Stars

15 stars

Watchers

3 watching

Forks

1 fork
Report repository

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
BranchesTags
Repository files navigation
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
Generic badgemade-with-pythonWebsite shields.ioPyPI version fury.io
VQ-VAE
This repository contains the code associated with the following publication:
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
 Samir Sadok, Simon Leglaive, Laurent Girin, Gaël Richard, Xavier Alameda-Pineda
 International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2025.
If you use this code for your research, please cite the above paper.
Useful links:
Setup
  •  Pypi:

    • pip install -i https://test.pypi.org/simple/ ancogen --no-deps
  •  Install the package locally (for use on your system):

    • In the current directory: pip install -e .
  •  Virtual Environment:

    • conda create -n ancogen python=3.9
    • conda activate ancogen
    • In the current directory: pip install -r requirements.txt
Usage
Pretrained models
After loading the weights of the pre-trained models: speechVQVAE, HIFIGAN and AnCoGen, put them all in the NestAnCoGen class.
fromsrcimportNestAnCoGenancogen=NestAnCoGen(ancogen=model, hifigan=generator, vqvae=vqvae, improved=False)
ModelLink
Speech-VQVAElink
HiFi-GANlink
AnCoGenlink / link (Improved)
Analysis
To do analysis with AnCoGen (link, test_analyse), which correspond to the estimation of the speech attributes from a Mel-spectrogram.
Please see the paper for a complete description of the attributes.
"""Test the analyse function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Pitch estimation + plottingaudio, attributes=ancogen.analyse(audio, apply_max=True, attribute_name="pitch", plot_bool=True)
Generation
To do speech analysis-resynthesis mapping wih AnCoGen (link, test_generation) which are simply obtained by using AnCoGen to map a Mel-spectrogram to the corresponding speech attributes (analysis stage) and then back to the Mel-spectrogram (generation stage).
"""Test the generation function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Generate outputgenerated=ancogen.generate(path=PATH_AUDIO, from_attributes=None, save_dir="wavs", return_metrics=True)
Control
To do analysis, transformation, and synthesis with AnCoGen, where the speech attributes are controlled between the analysis and generation stages in order to perform speech denoising (by increasing the SNR attribute), pitch shifting, dereverberation (by increasing the C50 attribute) or voice conversion (by controlling the speaker identity attribute).
"""Test the control function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Control with AnCoGenancogen.pitch_control(attributes, target_pitch, **kwargs) ancogen.content_control(attributes, target_content_index, **kwargs)
ancogen.snr_control(attributes, target_snr, **kwargs)
ancogen.c50_control(attributes, target_c50, **kwargs)
ancogen.voice_conversion(target_identity: str, source_signal: str, save_dir: str='')
License
GNU Affero General Public License (version 3), see LICENSE.txt.
About
[ICASSP 2025] AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
samsad35.github.io/site-ancogen/
Topics
audio-processinggenerative-modelmasked-autoencoderpitch-estimationpitch-shiftspeech-analysisspeech-synthesisvoice-conversion
Resources
Readme
AGPL-3.0 license
Activity
Stars
15 stars
Watchers
3 watching
Forks
1 fork
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
Generic badgemade-with-pythonWebsite shields.ioPyPI version fury.io
VQ-VAE
This repository contains the code associated with the following publication:
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
 Samir Sadok, Simon Leglaive, Laurent Girin, Gaël Richard, Xavier Alameda-Pineda
 International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2025.
If you use this code for your research, please cite the above paper.
Useful links:
Setup
  •  Pypi:

    • pip install -i https://test.pypi.org/simple/ ancogen --no-deps
  •  Install the package locally (for use on your system):

    • In the current directory: pip install -e .
  •  Virtual Environment:

    • conda create -n ancogen python=3.9
    • conda activate ancogen
    • In the current directory: pip install -r requirements.txt
Usage
Pretrained models
After loading the weights of the pre-trained models: speechVQVAE, HIFIGAN and AnCoGen, put them all in the NestAnCoGen class.
fromsrcimportNestAnCoGenancogen=NestAnCoGen(ancogen=model, hifigan=generator, vqvae=vqvae, improved=False)
ModelLink
Speech-VQVAElink
HiFi-GANlink
AnCoGenlink / link (Improved)
Analysis
To do analysis with AnCoGen (link, test_analyse), which correspond to the estimation of the speech attributes from a Mel-spectrogram.
Please see the paper for a complete description of the attributes.
"""Test the analyse function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Pitch estimation + plottingaudio, attributes=ancogen.analyse(audio, apply_max=True, attribute_name="pitch", plot_bool=True)
Generation
To do speech analysis-resynthesis mapping wih AnCoGen (link, test_generation) which are simply obtained by using AnCoGen to map a Mel-spectrogram to the corresponding speech attributes (analysis stage) and then back to the Mel-spectrogram (generation stage).
"""Test the generation function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Generate outputgenerated=ancogen.generate(path=PATH_AUDIO, from_attributes=None, save_dir="wavs", return_metrics=True)
Control
To do analysis, transformation, and synthesis with AnCoGen, where the speech attributes are controlled between the analysis and generation stages in order to perform speech denoising (by increasing the SNR attribute), pitch shifting, dereverberation (by increasing the C50 attribute) or voice conversion (by controlling the speaker identity attribute).
"""Test the control function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Control with AnCoGenancogen.pitch_control(attributes, target_pitch, **kwargs) ancogen.content_control(attributes, target_content_index, **kwargs)
ancogen.snr_control(attributes, target_snr, **kwargs)
ancogen.c50_control(attributes, target_c50, **kwargs)
ancogen.voice_conversion(target_identity: str, source_signal: str, save_dir: str='')
License
GNU Affero General Public License (version 3), see LICENSE.txt.
About
[ICASSP 2025] AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
samsad35.github.io/site-ancogen/
Topics
audio-processinggenerative-modelmasked-autoencoderpitch-estimationpitch-shiftspeech-analysisspeech-synthesisvoice-conversion
Resources
Readme
AGPL-3.0 license
Activity
Stars
15 stars
Watchers
3 watching
Forks
1 fork
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
Generic badgemade-with-pythonWebsite shields.ioPyPI version fury.io
VQ-VAE
This repository contains the code associated with the following publication:
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
 Samir Sadok, Simon Leglaive, Laurent Girin, Gaël Richard, Xavier Alameda-Pineda
 International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2025.
If you use this code for your research, please cite the above paper.
Useful links:
Setup
  •  Pypi:

    • pip install -i https://test.pypi.org/simple/ ancogen --no-deps
  •  Install the package locally (for use on your system):

    • In the current directory: pip install -e .
  •  Virtual Environment:

    • conda create -n ancogen python=3.9
    • conda activate ancogen
    • In the current directory: pip install -r requirements.txt
Usage
Pretrained models
After loading the weights of the pre-trained models: speechVQVAE, HIFIGAN and AnCoGen, put them all in the NestAnCoGen class.
fromsrcimportNestAnCoGenancogen=NestAnCoGen(ancogen=model, hifigan=generator, vqvae=vqvae, improved=False)
ModelLink
Speech-VQVAElink
HiFi-GANlink
AnCoGenlink / link (Improved)
Analysis
To do analysis with AnCoGen (link, test_analyse), which correspond to the estimation of the speech attributes from a Mel-spectrogram.
Please see the paper for a complete description of the attributes.
"""Test the analyse function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Pitch estimation + plottingaudio, attributes=ancogen.analyse(audio, apply_max=True, attribute_name="pitch", plot_bool=True)
Generation
To do speech analysis-resynthesis mapping wih AnCoGen (link, test_generation) which are simply obtained by using AnCoGen to map a Mel-spectrogram to the corresponding speech attributes (analysis stage) and then back to the Mel-spectrogram (generation stage).
"""Test the generation function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Generate outputgenerated=ancogen.generate(path=PATH_AUDIO, from_attributes=None, save_dir="wavs", return_metrics=True)
Control
To do analysis, transformation, and synthesis with AnCoGen, where the speech attributes are controlled between the analysis and generation stages in order to perform speech denoising (by increasing the SNR attribute), pitch shifting, dereverberation (by increasing the C50 attribute) or voice conversion (by controlling the speaker identity attribute).
"""Test the control function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Control with AnCoGenancogen.pitch_control(attributes, target_pitch, **kwargs) ancogen.content_control(attributes, target_content_index, **kwargs)
ancogen.snr_control(attributes, target_snr, **kwargs)
ancogen.c50_control(attributes, target_c50, **kwargs)
ancogen.voice_conversion(target_identity: str, source_signal: str, save_dir: str='')
License
GNU Affero General Public License (version 3), see LICENSE.txt.
About
[ICASSP 2025] AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
samsad35.github.io/site-ancogen/
Topics
audio-processinggenerative-modelmasked-autoencoderpitch-estimationpitch-shiftspeech-analysisspeech-synthesisvoice-conversion
Resources
Readme
AGPL-3.0 license
Activity
Stars
15 stars
Watchers
3 watching
Forks
1 fork
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
BranchesTags
Repository files navigation
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
Generic badgemade-with-pythonWebsite shields.ioPyPI version fury.io
VQ-VAE
This repository contains the code associated with the following publication:
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
 Samir Sadok, Simon Leglaive, Laurent Girin, Gaël Richard, Xavier Alameda-Pineda
 International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2025.
If you use this code for your research, please cite the above paper.
Useful links:
Setup
  •  Pypi:

    • pip install -i https://test.pypi.org/simple/ ancogen --no-deps
  •  Install the package locally (for use on your system):

    • In the current directory: pip install -e .
  •  Virtual Environment:

    • conda create -n ancogen python=3.9
    • conda activate ancogen
    • In the current directory: pip install -r requirements.txt
Usage
Pretrained models
After loading the weights of the pre-trained models: speechVQVAE, HIFIGAN and AnCoGen, put them all in the NestAnCoGen class.
fromsrcimportNestAnCoGenancogen=NestAnCoGen(ancogen=model, hifigan=generator, vqvae=vqvae, improved=False)
ModelLink
Speech-VQVAElink
HiFi-GANlink
AnCoGenlink / link (Improved)
Analysis
To do analysis with AnCoGen (link, test_analyse), which correspond to the estimation of the speech attributes from a Mel-spectrogram.
Please see the paper for a complete description of the attributes.
"""Test the analyse function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Pitch estimation + plottingaudio, attributes=ancogen.analyse(audio, apply_max=True, attribute_name="pitch", plot_bool=True)
Generation
To do speech analysis-resynthesis mapping wih AnCoGen (link, test_generation) which are simply obtained by using AnCoGen to map a Mel-spectrogram to the corresponding speech attributes (analysis stage) and then back to the Mel-spectrogram (generation stage).
"""Test the generation function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Generate outputgenerated=ancogen.generate(path=PATH_AUDIO, from_attributes=None, save_dir="wavs", return_metrics=True)
Control
To do analysis, transformation, and synthesis with AnCoGen, where the speech attributes are controlled between the analysis and generation stages in order to perform speech denoising (by increasing the SNR attribute), pitch shifting, dereverberation (by increasing the C50 attribute) or voice conversion (by controlling the speaker identity attribute).
"""Test the control function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Control with AnCoGenancogen.pitch_control(attributes, target_pitch, **kwargs) ancogen.content_control(attributes, target_content_index, **kwargs)
ancogen.snr_control(attributes, target_snr, **kwargs)
ancogen.c50_control(attributes, target_c50, **kwargs)
ancogen.voice_conversion(target_identity: str, source_signal: str, save_dir: str='')
License
GNU Affero General Public License (version 3), see LICENSE.txt.
About
[ICASSP 2025] AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
samsad35.github.io/site-ancogen/
Topics
audio-processinggenerative-modelmasked-autoencoderpitch-estimationpitch-shiftspeech-analysisspeech-synthesisvoice-conversion
Resources
Readme
AGPL-3.0 license
Activity
Stars
15 stars
Watchers
3 watching
Forks
1 fork
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
Generic badgemade-with-pythonWebsite shields.ioPyPI version fury.io
VQ-VAE
This repository contains the code associated with the following publication:
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
 Samir Sadok, Simon Leglaive, Laurent Girin, Gaël Richard, Xavier Alameda-Pineda
 International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2025.
If you use this code for your research, please cite the above paper.
Useful links:
Setup
  •  Pypi:

    • pip install -i https://test.pypi.org/simple/ ancogen --no-deps
  •  Install the package locally (for use on your system):

    • In the current directory: pip install -e .
  •  Virtual Environment:

    • conda create -n ancogen python=3.9
    • conda activate ancogen
    • In the current directory: pip install -r requirements.txt
Usage
Pretrained models
After loading the weights of the pre-trained models: speechVQVAE, HIFIGAN and AnCoGen, put them all in the NestAnCoGen class.
fromsrcimportNestAnCoGenancogen=NestAnCoGen(ancogen=model, hifigan=generator, vqvae=vqvae, improved=False)
ModelLink
Speech-VQVAElink
HiFi-GANlink
AnCoGenlink / link (Improved)
Analysis
To do analysis with AnCoGen (link, test_analyse), which correspond to the estimation of the speech attributes from a Mel-spectrogram.
Please see the paper for a complete description of the attributes.
"""Test the analyse function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Pitch estimation + plottingaudio, attributes=ancogen.analyse(audio, apply_max=True, attribute_name="pitch", plot_bool=True)
Generation
To do speech analysis-resynthesis mapping wih AnCoGen (link, test_generation) which are simply obtained by using AnCoGen to map a Mel-spectrogram to the corresponding speech attributes (analysis stage) and then back to the Mel-spectrogram (generation stage).
"""Test the generation function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Generate outputgenerated=ancogen.generate(path=PATH_AUDIO, from_attributes=None, save_dir="wavs", return_metrics=True)
Control
To do analysis, transformation, and synthesis with AnCoGen, where the speech attributes are controlled between the analysis and generation stages in order to perform speech denoising (by increasing the SNR attribute), pitch shifting, dereverberation (by increasing the C50 attribute) or voice conversion (by controlling the speaker identity attribute).
"""Test the control function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Control with AnCoGenancogen.pitch_control(attributes, target_pitch, **kwargs) ancogen.content_control(attributes, target_content_index, **kwargs)
ancogen.snr_control(attributes, target_snr, **kwargs)
ancogen.c50_control(attributes, target_c50, **kwargs)
ancogen.voice_conversion(target_identity: str, source_signal: str, save_dir: str='')
License
GNU Affero General Public License (version 3), see LICENSE.txt.
About
[ICASSP 2025] AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
samsad35.github.io/site-ancogen/
Topics
audio-processinggenerative-modelmasked-autoencoderpitch-estimationpitch-shiftspeech-analysisspeech-synthesisvoice-conversion
Resources
Readme
AGPL-3.0 license
Activity
Stars
15 stars
Watchers
3 watching
Forks
1 fork
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
Generic badgemade-with-pythonWebsite shields.ioPyPI version fury.io
VQ-VAE
This repository contains the code associated with the following publication:
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
 Samir Sadok, Simon Leglaive, Laurent Girin, Gaël Richard, Xavier Alameda-Pineda
 International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2025.
If you use this code for your research, please cite the above paper.
Useful links:
Setup
  •  Pypi:

    • pip install -i https://test.pypi.org/simple/ ancogen --no-deps
  •  Install the package locally (for use on your system):

    • In the current directory: pip install -e .
  •  Virtual Environment:

    • conda create -n ancogen python=3.9
    • conda activate ancogen
    • In the current directory: pip install -r requirements.txt
Usage
Pretrained models
After loading the weights of the pre-trained models: speechVQVAE, HIFIGAN and AnCoGen, put them all in the NestAnCoGen class.
fromsrcimportNestAnCoGenancogen=NestAnCoGen(ancogen=model, hifigan=generator, vqvae=vqvae, improved=False)
ModelLink
Speech-VQVAElink
HiFi-GANlink
AnCoGenlink / link (Improved)
Analysis
To do analysis with AnCoGen (link, test_analyse), which correspond to the estimation of the speech attributes from a Mel-spectrogram.
Please see the paper for a complete description of the attributes.
"""Test the analyse function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Pitch estimation + plottingaudio, attributes=ancogen.analyse(audio, apply_max=True, attribute_name="pitch", plot_bool=True)
Generation
To do speech analysis-resynthesis mapping wih AnCoGen (link, test_generation) which are simply obtained by using AnCoGen to map a Mel-spectrogram to the corresponding speech attributes (analysis stage) and then back to the Mel-spectrogram (generation stage).
"""Test the generation function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Generate outputgenerated=ancogen.generate(path=PATH_AUDIO, from_attributes=None, save_dir="wavs", return_metrics=True)
Control
To do analysis, transformation, and synthesis with AnCoGen, where the speech attributes are controlled between the analysis and generation stages in order to perform speech denoising (by increasing the SNR attribute), pitch shifting, dereverberation (by increasing the C50 attribute) or voice conversion (by controlling the speaker identity attribute).
"""Test the control function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Control with AnCoGenancogen.pitch_control(attributes, target_pitch, **kwargs) ancogen.content_control(attributes, target_content_index, **kwargs)
ancogen.snr_control(attributes, target_snr, **kwargs)
ancogen.c50_control(attributes, target_c50, **kwargs)
ancogen.voice_conversion(target_identity: str, source_signal: str, save_dir: str='')
License
GNU Affero General Public License (version 3), see LICENSE.txt.
About
[ICASSP 2025] AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
samsad35.github.io/site-ancogen/
Topics
audio-processinggenerative-modelmasked-autoencoderpitch-estimationpitch-shiftspeech-analysisspeech-synthesisvoice-conversion
Resources
Readme
AGPL-3.0 license
Activity
Stars
15 stars
Watchers
3 watching
Forks
1 fork
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
BranchesTags
Repository files navigation
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
Generic badgemade-with-pythonWebsite shields.ioPyPI version fury.io
VQ-VAE
This repository contains the code associated with the following publication:
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
 Samir Sadok, Simon Leglaive, Laurent Girin, Gaël Richard, Xavier Alameda-Pineda
 International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2025.
If you use this code for your research, please cite the above paper.
Useful links:
Setup
  •  Pypi:

    • pip install -i https://test.pypi.org/simple/ ancogen --no-deps
  •  Install the package locally (for use on your system):

    • In the current directory: pip install -e .
  •  Virtual Environment:

    • conda create -n ancogen python=3.9
    • conda activate ancogen
    • In the current directory: pip install -r requirements.txt
Usage
Pretrained models
After loading the weights of the pre-trained models: speechVQVAE, HIFIGAN and AnCoGen, put them all in the NestAnCoGen class.
fromsrcimportNestAnCoGenancogen=NestAnCoGen(ancogen=model, hifigan=generator, vqvae=vqvae, improved=False)
ModelLink
Speech-VQVAElink
HiFi-GANlink
AnCoGenlink / link (Improved)
Analysis
To do analysis with AnCoGen (link, test_analyse), which correspond to the estimation of the speech attributes from a Mel-spectrogram.
Please see the paper for a complete description of the attributes.
"""Test the analyse function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Pitch estimation + plottingaudio, attributes=ancogen.analyse(audio, apply_max=True, attribute_name="pitch", plot_bool=True)
Generation
To do speech analysis-resynthesis mapping wih AnCoGen (link, test_generation) which are simply obtained by using AnCoGen to map a Mel-spectrogram to the corresponding speech attributes (analysis stage) and then back to the Mel-spectrogram (generation stage).
"""Test the generation function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"# Generate outputgenerated=ancogen.generate(path=PATH_AUDIO, from_attributes=None, save_dir="wavs", return_metrics=True)
Control
To do analysis, transformation, and synthesis with AnCoGen, where the speech attributes are controlled between the analysis and generation stages in order to perform speech denoising (by increasing the SNR attribute), pitch shifting, dereverberation (by increasing the C50 attribute) or voice conversion (by controlling the speaker identity attribute).
"""Test the control function of AnCoGen. """PATH_AUDIO="path_wav_signal.wav"PATH_AUDIO="path_wav_signal.wav"# Preprocess the audioaudio=ancogen.preprocess(PATH_AUDIO)
# Analyse the audio with the AnCoGenaudio, attributes=ancogen.analyse(audio, apply_max=True)
# Control with AnCoGenancogen.pitch_control(attributes, target_pitch, **kwargs) ancogen.content_control(attributes, target_content_index, **kwargs)
ancogen.snr_control(attributes, target_snr, **kwargs)
ancogen.c50_control(attributes, target_c50, **kwargs)
ancogen.voice_conversion(target_identity: str, source_signal: str, save_dir: str='')
License
GNU Affero General Public License (version 3), see LICENSE.txt.
About
[ICASSP 2025] AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
samsad35.github.io/site-ancogen/
Topics
audio-processinggenerative-modelmasked-autoencoderpitch-estimationpitch-shiftspeech-analysisspeech-synthesisvoice-conversion
Resources
Readme
AGPL-3.0 license
Activity
Stars
15 stars
Watchers
3 watching
Forks
1 fork
Report repository
Releases
Packages
Used by
Contributors
Languages