BranchesTags

Repository files navigation

Decrecation notice: The author of Tapdance (AJ/Aaron Steers) has joined forces with Meltano to build the end-to-end DataOps platform of the future. Tapdance will remain free to use but I'm not actively developing it. See my blog post for more info on why I made this decision, and find me on Slack if you have questions about what we're building and how you can transition your data platform to Meltano.

Tapdance

Tapdance is an orchestration layer for the open source Singer tap platform.

What Makes Tapdance Different from other Singer and ETL Platforms

A bakers' dozen reasons to dance.

  1. Tapdance focuses on ease-of-use and is partially inspired by the ease-of-use of Pipelinewise.
  2. Tapdance aims to support all taps with no required plugin changes.
  3. Tapdance is dockerized at the core - we require that docker is installed, but we wrap it so you never have to run it directly.
  4. Tapdance has built-in IAC (Infrastructure-As-Code) using Terraform and near-zero infrastructure costs from always-on resources.
  5. Tapdance is opinionated in regards to ELT best practices.
  6. Tapdance supports DevOps best practices out of box, specifically: CI/CD and IAC.
  7. Tapdance is platform-agnostic and runs on Windows, Mac, and Linux alike using a docker-first approach.
  8. Tapdance plugins are curated - when multiple forks exist for a given plugin, we will curate the best we find and use those as the default. (See the latest list here.)
  9. Tapdance uses a data-lake-first approach - while it may be possible to load directly into an RDMS, we prioritize approaches where data lands first in the data lake before ingestion into a SQL DW.
  10. Tapdance is rules-based - instead of pointing and clicking a hundred times, simply tell tapdance what type of data you want (or what type of data you don't want).
  11. Tapdance knows your source schema is not static and it adapts automatically in response to upstream schema changes.
  12. Tapdance provides stream-isolation - data can be extracted (and retried) one-table-at-a-time, even if this is not a feature of the plugin.
  13. Tapdance automatically takes care of state - state files are managed automatically for you so that incremental data loads come for free with no additional effort.

Installation

Installation for Windows

  1. Install Chocolatey from an Administrative Command Prompt

    @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command " [System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))"&&SET"PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"

  2. Install Python3 (if not already installed)

    choco install -y python3

  3. Install Docker

    choco install -y docker-desktop

  4. Install Tapdance

    pip3 install tapdance

Installation for MacOS

  1. Install Homebrew

  2. Install Python (only if not installed)

    brew install python@3.8

  3. Install Docker

    brew install docker

  4. Install Tapdance

    pip3 install tapdance

Getting Started

Configuration

Tapdance looks for configuration information in 3 places:

  1. Environment Variables
  2. Rules Files
  3. Config Files

Command Line Execution

Once configuration is completed, run tapdance by executing the following two commands:

  1. tapdance plan {tap-id} - Runs discovery on the tap and creates a plan file.
    • The plan file shows which tables are columns will be included based upon your specified extraction rules.
  2. tapdance sync {tap-id} {target-id} - Syncs all data from tap to target, following extraction rules as documented in the previous step's plan file.

For more help, including a explanation of all optional parameters, run:

  • tapdance plan --help or
  • tapdance sync --help.

Tutorial

For step-by-step instructions, see the Tapdance tutorial.

About

Tapdance is an orchestration layer for the open source Singer tap platform.

Resources

Readme
MIT license
Activity
Custom properties

Stars

12 stars

Watchers

3 watching

Forks

0 forks
Report repository

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
BranchesTags
Repository files navigation
Decrecation notice: The author of Tapdance (AJ/Aaron Steers) has joined forces with Meltano to build the end-to-end DataOps platform of the future. Tapdance will remain free to use but I'm not actively developing it. See my blog post for more info on why I made this decision, and find me on Slack if you have questions about what we're building and how you can transition your data platform to Meltano.
Tapdance
Tapdance is an orchestration layer for the open source Singer tap platform.
What Makes Tapdance Different from other Singer and ETL Platforms
A bakers' dozen reasons to dance.
  1. Tapdance focuses on ease-of-use and is partially inspired by the ease-of-use of Pipelinewise.
  2. Tapdance aims to support all taps with no required plugin changes.
  3. Tapdance is dockerized at the core - we require that docker is installed, but we wrap it so you never have to run it directly.
  4. Tapdance has built-in IAC (Infrastructure-As-Code) using Terraform and near-zero infrastructure costs from always-on resources.
  5. Tapdance is opinionated in regards to ELT best practices.
  6. Tapdance supports DevOps best practices out of box, specifically: CI/CD and IAC.
  7. Tapdance is platform-agnostic and runs on Windows, Mac, and Linux alike using a docker-first approach.
  8. Tapdance plugins are curated - when multiple forks exist for a given plugin, we will curate the best we find and use those as the default. (See the latest list here.)
  9. Tapdance uses a data-lake-first approach - while it may be possible to load directly into an RDMS, we prioritize approaches where data lands first in the data lake before ingestion into a SQL DW.
  10. Tapdance is rules-based - instead of pointing and clicking a hundred times, simply tell tapdance what type of data you want (or what type of data you don't want).
  11. Tapdance knows your source schema is not static and it adapts automatically in response to upstream schema changes.
  12. Tapdance provides stream-isolation - data can be extracted (and retried) one-table-at-a-time, even if this is not a feature of the plugin.
  13. Tapdance automatically takes care of state - state files are managed automatically for you so that incremental data loads come for free with no additional effort.
Installation
Installation for Windows
  1. Install Chocolatey from an Administrative Command Prompt
    @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command " [System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))"&&SET"PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
  2. Install Python3 (if not already installed)
    choco install -y python3
  3. Install Docker
    choco install -y docker-desktop
  4. Install Tapdance
    pip3 install tapdance
Installation for MacOS
  1. Install Homebrew
  2. Install Python (only if not installed)
    brew install python@3.8
  3. Install Docker
    brew install docker
  4. Install Tapdance
    pip3 install tapdance
Getting Started
Configuration
Tapdance looks for configuration information in 3 places:
  1. Environment Variables
  2. Rules Files
  3. Config Files
Command Line Execution
Once configuration is completed, run tapdance by executing the following two commands:
  1. tapdance plan {tap-id} - Runs discovery on the tap and creates a plan file.

    • The plan file shows which tables are columns will be included based upon your
specified extraction rules.
  2. tapdance sync {tap-id} {target-id} - Syncs all data from tap to target, following
extraction rules as documented in the previous step's plan file.
For more help, including a explanation of all optional parameters, run:
  • tapdance plan --help or
  • tapdance sync --help.
Tutorial
For step-by-step instructions, see the Tapdance tutorial.
About
Tapdance is an orchestration layer for the open source Singer tap platform.
Resources
Readme
MIT license
Activity
Custom properties
Stars
12 stars
Watchers
3 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
Decrecation notice: The author of Tapdance (AJ/Aaron Steers) has joined forces with Meltano to build the end-to-end DataOps platform of the future. Tapdance will remain free to use but I'm not actively developing it. See my blog post for more info on why I made this decision, and find me on Slack if you have questions about what we're building and how you can transition your data platform to Meltano.
Tapdance
Tapdance is an orchestration layer for the open source Singer tap platform.
What Makes Tapdance Different from other Singer and ETL Platforms
A bakers' dozen reasons to dance.
  1. Tapdance focuses on ease-of-use and is partially inspired by the ease-of-use of Pipelinewise.
  2. Tapdance aims to support all taps with no required plugin changes.
  3. Tapdance is dockerized at the core - we require that docker is installed, but we wrap it so you never have to run it directly.
  4. Tapdance has built-in IAC (Infrastructure-As-Code) using Terraform and near-zero infrastructure costs from always-on resources.
  5. Tapdance is opinionated in regards to ELT best practices.
  6. Tapdance supports DevOps best practices out of box, specifically: CI/CD and IAC.
  7. Tapdance is platform-agnostic and runs on Windows, Mac, and Linux alike using a docker-first approach.
  8. Tapdance plugins are curated - when multiple forks exist for a given plugin, we will curate the best we find and use those as the default. (See the latest list here.)
  9. Tapdance uses a data-lake-first approach - while it may be possible to load directly into an RDMS, we prioritize approaches where data lands first in the data lake before ingestion into a SQL DW.
  10. Tapdance is rules-based - instead of pointing and clicking a hundred times, simply tell tapdance what type of data you want (or what type of data you don't want).
  11. Tapdance knows your source schema is not static and it adapts automatically in response to upstream schema changes.
  12. Tapdance provides stream-isolation - data can be extracted (and retried) one-table-at-a-time, even if this is not a feature of the plugin.
  13. Tapdance automatically takes care of state - state files are managed automatically for you so that incremental data loads come for free with no additional effort.
Installation
Installation for Windows
  1. Install Chocolatey from an Administrative Command Prompt
    @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command " [System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))"&&SET"PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
  2. Install Python3 (if not already installed)
    choco install -y python3
  3. Install Docker
    choco install -y docker-desktop
  4. Install Tapdance
    pip3 install tapdance
Installation for MacOS
  1. Install Homebrew
  2. Install Python (only if not installed)
    brew install python@3.8
  3. Install Docker
    brew install docker
  4. Install Tapdance
    pip3 install tapdance
Getting Started
Configuration
Tapdance looks for configuration information in 3 places:
  1. Environment Variables
  2. Rules Files
  3. Config Files
Command Line Execution
Once configuration is completed, run tapdance by executing the following two commands:
  1. tapdance plan {tap-id} - Runs discovery on the tap and creates a plan file.

    • The plan file shows which tables are columns will be included based upon your
specified extraction rules.
  2. tapdance sync {tap-id} {target-id} - Syncs all data from tap to target, following
extraction rules as documented in the previous step's plan file.
For more help, including a explanation of all optional parameters, run:
  • tapdance plan --help or
  • tapdance sync --help.
Tutorial
For step-by-step instructions, see the Tapdance tutorial.
About
Tapdance is an orchestration layer for the open source Singer tap platform.
Resources
Readme
MIT license
Activity
Custom properties
Stars
12 stars
Watchers
3 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
Decrecation notice: The author of Tapdance (AJ/Aaron Steers) has joined forces with Meltano to build the end-to-end DataOps platform of the future. Tapdance will remain free to use but I'm not actively developing it. See my blog post for more info on why I made this decision, and find me on Slack if you have questions about what we're building and how you can transition your data platform to Meltano.
Tapdance
Tapdance is an orchestration layer for the open source Singer tap platform.
What Makes Tapdance Different from other Singer and ETL Platforms
A bakers' dozen reasons to dance.
  1. Tapdance focuses on ease-of-use and is partially inspired by the ease-of-use of Pipelinewise.
  2. Tapdance aims to support all taps with no required plugin changes.
  3. Tapdance is dockerized at the core - we require that docker is installed, but we wrap it so you never have to run it directly.
  4. Tapdance has built-in IAC (Infrastructure-As-Code) using Terraform and near-zero infrastructure costs from always-on resources.
  5. Tapdance is opinionated in regards to ELT best practices.
  6. Tapdance supports DevOps best practices out of box, specifically: CI/CD and IAC.
  7. Tapdance is platform-agnostic and runs on Windows, Mac, and Linux alike using a docker-first approach.
  8. Tapdance plugins are curated - when multiple forks exist for a given plugin, we will curate the best we find and use those as the default. (See the latest list here.)
  9. Tapdance uses a data-lake-first approach - while it may be possible to load directly into an RDMS, we prioritize approaches where data lands first in the data lake before ingestion into a SQL DW.
  10. Tapdance is rules-based - instead of pointing and clicking a hundred times, simply tell tapdance what type of data you want (or what type of data you don't want).
  11. Tapdance knows your source schema is not static and it adapts automatically in response to upstream schema changes.
  12. Tapdance provides stream-isolation - data can be extracted (and retried) one-table-at-a-time, even if this is not a feature of the plugin.
  13. Tapdance automatically takes care of state - state files are managed automatically for you so that incremental data loads come for free with no additional effort.
Installation
Installation for Windows
  1. Install Chocolatey from an Administrative Command Prompt
    @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command " [System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))"&&SET"PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
  2. Install Python3 (if not already installed)
    choco install -y python3
  3. Install Docker
    choco install -y docker-desktop
  4. Install Tapdance
    pip3 install tapdance
Installation for MacOS
  1. Install Homebrew
  2. Install Python (only if not installed)
    brew install python@3.8
  3. Install Docker
    brew install docker
  4. Install Tapdance
    pip3 install tapdance
Getting Started
Configuration
Tapdance looks for configuration information in 3 places:
  1. Environment Variables
  2. Rules Files
  3. Config Files
Command Line Execution
Once configuration is completed, run tapdance by executing the following two commands:
  1. tapdance plan {tap-id} - Runs discovery on the tap and creates a plan file.

    • The plan file shows which tables are columns will be included based upon your
specified extraction rules.
  2. tapdance sync {tap-id} {target-id} - Syncs all data from tap to target, following
extraction rules as documented in the previous step's plan file.
For more help, including a explanation of all optional parameters, run:
  • tapdance plan --help or
  • tapdance sync --help.
Tutorial
For step-by-step instructions, see the Tapdance tutorial.
About
Tapdance is an orchestration layer for the open source Singer tap platform.
Resources
Readme
MIT license
Activity
Custom properties
Stars
12 stars
Watchers
3 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
BranchesTags
Repository files navigation
Decrecation notice: The author of Tapdance (AJ/Aaron Steers) has joined forces with Meltano to build the end-to-end DataOps platform of the future. Tapdance will remain free to use but I'm not actively developing it. See my blog post for more info on why I made this decision, and find me on Slack if you have questions about what we're building and how you can transition your data platform to Meltano.
Tapdance
Tapdance is an orchestration layer for the open source Singer tap platform.
What Makes Tapdance Different from other Singer and ETL Platforms
A bakers' dozen reasons to dance.
  1. Tapdance focuses on ease-of-use and is partially inspired by the ease-of-use of Pipelinewise.
  2. Tapdance aims to support all taps with no required plugin changes.
  3. Tapdance is dockerized at the core - we require that docker is installed, but we wrap it so you never have to run it directly.
  4. Tapdance has built-in IAC (Infrastructure-As-Code) using Terraform and near-zero infrastructure costs from always-on resources.
  5. Tapdance is opinionated in regards to ELT best practices.
  6. Tapdance supports DevOps best practices out of box, specifically: CI/CD and IAC.
  7. Tapdance is platform-agnostic and runs on Windows, Mac, and Linux alike using a docker-first approach.
  8. Tapdance plugins are curated - when multiple forks exist for a given plugin, we will curate the best we find and use those as the default. (See the latest list here.)
  9. Tapdance uses a data-lake-first approach - while it may be possible to load directly into an RDMS, we prioritize approaches where data lands first in the data lake before ingestion into a SQL DW.
  10. Tapdance is rules-based - instead of pointing and clicking a hundred times, simply tell tapdance what type of data you want (or what type of data you don't want).
  11. Tapdance knows your source schema is not static and it adapts automatically in response to upstream schema changes.
  12. Tapdance provides stream-isolation - data can be extracted (and retried) one-table-at-a-time, even if this is not a feature of the plugin.
  13. Tapdance automatically takes care of state - state files are managed automatically for you so that incremental data loads come for free with no additional effort.
Installation
Installation for Windows
  1. Install Chocolatey from an Administrative Command Prompt
    @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command " [System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))"&&SET"PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
  2. Install Python3 (if not already installed)
    choco install -y python3
  3. Install Docker
    choco install -y docker-desktop
  4. Install Tapdance
    pip3 install tapdance
Installation for MacOS
  1. Install Homebrew
  2. Install Python (only if not installed)
    brew install python@3.8
  3. Install Docker
    brew install docker
  4. Install Tapdance
    pip3 install tapdance
Getting Started
Configuration
Tapdance looks for configuration information in 3 places:
  1. Environment Variables
  2. Rules Files
  3. Config Files
Command Line Execution
Once configuration is completed, run tapdance by executing the following two commands:
  1. tapdance plan {tap-id} - Runs discovery on the tap and creates a plan file.

    • The plan file shows which tables are columns will be included based upon your
specified extraction rules.
  2. tapdance sync {tap-id} {target-id} - Syncs all data from tap to target, following
extraction rules as documented in the previous step's plan file.
For more help, including a explanation of all optional parameters, run:
  • tapdance plan --help or
  • tapdance sync --help.
Tutorial
For step-by-step instructions, see the Tapdance tutorial.
About
Tapdance is an orchestration layer for the open source Singer tap platform.
Resources
Readme
MIT license
Activity
Custom properties
Stars
12 stars
Watchers
3 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
Decrecation notice: The author of Tapdance (AJ/Aaron Steers) has joined forces with Meltano to build the end-to-end DataOps platform of the future. Tapdance will remain free to use but I'm not actively developing it. See my blog post for more info on why I made this decision, and find me on Slack if you have questions about what we're building and how you can transition your data platform to Meltano.
Tapdance
Tapdance is an orchestration layer for the open source Singer tap platform.
What Makes Tapdance Different from other Singer and ETL Platforms
A bakers' dozen reasons to dance.
  1. Tapdance focuses on ease-of-use and is partially inspired by the ease-of-use of Pipelinewise.
  2. Tapdance aims to support all taps with no required plugin changes.
  3. Tapdance is dockerized at the core - we require that docker is installed, but we wrap it so you never have to run it directly.
  4. Tapdance has built-in IAC (Infrastructure-As-Code) using Terraform and near-zero infrastructure costs from always-on resources.
  5. Tapdance is opinionated in regards to ELT best practices.
  6. Tapdance supports DevOps best practices out of box, specifically: CI/CD and IAC.
  7. Tapdance is platform-agnostic and runs on Windows, Mac, and Linux alike using a docker-first approach.
  8. Tapdance plugins are curated - when multiple forks exist for a given plugin, we will curate the best we find and use those as the default. (See the latest list here.)
  9. Tapdance uses a data-lake-first approach - while it may be possible to load directly into an RDMS, we prioritize approaches where data lands first in the data lake before ingestion into a SQL DW.
  10. Tapdance is rules-based - instead of pointing and clicking a hundred times, simply tell tapdance what type of data you want (or what type of data you don't want).
  11. Tapdance knows your source schema is not static and it adapts automatically in response to upstream schema changes.
  12. Tapdance provides stream-isolation - data can be extracted (and retried) one-table-at-a-time, even if this is not a feature of the plugin.
  13. Tapdance automatically takes care of state - state files are managed automatically for you so that incremental data loads come for free with no additional effort.
Installation
Installation for Windows
  1. Install Chocolatey from an Administrative Command Prompt
    @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command " [System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))"&&SET"PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
  2. Install Python3 (if not already installed)
    choco install -y python3
  3. Install Docker
    choco install -y docker-desktop
  4. Install Tapdance
    pip3 install tapdance
Installation for MacOS
  1. Install Homebrew
  2. Install Python (only if not installed)
    brew install python@3.8
  3. Install Docker
    brew install docker
  4. Install Tapdance
    pip3 install tapdance
Getting Started
Configuration
Tapdance looks for configuration information in 3 places:
  1. Environment Variables
  2. Rules Files
  3. Config Files
Command Line Execution
Once configuration is completed, run tapdance by executing the following two commands:
  1. tapdance plan {tap-id} - Runs discovery on the tap and creates a plan file.

    • The plan file shows which tables are columns will be included based upon your
specified extraction rules.
  2. tapdance sync {tap-id} {target-id} - Syncs all data from tap to target, following
extraction rules as documented in the previous step's plan file.
For more help, including a explanation of all optional parameters, run:
  • tapdance plan --help or
  • tapdance sync --help.
Tutorial
For step-by-step instructions, see the Tapdance tutorial.
About
Tapdance is an orchestration layer for the open source Singer tap platform.
Resources
Readme
MIT license
Activity
Custom properties
Stars
12 stars
Watchers
3 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
Decrecation notice: The author of Tapdance (AJ/Aaron Steers) has joined forces with Meltano to build the end-to-end DataOps platform of the future. Tapdance will remain free to use but I'm not actively developing it. See my blog post for more info on why I made this decision, and find me on Slack if you have questions about what we're building and how you can transition your data platform to Meltano.
Tapdance
Tapdance is an orchestration layer for the open source Singer tap platform.
What Makes Tapdance Different from other Singer and ETL Platforms
A bakers' dozen reasons to dance.
  1. Tapdance focuses on ease-of-use and is partially inspired by the ease-of-use of Pipelinewise.
  2. Tapdance aims to support all taps with no required plugin changes.
  3. Tapdance is dockerized at the core - we require that docker is installed, but we wrap it so you never have to run it directly.
  4. Tapdance has built-in IAC (Infrastructure-As-Code) using Terraform and near-zero infrastructure costs from always-on resources.
  5. Tapdance is opinionated in regards to ELT best practices.
  6. Tapdance supports DevOps best practices out of box, specifically: CI/CD and IAC.
  7. Tapdance is platform-agnostic and runs on Windows, Mac, and Linux alike using a docker-first approach.
  8. Tapdance plugins are curated - when multiple forks exist for a given plugin, we will curate the best we find and use those as the default. (See the latest list here.)
  9. Tapdance uses a data-lake-first approach - while it may be possible to load directly into an RDMS, we prioritize approaches where data lands first in the data lake before ingestion into a SQL DW.
  10. Tapdance is rules-based - instead of pointing and clicking a hundred times, simply tell tapdance what type of data you want (or what type of data you don't want).
  11. Tapdance knows your source schema is not static and it adapts automatically in response to upstream schema changes.
  12. Tapdance provides stream-isolation - data can be extracted (and retried) one-table-at-a-time, even if this is not a feature of the plugin.
  13. Tapdance automatically takes care of state - state files are managed automatically for you so that incremental data loads come for free with no additional effort.
Installation
Installation for Windows
  1. Install Chocolatey from an Administrative Command Prompt
    @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command " [System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))"&&SET"PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
  2. Install Python3 (if not already installed)
    choco install -y python3
  3. Install Docker
    choco install -y docker-desktop
  4. Install Tapdance
    pip3 install tapdance
Installation for MacOS
  1. Install Homebrew
  2. Install Python (only if not installed)
    brew install python@3.8
  3. Install Docker
    brew install docker
  4. Install Tapdance
    pip3 install tapdance
Getting Started
Configuration
Tapdance looks for configuration information in 3 places:
  1. Environment Variables
  2. Rules Files
  3. Config Files
Command Line Execution
Once configuration is completed, run tapdance by executing the following two commands:
  1. tapdance plan {tap-id} - Runs discovery on the tap and creates a plan file.

    • The plan file shows which tables are columns will be included based upon your
specified extraction rules.
  2. tapdance sync {tap-id} {target-id} - Syncs all data from tap to target, following
extraction rules as documented in the previous step's plan file.
For more help, including a explanation of all optional parameters, run:
  • tapdance plan --help or
  • tapdance sync --help.
Tutorial
For step-by-step instructions, see the Tapdance tutorial.
About
Tapdance is an orchestration layer for the open source Singer tap platform.
Resources
Readme
MIT license
Activity
Custom properties
Stars
12 stars
Watchers
3 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
BranchesTags
Repository files navigation
Decrecation notice: The author of Tapdance (AJ/Aaron Steers) has joined forces with Meltano to build the end-to-end DataOps platform of the future. Tapdance will remain free to use but I'm not actively developing it. See my blog post for more info on why I made this decision, and find me on Slack if you have questions about what we're building and how you can transition your data platform to Meltano.
Tapdance
Tapdance is an orchestration layer for the open source Singer tap platform.
What Makes Tapdance Different from other Singer and ETL Platforms
A bakers' dozen reasons to dance.
  1. Tapdance focuses on ease-of-use and is partially inspired by the ease-of-use of Pipelinewise.
  2. Tapdance aims to support all taps with no required plugin changes.
  3. Tapdance is dockerized at the core - we require that docker is installed, but we wrap it so you never have to run it directly.
  4. Tapdance has built-in IAC (Infrastructure-As-Code) using Terraform and near-zero infrastructure costs from always-on resources.
  5. Tapdance is opinionated in regards to ELT best practices.
  6. Tapdance supports DevOps best practices out of box, specifically: CI/CD and IAC.
  7. Tapdance is platform-agnostic and runs on Windows, Mac, and Linux alike using a docker-first approach.
  8. Tapdance plugins are curated - when multiple forks exist for a given plugin, we will curate the best we find and use those as the default. (See the latest list here.)
  9. Tapdance uses a data-lake-first approach - while it may be possible to load directly into an RDMS, we prioritize approaches where data lands first in the data lake before ingestion into a SQL DW.
  10. Tapdance is rules-based - instead of pointing and clicking a hundred times, simply tell tapdance what type of data you want (or what type of data you don't want).
  11. Tapdance knows your source schema is not static and it adapts automatically in response to upstream schema changes.
  12. Tapdance provides stream-isolation - data can be extracted (and retried) one-table-at-a-time, even if this is not a feature of the plugin.
  13. Tapdance automatically takes care of state - state files are managed automatically for you so that incremental data loads come for free with no additional effort.
Installation
Installation for Windows
  1. Install Chocolatey from an Administrative Command Prompt
    @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command " [System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))"&&SET"PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
  2. Install Python3 (if not already installed)
    choco install -y python3
  3. Install Docker
    choco install -y docker-desktop
  4. Install Tapdance
    pip3 install tapdance
Installation for MacOS
  1. Install Homebrew
  2. Install Python (only if not installed)
    brew install python@3.8
  3. Install Docker
    brew install docker
  4. Install Tapdance
    pip3 install tapdance
Getting Started
Configuration
Tapdance looks for configuration information in 3 places:
  1. Environment Variables
  2. Rules Files
  3. Config Files
Command Line Execution
Once configuration is completed, run tapdance by executing the following two commands:
  1. tapdance plan {tap-id} - Runs discovery on the tap and creates a plan file.

    • The plan file shows which tables are columns will be included based upon your
specified extraction rules.
  2. tapdance sync {tap-id} {target-id} - Syncs all data from tap to target, following
extraction rules as documented in the previous step's plan file.
For more help, including a explanation of all optional parameters, run:
  • tapdance plan --help or
  • tapdance sync --help.
Tutorial
For step-by-step instructions, see the Tapdance tutorial.
About
Tapdance is an orchestration layer for the open source Singer tap platform.
Resources
Readme
MIT license
Activity
Custom properties
Stars
12 stars
Watchers
3 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages