BranchesTags

Repository files navigation

Experiments with distributed counters

This project implements simple distributed counters in Erlang. Counters are distributed values with custom merge policies. A merge policy is a function that takes two existing counters and returns a new one. In this context, counters are not just values that are incremented by an arbitrary amount but any value that might be wrapped in a container and merged with an other.

Several functions can be implemented that way, including:

  • SUM (implemented in ctr_sum.erl)
  • MIN (implemented in ctr_min.erl)
  • MAX (implemented in ctr_max.erl)
  • AVG (implemented in ctr_avg.erl)
  • STDDEV
  • Count-Min
  • (Hyper)LogLog
  • …

Counter definition

An Erlang “behaviour” is used to implement a few functions per counter, namely:

  • bottom/0 returns a new, empty counter (think ⊥).
  • new/1 returns a new counter containing a single value.
  • merge/2 takes two counters and returns a new one.
  • value/1 extracts the value from a counter.
  • is_idempotent/0true/false depending on the counter; MIN is idempotent, SUM is not.
  • gc_info/1 return a data structure used in GC.
  • gc_merge/3 gives the GC data structure to an existing counter with a unique identifier and ask the counter to clean up old (and presumably irrelevant) data.

Idempotent counter types have trivial implementations for gc_info/1 and gc_merge/3.

Distribution

The demo program starts 3 nodes as separate Erlang processes and sends them a few thousand counter updates with a high probability of dropped and duplicate messages. Once all the updates have been sent, each node is asked to give its own opinion about the total count and a merged value of these 3 values is displayed. If needed, a GC process is run to update the nodes.

Running the demo

Run make clean all demo to run the demo. Here is what you should see:

==============================
Testing ctr_sum on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -110836610
Gettting full counters on all nodes...
Value of resolved counters: -110836610
Counter according to each node: [-109582605,-110537717,-112440005]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-110836610,-110836610,-110836610]
==============================
Testing ctr_min on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -999999
Gettting full counters on all nodes...
Value of resolved counters: -999999
Counter according to each node: [-999999,-999999,-999999]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-999999,-999999,-999999]
==============================
Testing ctr_max on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is 999650
Gettting full counters on all nodes...
Value of resolved counters: 999650
Counter according to each node: [999650,999650,999650]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [999650,999650,999650]
==============================
Testing ctr_avg on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is {-99168540,10000}
Gettting full counters on all nodes...
Value of resolved counters: {-99168540,10000}
Counter according to each node: [{-100403932,9985},{-97717578,9991},{-101242460,9984}]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [{-99168540,10000},{-99168540,10000},{-99168540,10000}]

Limitations

This code is quite inefficient and only intended as an experiment. The GcInfo object should use a more compact representation; Merkle Trees would probably help.

License

Released in the Public Domain. Originally developed as an internal prototype at Acunu.

About

Experiments with distributed counters

Resources

Readme
Activity

Stars

6 stars

Watchers

1 watching

Forks

1 fork
Report repository

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
BranchesTags
Repository files navigation
Experiments with distributed counters
This project implements simple distributed counters in Erlang.
Counters are distributed values with custom merge policies. A merge policy is a function that takes two existing counters and returns a new one.
In this context, counters are not just values that are incremented by an arbitrary amount but any value that might be wrapped in a container and merged with an other.
Several functions can be implemented that way, including:
  • SUM (implemented in ctr_sum.erl)
  • MIN (implemented in ctr_min.erl)
  • MAX (implemented in ctr_max.erl)
  • AVG (implemented in ctr_avg.erl)
  • STDDEV
  • Count-Min
  • (Hyper)LogLog
  • …
Counter definition
An Erlang “behaviour” is used to implement a few functions per counter, namely:
  • bottom/0 returns a new, empty counter (think ⊥).
  • new/1 returns a new counter containing a single value.
  • merge/2 takes two counters and returns a new one.
  • value/1 extracts the value from a counter.
  • is_idempotent/0true/false depending on the counter; MIN is idempotent, SUM is not.
  • gc_info/1 return a data structure used in GC.
  • gc_merge/3 gives the GC data structure to an existing counter with a unique identifier and ask the counter to clean up old (and presumably irrelevant) data.
Idempotent counter types have trivial implementations for gc_info/1 and gc_merge/3.
Distribution
The demo program starts 3 nodes as separate Erlang processes and sends them a few thousand counter updates with a high probability of dropped and duplicate messages. Once all the updates have been sent, each node is asked to give its own opinion about the total count and a merged value of these 3 values is displayed. If needed, a GC process is run to update the nodes.
Running the demo
Run make clean all demo to run the demo.
Here is what you should see:
==============================
Testing ctr_sum on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -110836610
Gettting full counters on all nodes...
Value of resolved counters: -110836610
Counter according to each node: [-109582605,-110537717,-112440005]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-110836610,-110836610,-110836610]
==============================
Testing ctr_min on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -999999
Gettting full counters on all nodes...
Value of resolved counters: -999999
Counter according to each node: [-999999,-999999,-999999]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-999999,-999999,-999999]
==============================
Testing ctr_max on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is 999650
Gettting full counters on all nodes...
Value of resolved counters: 999650
Counter according to each node: [999650,999650,999650]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [999650,999650,999650]
==============================
Testing ctr_avg on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is {-99168540,10000}
Gettting full counters on all nodes...
Value of resolved counters: {-99168540,10000}
Counter according to each node: [{-100403932,9985},{-97717578,9991},{-101242460,9984}]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [{-99168540,10000},{-99168540,10000},{-99168540,10000}]

Limitations
This code is quite inefficient and only intended as an experiment.
The GcInfo object should use a more compact representation; Merkle Trees would probably help.
License
Released in the Public Domain.
Originally developed as an internal prototype at Acunu.
About
Experiments with distributed counters
Resources
Readme
Activity
Stars
6 stars
Watchers
1 watching
Forks
1 fork
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
Experiments with distributed counters
This project implements simple distributed counters in Erlang.
Counters are distributed values with custom merge policies. A merge policy is a function that takes two existing counters and returns a new one.
In this context, counters are not just values that are incremented by an arbitrary amount but any value that might be wrapped in a container and merged with an other.
Several functions can be implemented that way, including:
  • SUM (implemented in ctr_sum.erl)
  • MIN (implemented in ctr_min.erl)
  • MAX (implemented in ctr_max.erl)
  • AVG (implemented in ctr_avg.erl)
  • STDDEV
  • Count-Min
  • (Hyper)LogLog
  • …
Counter definition
An Erlang “behaviour” is used to implement a few functions per counter, namely:
  • bottom/0 returns a new, empty counter (think ⊥).
  • new/1 returns a new counter containing a single value.
  • merge/2 takes two counters and returns a new one.
  • value/1 extracts the value from a counter.
  • is_idempotent/0true/false depending on the counter; MIN is idempotent, SUM is not.
  • gc_info/1 return a data structure used in GC.
  • gc_merge/3 gives the GC data structure to an existing counter with a unique identifier and ask the counter to clean up old (and presumably irrelevant) data.
Idempotent counter types have trivial implementations for gc_info/1 and gc_merge/3.
Distribution
The demo program starts 3 nodes as separate Erlang processes and sends them a few thousand counter updates with a high probability of dropped and duplicate messages. Once all the updates have been sent, each node is asked to give its own opinion about the total count and a merged value of these 3 values is displayed. If needed, a GC process is run to update the nodes.
Running the demo
Run make clean all demo to run the demo.
Here is what you should see:
==============================
Testing ctr_sum on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -110836610
Gettting full counters on all nodes...
Value of resolved counters: -110836610
Counter according to each node: [-109582605,-110537717,-112440005]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-110836610,-110836610,-110836610]
==============================
Testing ctr_min on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -999999
Gettting full counters on all nodes...
Value of resolved counters: -999999
Counter according to each node: [-999999,-999999,-999999]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-999999,-999999,-999999]
==============================
Testing ctr_max on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is 999650
Gettting full counters on all nodes...
Value of resolved counters: 999650
Counter according to each node: [999650,999650,999650]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [999650,999650,999650]
==============================
Testing ctr_avg on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is {-99168540,10000}
Gettting full counters on all nodes...
Value of resolved counters: {-99168540,10000}
Counter according to each node: [{-100403932,9985},{-97717578,9991},{-101242460,9984}]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [{-99168540,10000},{-99168540,10000},{-99168540,10000}]

Limitations
This code is quite inefficient and only intended as an experiment.
The GcInfo object should use a more compact representation; Merkle Trees would probably help.
License
Released in the Public Domain.
Originally developed as an internal prototype at Acunu.
About
Experiments with distributed counters
Resources
Readme
Activity
Stars
6 stars
Watchers
1 watching
Forks
1 fork
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
Experiments with distributed counters
This project implements simple distributed counters in Erlang.
Counters are distributed values with custom merge policies. A merge policy is a function that takes two existing counters and returns a new one.
In this context, counters are not just values that are incremented by an arbitrary amount but any value that might be wrapped in a container and merged with an other.
Several functions can be implemented that way, including:
  • SUM (implemented in ctr_sum.erl)
  • MIN (implemented in ctr_min.erl)
  • MAX (implemented in ctr_max.erl)
  • AVG (implemented in ctr_avg.erl)
  • STDDEV
  • Count-Min
  • (Hyper)LogLog
  • …
Counter definition
An Erlang “behaviour” is used to implement a few functions per counter, namely:
  • bottom/0 returns a new, empty counter (think ⊥).
  • new/1 returns a new counter containing a single value.
  • merge/2 takes two counters and returns a new one.
  • value/1 extracts the value from a counter.
  • is_idempotent/0true/false depending on the counter; MIN is idempotent, SUM is not.
  • gc_info/1 return a data structure used in GC.
  • gc_merge/3 gives the GC data structure to an existing counter with a unique identifier and ask the counter to clean up old (and presumably irrelevant) data.
Idempotent counter types have trivial implementations for gc_info/1 and gc_merge/3.
Distribution
The demo program starts 3 nodes as separate Erlang processes and sends them a few thousand counter updates with a high probability of dropped and duplicate messages. Once all the updates have been sent, each node is asked to give its own opinion about the total count and a merged value of these 3 values is displayed. If needed, a GC process is run to update the nodes.
Running the demo
Run make clean all demo to run the demo.
Here is what you should see:
==============================
Testing ctr_sum on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -110836610
Gettting full counters on all nodes...
Value of resolved counters: -110836610
Counter according to each node: [-109582605,-110537717,-112440005]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-110836610,-110836610,-110836610]
==============================
Testing ctr_min on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -999999
Gettting full counters on all nodes...
Value of resolved counters: -999999
Counter according to each node: [-999999,-999999,-999999]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-999999,-999999,-999999]
==============================
Testing ctr_max on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is 999650
Gettting full counters on all nodes...
Value of resolved counters: 999650
Counter according to each node: [999650,999650,999650]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [999650,999650,999650]
==============================
Testing ctr_avg on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is {-99168540,10000}
Gettting full counters on all nodes...
Value of resolved counters: {-99168540,10000}
Counter according to each node: [{-100403932,9985},{-97717578,9991},{-101242460,9984}]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [{-99168540,10000},{-99168540,10000},{-99168540,10000}]

Limitations
This code is quite inefficient and only intended as an experiment.
The GcInfo object should use a more compact representation; Merkle Trees would probably help.
License
Released in the Public Domain.
Originally developed as an internal prototype at Acunu.
About
Experiments with distributed counters
Resources
Readme
Activity
Stars
6 stars
Watchers
1 watching
Forks
1 fork
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
BranchesTags
Repository files navigation
Experiments with distributed counters
This project implements simple distributed counters in Erlang.
Counters are distributed values with custom merge policies. A merge policy is a function that takes two existing counters and returns a new one.
In this context, counters are not just values that are incremented by an arbitrary amount but any value that might be wrapped in a container and merged with an other.
Several functions can be implemented that way, including:
  • SUM (implemented in ctr_sum.erl)
  • MIN (implemented in ctr_min.erl)
  • MAX (implemented in ctr_max.erl)
  • AVG (implemented in ctr_avg.erl)
  • STDDEV
  • Count-Min
  • (Hyper)LogLog
  • …
Counter definition
An Erlang “behaviour” is used to implement a few functions per counter, namely:
  • bottom/0 returns a new, empty counter (think ⊥).
  • new/1 returns a new counter containing a single value.
  • merge/2 takes two counters and returns a new one.
  • value/1 extracts the value from a counter.
  • is_idempotent/0true/false depending on the counter; MIN is idempotent, SUM is not.
  • gc_info/1 return a data structure used in GC.
  • gc_merge/3 gives the GC data structure to an existing counter with a unique identifier and ask the counter to clean up old (and presumably irrelevant) data.
Idempotent counter types have trivial implementations for gc_info/1 and gc_merge/3.
Distribution
The demo program starts 3 nodes as separate Erlang processes and sends them a few thousand counter updates with a high probability of dropped and duplicate messages. Once all the updates have been sent, each node is asked to give its own opinion about the total count and a merged value of these 3 values is displayed. If needed, a GC process is run to update the nodes.
Running the demo
Run make clean all demo to run the demo.
Here is what you should see:
==============================
Testing ctr_sum on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -110836610
Gettting full counters on all nodes...
Value of resolved counters: -110836610
Counter according to each node: [-109582605,-110537717,-112440005]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-110836610,-110836610,-110836610]
==============================
Testing ctr_min on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -999999
Gettting full counters on all nodes...
Value of resolved counters: -999999
Counter according to each node: [-999999,-999999,-999999]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-999999,-999999,-999999]
==============================
Testing ctr_max on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is 999650
Gettting full counters on all nodes...
Value of resolved counters: 999650
Counter according to each node: [999650,999650,999650]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [999650,999650,999650]
==============================
Testing ctr_avg on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is {-99168540,10000}
Gettting full counters on all nodes...
Value of resolved counters: {-99168540,10000}
Counter according to each node: [{-100403932,9985},{-97717578,9991},{-101242460,9984}]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [{-99168540,10000},{-99168540,10000},{-99168540,10000}]

Limitations
This code is quite inefficient and only intended as an experiment.
The GcInfo object should use a more compact representation; Merkle Trees would probably help.
License
Released in the Public Domain.
Originally developed as an internal prototype at Acunu.
About
Experiments with distributed counters
Resources
Readme
Activity
Stars
6 stars
Watchers
1 watching
Forks
1 fork
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
Experiments with distributed counters
This project implements simple distributed counters in Erlang.
Counters are distributed values with custom merge policies. A merge policy is a function that takes two existing counters and returns a new one.
In this context, counters are not just values that are incremented by an arbitrary amount but any value that might be wrapped in a container and merged with an other.
Several functions can be implemented that way, including:
  • SUM (implemented in ctr_sum.erl)
  • MIN (implemented in ctr_min.erl)
  • MAX (implemented in ctr_max.erl)
  • AVG (implemented in ctr_avg.erl)
  • STDDEV
  • Count-Min
  • (Hyper)LogLog
  • …
Counter definition
An Erlang “behaviour” is used to implement a few functions per counter, namely:
  • bottom/0 returns a new, empty counter (think ⊥).
  • new/1 returns a new counter containing a single value.
  • merge/2 takes two counters and returns a new one.
  • value/1 extracts the value from a counter.
  • is_idempotent/0true/false depending on the counter; MIN is idempotent, SUM is not.
  • gc_info/1 return a data structure used in GC.
  • gc_merge/3 gives the GC data structure to an existing counter with a unique identifier and ask the counter to clean up old (and presumably irrelevant) data.
Idempotent counter types have trivial implementations for gc_info/1 and gc_merge/3.
Distribution
The demo program starts 3 nodes as separate Erlang processes and sends them a few thousand counter updates with a high probability of dropped and duplicate messages. Once all the updates have been sent, each node is asked to give its own opinion about the total count and a merged value of these 3 values is displayed. If needed, a GC process is run to update the nodes.
Running the demo
Run make clean all demo to run the demo.
Here is what you should see:
==============================
Testing ctr_sum on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -110836610
Gettting full counters on all nodes...
Value of resolved counters: -110836610
Counter according to each node: [-109582605,-110537717,-112440005]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-110836610,-110836610,-110836610]
==============================
Testing ctr_min on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -999999
Gettting full counters on all nodes...
Value of resolved counters: -999999
Counter according to each node: [-999999,-999999,-999999]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-999999,-999999,-999999]
==============================
Testing ctr_max on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is 999650
Gettting full counters on all nodes...
Value of resolved counters: 999650
Counter according to each node: [999650,999650,999650]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [999650,999650,999650]
==============================
Testing ctr_avg on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is {-99168540,10000}
Gettting full counters on all nodes...
Value of resolved counters: {-99168540,10000}
Counter according to each node: [{-100403932,9985},{-97717578,9991},{-101242460,9984}]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [{-99168540,10000},{-99168540,10000},{-99168540,10000}]

Limitations
This code is quite inefficient and only intended as an experiment.
The GcInfo object should use a more compact representation; Merkle Trees would probably help.
License
Released in the Public Domain.
Originally developed as an internal prototype at Acunu.
About
Experiments with distributed counters
Resources
Readme
Activity
Stars
6 stars
Watchers
1 watching
Forks
1 fork
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
Experiments with distributed counters
This project implements simple distributed counters in Erlang.
Counters are distributed values with custom merge policies. A merge policy is a function that takes two existing counters and returns a new one.
In this context, counters are not just values that are incremented by an arbitrary amount but any value that might be wrapped in a container and merged with an other.
Several functions can be implemented that way, including:
  • SUM (implemented in ctr_sum.erl)
  • MIN (implemented in ctr_min.erl)
  • MAX (implemented in ctr_max.erl)
  • AVG (implemented in ctr_avg.erl)
  • STDDEV
  • Count-Min
  • (Hyper)LogLog
  • …
Counter definition
An Erlang “behaviour” is used to implement a few functions per counter, namely:
  • bottom/0 returns a new, empty counter (think ⊥).
  • new/1 returns a new counter containing a single value.
  • merge/2 takes two counters and returns a new one.
  • value/1 extracts the value from a counter.
  • is_idempotent/0true/false depending on the counter; MIN is idempotent, SUM is not.
  • gc_info/1 return a data structure used in GC.
  • gc_merge/3 gives the GC data structure to an existing counter with a unique identifier and ask the counter to clean up old (and presumably irrelevant) data.
Idempotent counter types have trivial implementations for gc_info/1 and gc_merge/3.
Distribution
The demo program starts 3 nodes as separate Erlang processes and sends them a few thousand counter updates with a high probability of dropped and duplicate messages. Once all the updates have been sent, each node is asked to give its own opinion about the total count and a merged value of these 3 values is displayed. If needed, a GC process is run to update the nodes.
Running the demo
Run make clean all demo to run the demo.
Here is what you should see:
==============================
Testing ctr_sum on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -110836610
Gettting full counters on all nodes...
Value of resolved counters: -110836610
Counter according to each node: [-109582605,-110537717,-112440005]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-110836610,-110836610,-110836610]
==============================
Testing ctr_min on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -999999
Gettting full counters on all nodes...
Value of resolved counters: -999999
Counter according to each node: [-999999,-999999,-999999]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-999999,-999999,-999999]
==============================
Testing ctr_max on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is 999650
Gettting full counters on all nodes...
Value of resolved counters: 999650
Counter according to each node: [999650,999650,999650]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [999650,999650,999650]
==============================
Testing ctr_avg on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is {-99168540,10000}
Gettting full counters on all nodes...
Value of resolved counters: {-99168540,10000}
Counter according to each node: [{-100403932,9985},{-97717578,9991},{-101242460,9984}]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [{-99168540,10000},{-99168540,10000},{-99168540,10000}]

Limitations
This code is quite inefficient and only intended as an experiment.
The GcInfo object should use a more compact representation; Merkle Trees would probably help.
License
Released in the Public Domain.
Originally developed as an internal prototype at Acunu.
About
Experiments with distributed counters
Resources
Readme
Activity
Stars
6 stars
Watchers
1 watching
Forks
1 fork
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
BranchesTags
Repository files navigation
Experiments with distributed counters
This project implements simple distributed counters in Erlang.
Counters are distributed values with custom merge policies. A merge policy is a function that takes two existing counters and returns a new one.
In this context, counters are not just values that are incremented by an arbitrary amount but any value that might be wrapped in a container and merged with an other.
Several functions can be implemented that way, including:
  • SUM (implemented in ctr_sum.erl)
  • MIN (implemented in ctr_min.erl)
  • MAX (implemented in ctr_max.erl)
  • AVG (implemented in ctr_avg.erl)
  • STDDEV
  • Count-Min
  • (Hyper)LogLog
  • …
Counter definition
An Erlang “behaviour” is used to implement a few functions per counter, namely:
  • bottom/0 returns a new, empty counter (think ⊥).
  • new/1 returns a new counter containing a single value.
  • merge/2 takes two counters and returns a new one.
  • value/1 extracts the value from a counter.
  • is_idempotent/0true/false depending on the counter; MIN is idempotent, SUM is not.
  • gc_info/1 return a data structure used in GC.
  • gc_merge/3 gives the GC data structure to an existing counter with a unique identifier and ask the counter to clean up old (and presumably irrelevant) data.
Idempotent counter types have trivial implementations for gc_info/1 and gc_merge/3.
Distribution
The demo program starts 3 nodes as separate Erlang processes and sends them a few thousand counter updates with a high probability of dropped and duplicate messages. Once all the updates have been sent, each node is asked to give its own opinion about the total count and a merged value of these 3 values is displayed. If needed, a GC process is run to update the nodes.
Running the demo
Run make clean all demo to run the demo.
Here is what you should see:
==============================
Testing ctr_sum on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -110836610
Gettting full counters on all nodes...
Value of resolved counters: -110836610
Counter according to each node: [-109582605,-110537717,-112440005]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-110836610,-110836610,-110836610]
==============================
Testing ctr_min on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is -999999
Gettting full counters on all nodes...
Value of resolved counters: -999999
Counter according to each node: [-999999,-999999,-999999]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [-999999,-999999,-999999]
==============================
Testing ctr_max on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is 999650
Gettting full counters on all nodes...
Value of resolved counters: 999650
Counter according to each node: [999650,999650,999650]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [999650,999650,999650]
==============================
Testing ctr_avg on 3 nodes
Sent 1000 messages
Sent 2000 messages
Sent 3000 messages
Sent 4000 messages
Sent 5000 messages
Sent 6000 messages
Sent 7000 messages
Sent 8000 messages
Sent 9000 messages
Sent 10000 messages
Expected value is {-99168540,10000}
Gettting full counters on all nodes...
Value of resolved counters: {-99168540,10000}
Counter according to each node: [{-100403932,9985},{-97717578,9991},{-101242460,9984}]
Waiting a second before running GC...
Trigger GC
Counter according to each node: [{-99168540,10000},{-99168540,10000},{-99168540,10000}]

Limitations
This code is quite inefficient and only intended as an experiment.
The GcInfo object should use a more compact representation; Merkle Trees would probably help.
License
Released in the Public Domain.
Originally developed as an internal prototype at Acunu.
About
Experiments with distributed counters
Resources
Readme
Activity
Stars
6 stars
Watchers
1 watching
Forks
1 fork
Report repository
Releases
Packages
Contributors
Languages