fix(cache): match client-go's reconnect backoff algorithm - #3026

Merged
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom
davidgamero:fix/watch-backoff-jitter
Sep 1, 2026
Merged

fix(cache): match client-go's reconnect backoff algorithm#3026
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom
davidgamero:fix/watch-backoff-jitter

Conversation

@davidgamero

@davidgamerodavidgamero commented Aug 30, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor
  • Swapped the backoff constants for client-go's defaultBackoff*: 800ms base, ×2, 30s cap, full jitter, 2min reset. 30s cap is unchanged.
  • Added full jitter — delays now land in [level, 2*level),
  • Added a 2-minute idle reset, so a recovered client stops paying for old failures. New lastBackoffAt field.
  • Added nowFn/randFn injection for deterministic tests; updated 3 existing tests to 800/1600/3200, added 2 for jitter spread and cap-then-reset

followup on #3022

ai disclosure: used opencode + claude

@kubernetes-prowkubernetes-prowBot added approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 30, 2026
@davidgamero
davidgamero marked this pull request as draft August 30, 2026 02:42
@kubernetes-prowkubernetes-prowBot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 30, 2026
@davidgamero

davidgamero commented Aug 30, 2026

Copy link
Copy Markdown
ContributorAuthor

/hold waiting for CI fix

@kubernetes-prowkubernetes-prowBot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 30, 2026
@mstruebing

mstruebing commented Aug 31, 2026

Copy link
Copy Markdown
Member

@davidgamero CI should have been landed.

@davidgamero
fix(cache): match client-go's reconnect backoff algorithm
8f23ee7
The reconnect delay doubled from a 1s base with no jitter, so every
client disconnected by the same event (an apiserver rollout, a load
balancer cycling) retried at the same instants and kept colliding.
Adopt client-go's defaultBackoff* values from the reflector's
ExponentialBackoffManager: an 800ms base, a factor of 2, a 30s cap, full
jitter, and a reset once the backoff has gone unused for two minutes.
Full jitter spreads each retry over [level, 2*level), which is what
breaks up the lockstep, and the idle reset means a client that recovered
stops paying for failures it has already outlived.
Only the delay computation changes. When a reconnect backs off at all is
untouched: the first reconnect is still free, and receiving a watch
event or a TimeoutError still clears the delay.
nowFn and randFn join delayFn as injectable options so the backoff can
be asserted deterministically instead of by sleeping.
@davidgamero
davidgameroforce-pushed the fix/watch-backoff-jitter branch from 925a4f2 to 8f23ee7CompareAugust 31, 2026 03:41
@brendandburns

brendandburns commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

/lgtm
/approve

@kubernetes-prowkubernetes-prowBot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 31, 2026
@brendandburns
brendandburns marked this pull request as ready for review August 31, 2026 18:01
@kubernetes-prowkubernetes-prowBot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 31, 2026
@kubernetes-prow

kubernetes-prowBot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: brendandburns, davidgamero

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [brendandburns,davidgamero]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@davidgamero

davidgamero commented Sep 1, 2026

Copy link
Copy Markdown
ContributorAuthor

/unhold

@kubernetes-prowkubernetes-prowBot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 1, 2026
@kubernetes-prow
kubernetes-prowBot merged commit 202ce1b into kubernetes-client:mainSep 1, 2026
10 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@brendandburnsbrendandburnsbrendandburns approved these changes
@cjihrigcjihrigAwaiting requested review from cjihrig
@mstruebingmstruebingAwaiting requested review from mstruebing

Assignees

@brendandburnsbrendandburns

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.cncf-cla: yesIndicates the PR's author has signed the CNCF CLA.lgtm"Looks good to me", indicates that a PR is ready to be merged.size/LDenotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@davidgamero@mstruebing@brendandburns
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
fix(cache): match client-go's reconnect backoff algorithm - #3026
Merged
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom 
davidgamero:fix/watch-backoff-jitter
Sep 1, 2026
Merged
fix(cache): match client-go's reconnect backoff algorithm#3026
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom 
davidgamero:fix/watch-backoff-jitter
Conversation
@davidgamero
@davidgamerodavidgamero
commented
Aug 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
  • Swapped the backoff constants for client-go's defaultBackoff*: 800ms base, ×2, 30s cap, full jitter, 2min reset. 30s cap is unchanged.
  • Added full jitter — delays now land in [level, 2*level),
  • Added a 2-minute idle reset, so a recovered client stops paying for old failures. New lastBackoffAt field.
  • Added nowFn/randFn injection for deterministic tests; updated 3 existing tests to 800/1600/3200, added 2 for jitter spread and cap-then-reset
followup on #3022
ai disclosure: used opencode + claude
@kubernetes-prowkubernetes-prowBot
added

approved
Indicates a PR has been approved by an approver from all required OWNERS files.
cncf-cla: yes
Indicates the PR's author has signed the CNCF CLA.
size/L
Denotes a PR that changes 100-499 lines, ignoring generated files.
labels
Aug 30, 2026
@davidgamero
davidgamero
marked this pull request as draft
August 30, 2026 02:42
@kubernetes-prowkubernetes-prowBot
added
the 
do-not-merge/work-in-progress
Indicates that a PR should not merge because it is a work in progress.
label
Aug 30, 2026
@davidgamero
davidgamero
commented
Aug 30, 2026

Copy link


Copy Markdown

ContributorAuthor
/hold waiting for CI fix
@kubernetes-prowkubernetes-prowBot
added
the 
do-not-merge/hold
Indicates that a PR should not merge because someone has issued a /hold command.
label
Aug 30, 2026
@mstruebing
mstruebing
commented
Aug 31, 2026

Copy link


Copy Markdown

Member
@davidgamero CI should have been landed.
@davidgamero
fix(cache): match client-go's reconnect backoff algorithm
8f23ee7
The reconnect delay doubled from a 1s base with no jitter, so every
client disconnected by the same event (an apiserver rollout, a load
balancer cycling) retried at the same instants and kept colliding.
Adopt client-go's defaultBackoff* values from the reflector's
ExponentialBackoffManager: an 800ms base, a factor of 2, a 30s cap, full
jitter, and a reset once the backoff has gone unused for two minutes.
Full jitter spreads each retry over [level, 2*level), which is what
breaks up the lockstep, and the idle reset means a client that recovered
stops paying for failures it has already outlived.
Only the delay computation changes. When a reconnect backs off at all is
untouched: the first reconnect is still free, and receiving a watch
event or a TimeoutError still clears the delay.
nowFn and randFn join delayFn as injectable options so the backoff can
be asserted deterministically instead of by sleeping.
@davidgamero
davidgameroforce-pushed
the
fix/watch-backoff-jitter
branch
from
925a4f2 to
8f23ee7CompareAugust 31, 2026 03:41
@brendandburns
brendandburns
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
/lgtm

/approve
@kubernetes-prowkubernetes-prowBot
added
the 
lgtm
"Looks good to me", indicates that a PR is ready to be merged.
label
Aug 31, 2026
@brendandburns
brendandburns
marked this pull request as ready for review
August 31, 2026 18:01
@kubernetes-prowkubernetes-prowBot
removed
the 
do-not-merge/work-in-progress
Indicates that a PR should not merge because it is a work in progress.
label
Aug 31, 2026
@kubernetes-prow
kubernetes-prowBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: brendandburns, davidgamero
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Details
Needs approval from an approver in each of these files:

  • OWNERS [brendandburns,davidgamero]
Approvers can indicate their approval by writing /approve in a comment

Approvers can cancel approval by writing /approve cancel in a comment
@davidgamero
davidgamero
commented
Sep 1, 2026

Copy link


Copy Markdown

ContributorAuthor
/unhold
@kubernetes-prowkubernetes-prowBot
removed
the 
do-not-merge/hold
Indicates that a PR should not merge because someone has issued a /hold command.
label
Sep 1, 2026
@kubernetes-prow
kubernetes-prowBot
merged commit 202ce1b
into
kubernetes-client:mainSep 1, 2026

10 checks passed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@brendandburnsbrendandburnsbrendandburns approved these changes
@cjihrigcjihrigAwaiting requested review from cjihrig
@mstruebingmstruebingAwaiting requested review from mstruebing

Assignees

@brendandburnsbrendandburns

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.cncf-cla: yesIndicates the PR's author has signed the CNCF CLA.lgtm"Looks good to me", indicates that a PR is ready to be merged.size/LDenotes a PR that changes 100-499 lines, ignoring generated files.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@davidgamero@mstruebing@brendandburns

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix(cache): match client-go's reconnect backoff algorithm - #3026
Merged
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom 
davidgamero:fix/watch-backoff-jitter
Sep 1, 2026
Merged
fix(cache): match client-go's reconnect backoff algorithm#3026
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom 
davidgamero:fix/watch-backoff-jitter
Conversation
@davidgamero
@davidgamerodavidgamero
commented
Aug 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
  • Swapped the backoff constants for client-go's defaultBackoff*: 800ms base, ×2, 30s cap, full jitter, 2min reset. 30s cap is unchanged.
  • Added full jitter — delays now land in [level, 2*level),
  • Added a 2-minute idle reset, so a recovered client stops paying for old failures. New lastBackoffAt field.
  • Added nowFn/randFn injection for deterministic tests; updated 3 existing tests to 800/1600/3200, added 2 for jitter spread and cap-then-reset
followup on #3022
ai disclosure: used opencode + claude
@kubernetes-prowkubernetes-prowBot
added

approved
Indicates a PR has been approved by an approver from all required OWNERS files.
cncf-cla: yes
Indicates the PR's author has signed the CNCF CLA.
size/L
Denotes a PR that changes 100-499 lines, ignoring generated files.
labels
Aug 30, 2026
@davidgamero
davidgamero
marked this pull request as draft
August 30, 2026 02:42
@kubernetes-prowkubernetes-prowBot
added
the 
do-not-merge/work-in-progress
Indicates that a PR should not merge because it is a work in progress.
label
Aug 30, 2026
@davidgamero
davidgamero
commented
Aug 30, 2026

Copy link


Copy Markdown

ContributorAuthor
/hold waiting for CI fix
@kubernetes-prowkubernetes-prowBot
added
the 
do-not-merge/hold
Indicates that a PR should not merge because someone has issued a /hold command.
label
Aug 30, 2026
@mstruebing
mstruebing
commented
Aug 31, 2026

Copy link


Copy Markdown

Member
@davidgamero CI should have been landed.
@davidgamero
fix(cache): match client-go's reconnect backoff algorithm
8f23ee7
The reconnect delay doubled from a 1s base with no jitter, so every
client disconnected by the same event (an apiserver rollout, a load
balancer cycling) retried at the same instants and kept colliding.
Adopt client-go's defaultBackoff* values from the reflector's
ExponentialBackoffManager: an 800ms base, a factor of 2, a 30s cap, full
jitter, and a reset once the backoff has gone unused for two minutes.
Full jitter spreads each retry over [level, 2*level), which is what
breaks up the lockstep, and the idle reset means a client that recovered
stops paying for failures it has already outlived.
Only the delay computation changes. When a reconnect backs off at all is
untouched: the first reconnect is still free, and receiving a watch
event or a TimeoutError still clears the delay.
nowFn and randFn join delayFn as injectable options so the backoff can
be asserted deterministically instead of by sleeping.
@davidgamero
davidgameroforce-pushed
the
fix/watch-backoff-jitter
branch
from
925a4f2 to
8f23ee7CompareAugust 31, 2026 03:41
@brendandburns
brendandburns
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
/lgtm

/approve
@kubernetes-prowkubernetes-prowBot
added
the 
lgtm
"Looks good to me", indicates that a PR is ready to be merged.
label
Aug 31, 2026
@brendandburns
brendandburns
marked this pull request as ready for review
August 31, 2026 18:01
@kubernetes-prowkubernetes-prowBot
removed
the 
do-not-merge/work-in-progress
Indicates that a PR should not merge because it is a work in progress.
label
Aug 31, 2026
@kubernetes-prow
kubernetes-prowBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: brendandburns, davidgamero
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Details
Needs approval from an approver in each of these files:

  • OWNERS [brendandburns,davidgamero]
Approvers can indicate their approval by writing /approve in a comment

Approvers can cancel approval by writing /approve cancel in a comment
@davidgamero
davidgamero
commented
Sep 1, 2026

Copy link


Copy Markdown

ContributorAuthor
/unhold
@kubernetes-prowkubernetes-prowBot
removed
the 
do-not-merge/hold
Indicates that a PR should not merge because someone has issued a /hold command.
label
Sep 1, 2026
@kubernetes-prow
kubernetes-prowBot
merged commit 202ce1b
into
kubernetes-client:mainSep 1, 2026

10 checks passed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@brendandburnsbrendandburnsbrendandburns approved these changes
@cjihrigcjihrigAwaiting requested review from cjihrig
@mstruebingmstruebingAwaiting requested review from mstruebing

Assignees

@brendandburnsbrendandburns

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.cncf-cla: yesIndicates the PR's author has signed the CNCF CLA.lgtm"Looks good to me", indicates that a PR is ready to be merged.size/LDenotes a PR that changes 100-499 lines, ignoring generated files.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@davidgamero@mstruebing@brendandburns

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix(cache): match client-go's reconnect backoff algorithm - #3026
Merged
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom 
davidgamero:fix/watch-backoff-jitter
Sep 1, 2026
Merged
fix(cache): match client-go's reconnect backoff algorithm#3026
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom 
davidgamero:fix/watch-backoff-jitter
Conversation
@davidgamero
@davidgamerodavidgamero
commented
Aug 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
  • Swapped the backoff constants for client-go's defaultBackoff*: 800ms base, ×2, 30s cap, full jitter, 2min reset. 30s cap is unchanged.
  • Added full jitter — delays now land in [level, 2*level),
  • Added a 2-minute idle reset, so a recovered client stops paying for old failures. New lastBackoffAt field.
  • Added nowFn/randFn injection for deterministic tests; updated 3 existing tests to 800/1600/3200, added 2 for jitter spread and cap-then-reset
followup on #3022
ai disclosure: used opencode + claude
@kubernetes-prowkubernetes-prowBot
added

approved
Indicates a PR has been approved by an approver from all required OWNERS files.
cncf-cla: yes
Indicates the PR's author has signed the CNCF CLA.
size/L
Denotes a PR that changes 100-499 lines, ignoring generated files.
labels
Aug 30, 2026
@davidgamero
davidgamero
marked this pull request as draft
August 30, 2026 02:42
@kubernetes-prowkubernetes-prowBot
added
the 
do-not-merge/work-in-progress
Indicates that a PR should not merge because it is a work in progress.
label
Aug 30, 2026
@davidgamero
davidgamero
commented
Aug 30, 2026

Copy link


Copy Markdown

ContributorAuthor
/hold waiting for CI fix
@kubernetes-prowkubernetes-prowBot
added
the 
do-not-merge/hold
Indicates that a PR should not merge because someone has issued a /hold command.
label
Aug 30, 2026
@mstruebing
mstruebing
commented
Aug 31, 2026

Copy link


Copy Markdown

Member
@davidgamero CI should have been landed.
@davidgamero
fix(cache): match client-go's reconnect backoff algorithm
8f23ee7
The reconnect delay doubled from a 1s base with no jitter, so every
client disconnected by the same event (an apiserver rollout, a load
balancer cycling) retried at the same instants and kept colliding.
Adopt client-go's defaultBackoff* values from the reflector's
ExponentialBackoffManager: an 800ms base, a factor of 2, a 30s cap, full
jitter, and a reset once the backoff has gone unused for two minutes.
Full jitter spreads each retry over [level, 2*level), which is what
breaks up the lockstep, and the idle reset means a client that recovered
stops paying for failures it has already outlived.
Only the delay computation changes. When a reconnect backs off at all is
untouched: the first reconnect is still free, and receiving a watch
event or a TimeoutError still clears the delay.
nowFn and randFn join delayFn as injectable options so the backoff can
be asserted deterministically instead of by sleeping.
@davidgamero
davidgameroforce-pushed
the
fix/watch-backoff-jitter
branch
from
925a4f2 to
8f23ee7CompareAugust 31, 2026 03:41
@brendandburns
brendandburns
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
/lgtm

/approve
@kubernetes-prowkubernetes-prowBot
added
the 
lgtm
"Looks good to me", indicates that a PR is ready to be merged.
label
Aug 31, 2026
@brendandburns
brendandburns
marked this pull request as ready for review
August 31, 2026 18:01
@kubernetes-prowkubernetes-prowBot
removed
the 
do-not-merge/work-in-progress
Indicates that a PR should not merge because it is a work in progress.
label
Aug 31, 2026
@kubernetes-prow
kubernetes-prowBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: brendandburns, davidgamero
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Details
Needs approval from an approver in each of these files:

  • OWNERS [brendandburns,davidgamero]
Approvers can indicate their approval by writing /approve in a comment

Approvers can cancel approval by writing /approve cancel in a comment
@davidgamero
davidgamero
commented
Sep 1, 2026

Copy link


Copy Markdown

ContributorAuthor
/unhold
@kubernetes-prowkubernetes-prowBot
removed
the 
do-not-merge/hold
Indicates that a PR should not merge because someone has issued a /hold command.
label
Sep 1, 2026
@kubernetes-prow
kubernetes-prowBot
merged commit 202ce1b
into
kubernetes-client:mainSep 1, 2026

10 checks passed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@brendandburnsbrendandburnsbrendandburns approved these changes
@cjihrigcjihrigAwaiting requested review from cjihrig
@mstruebingmstruebingAwaiting requested review from mstruebing

Assignees

@brendandburnsbrendandburns

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.cncf-cla: yesIndicates the PR's author has signed the CNCF CLA.lgtm"Looks good to me", indicates that a PR is ready to be merged.size/LDenotes a PR that changes 100-499 lines, ignoring generated files.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@davidgamero@mstruebing@brendandburns

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
fix(cache): match client-go's reconnect backoff algorithm - #3026
Merged
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom 
davidgamero:fix/watch-backoff-jitter
Sep 1, 2026
Merged
fix(cache): match client-go's reconnect backoff algorithm#3026
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom 
davidgamero:fix/watch-backoff-jitter
Conversation
@davidgamero
@davidgamerodavidgamero
commented
Aug 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
  • Swapped the backoff constants for client-go's defaultBackoff*: 800ms base, ×2, 30s cap, full jitter, 2min reset. 30s cap is unchanged.
  • Added full jitter — delays now land in [level, 2*level),
  • Added a 2-minute idle reset, so a recovered client stops paying for old failures. New lastBackoffAt field.
  • Added nowFn/randFn injection for deterministic tests; updated 3 existing tests to 800/1600/3200, added 2 for jitter spread and cap-then-reset
followup on #3022
ai disclosure: used opencode + claude
@kubernetes-prowkubernetes-prowBot
added

approved
Indicates a PR has been approved by an approver from all required OWNERS files.
cncf-cla: yes
Indicates the PR's author has signed the CNCF CLA.
size/L
Denotes a PR that changes 100-499 lines, ignoring generated files.
labels
Aug 30, 2026
@davidgamero
davidgamero
marked this pull request as draft
August 30, 2026 02:42
@kubernetes-prowkubernetes-prowBot
added
the 
do-not-merge/work-in-progress
Indicates that a PR should not merge because it is a work in progress.
label
Aug 30, 2026
@davidgamero
davidgamero
commented
Aug 30, 2026

Copy link


Copy Markdown

ContributorAuthor
/hold waiting for CI fix
@kubernetes-prowkubernetes-prowBot
added
the 
do-not-merge/hold
Indicates that a PR should not merge because someone has issued a /hold command.
label
Aug 30, 2026
@mstruebing
mstruebing
commented
Aug 31, 2026

Copy link


Copy Markdown

Member
@davidgamero CI should have been landed.
@davidgamero
fix(cache): match client-go's reconnect backoff algorithm
8f23ee7
The reconnect delay doubled from a 1s base with no jitter, so every
client disconnected by the same event (an apiserver rollout, a load
balancer cycling) retried at the same instants and kept colliding.
Adopt client-go's defaultBackoff* values from the reflector's
ExponentialBackoffManager: an 800ms base, a factor of 2, a 30s cap, full
jitter, and a reset once the backoff has gone unused for two minutes.
Full jitter spreads each retry over [level, 2*level), which is what
breaks up the lockstep, and the idle reset means a client that recovered
stops paying for failures it has already outlived.
Only the delay computation changes. When a reconnect backs off at all is
untouched: the first reconnect is still free, and receiving a watch
event or a TimeoutError still clears the delay.
nowFn and randFn join delayFn as injectable options so the backoff can
be asserted deterministically instead of by sleeping.
@davidgamero
davidgameroforce-pushed
the
fix/watch-backoff-jitter
branch
from
925a4f2 to
8f23ee7CompareAugust 31, 2026 03:41
@brendandburns
brendandburns
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
/lgtm

/approve
@kubernetes-prowkubernetes-prowBot
added
the 
lgtm
"Looks good to me", indicates that a PR is ready to be merged.
label
Aug 31, 2026
@brendandburns
brendandburns
marked this pull request as ready for review
August 31, 2026 18:01
@kubernetes-prowkubernetes-prowBot
removed
the 
do-not-merge/work-in-progress
Indicates that a PR should not merge because it is a work in progress.
label
Aug 31, 2026
@kubernetes-prow
kubernetes-prowBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: brendandburns, davidgamero
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Details
Needs approval from an approver in each of these files:

  • OWNERS [brendandburns,davidgamero]
Approvers can indicate their approval by writing /approve in a comment

Approvers can cancel approval by writing /approve cancel in a comment
@davidgamero
davidgamero
commented
Sep 1, 2026

Copy link


Copy Markdown

ContributorAuthor
/unhold
@kubernetes-prowkubernetes-prowBot
removed
the 
do-not-merge/hold
Indicates that a PR should not merge because someone has issued a /hold command.
label
Sep 1, 2026
@kubernetes-prow
kubernetes-prowBot
merged commit 202ce1b
into
kubernetes-client:mainSep 1, 2026

10 checks passed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@brendandburnsbrendandburnsbrendandburns approved these changes
@cjihrigcjihrigAwaiting requested review from cjihrig
@mstruebingmstruebingAwaiting requested review from mstruebing

Assignees

@brendandburnsbrendandburns

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.cncf-cla: yesIndicates the PR's author has signed the CNCF CLA.lgtm"Looks good to me", indicates that a PR is ready to be merged.size/LDenotes a PR that changes 100-499 lines, ignoring generated files.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@davidgamero@mstruebing@brendandburns

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix(cache): match client-go's reconnect backoff algorithm - #3026
Merged
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom 
davidgamero:fix/watch-backoff-jitter
Sep 1, 2026
Merged
fix(cache): match client-go's reconnect backoff algorithm#3026
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom 
davidgamero:fix/watch-backoff-jitter
Conversation
@davidgamero
@davidgamerodavidgamero
commented
Aug 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
  • Swapped the backoff constants for client-go's defaultBackoff*: 800ms base, ×2, 30s cap, full jitter, 2min reset. 30s cap is unchanged.
  • Added full jitter — delays now land in [level, 2*level),
  • Added a 2-minute idle reset, so a recovered client stops paying for old failures. New lastBackoffAt field.
  • Added nowFn/randFn injection for deterministic tests; updated 3 existing tests to 800/1600/3200, added 2 for jitter spread and cap-then-reset
followup on #3022
ai disclosure: used opencode + claude
@kubernetes-prowkubernetes-prowBot
added

approved
Indicates a PR has been approved by an approver from all required OWNERS files.
cncf-cla: yes
Indicates the PR's author has signed the CNCF CLA.
size/L
Denotes a PR that changes 100-499 lines, ignoring generated files.
labels
Aug 30, 2026
@davidgamero
davidgamero
marked this pull request as draft
August 30, 2026 02:42
@kubernetes-prowkubernetes-prowBot
added
the 
do-not-merge/work-in-progress
Indicates that a PR should not merge because it is a work in progress.
label
Aug 30, 2026
@davidgamero
davidgamero
commented
Aug 30, 2026

Copy link


Copy Markdown

ContributorAuthor
/hold waiting for CI fix
@kubernetes-prowkubernetes-prowBot
added
the 
do-not-merge/hold
Indicates that a PR should not merge because someone has issued a /hold command.
label
Aug 30, 2026
@mstruebing
mstruebing
commented
Aug 31, 2026

Copy link


Copy Markdown

Member
@davidgamero CI should have been landed.
@davidgamero
fix(cache): match client-go's reconnect backoff algorithm
8f23ee7
The reconnect delay doubled from a 1s base with no jitter, so every
client disconnected by the same event (an apiserver rollout, a load
balancer cycling) retried at the same instants and kept colliding.
Adopt client-go's defaultBackoff* values from the reflector's
ExponentialBackoffManager: an 800ms base, a factor of 2, a 30s cap, full
jitter, and a reset once the backoff has gone unused for two minutes.
Full jitter spreads each retry over [level, 2*level), which is what
breaks up the lockstep, and the idle reset means a client that recovered
stops paying for failures it has already outlived.
Only the delay computation changes. When a reconnect backs off at all is
untouched: the first reconnect is still free, and receiving a watch
event or a TimeoutError still clears the delay.
nowFn and randFn join delayFn as injectable options so the backoff can
be asserted deterministically instead of by sleeping.
@davidgamero
davidgameroforce-pushed
the
fix/watch-backoff-jitter
branch
from
925a4f2 to
8f23ee7CompareAugust 31, 2026 03:41
@brendandburns
brendandburns
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
/lgtm

/approve
@kubernetes-prowkubernetes-prowBot
added
the 
lgtm
"Looks good to me", indicates that a PR is ready to be merged.
label
Aug 31, 2026
@brendandburns
brendandburns
marked this pull request as ready for review
August 31, 2026 18:01
@kubernetes-prowkubernetes-prowBot
removed
the 
do-not-merge/work-in-progress
Indicates that a PR should not merge because it is a work in progress.
label
Aug 31, 2026
@kubernetes-prow
kubernetes-prowBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: brendandburns, davidgamero
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Details
Needs approval from an approver in each of these files:

  • OWNERS [brendandburns,davidgamero]
Approvers can indicate their approval by writing /approve in a comment

Approvers can cancel approval by writing /approve cancel in a comment
@davidgamero
davidgamero
commented
Sep 1, 2026

Copy link


Copy Markdown

ContributorAuthor
/unhold
@kubernetes-prowkubernetes-prowBot
removed
the 
do-not-merge/hold
Indicates that a PR should not merge because someone has issued a /hold command.
label
Sep 1, 2026
@kubernetes-prow
kubernetes-prowBot
merged commit 202ce1b
into
kubernetes-client:mainSep 1, 2026

10 checks passed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@brendandburnsbrendandburnsbrendandburns approved these changes
@cjihrigcjihrigAwaiting requested review from cjihrig
@mstruebingmstruebingAwaiting requested review from mstruebing

Assignees

@brendandburnsbrendandburns

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.cncf-cla: yesIndicates the PR's author has signed the CNCF CLA.lgtm"Looks good to me", indicates that a PR is ready to be merged.size/LDenotes a PR that changes 100-499 lines, ignoring generated files.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@davidgamero@mstruebing@brendandburns

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix(cache): match client-go's reconnect backoff algorithm - #3026
Merged
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom 
davidgamero:fix/watch-backoff-jitter
Sep 1, 2026
Merged
fix(cache): match client-go's reconnect backoff algorithm#3026
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom 
davidgamero:fix/watch-backoff-jitter
Conversation
@davidgamero
@davidgamerodavidgamero
commented
Aug 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
  • Swapped the backoff constants for client-go's defaultBackoff*: 800ms base, ×2, 30s cap, full jitter, 2min reset. 30s cap is unchanged.
  • Added full jitter — delays now land in [level, 2*level),
  • Added a 2-minute idle reset, so a recovered client stops paying for old failures. New lastBackoffAt field.
  • Added nowFn/randFn injection for deterministic tests; updated 3 existing tests to 800/1600/3200, added 2 for jitter spread and cap-then-reset
followup on #3022
ai disclosure: used opencode + claude
@kubernetes-prowkubernetes-prowBot
added

approved
Indicates a PR has been approved by an approver from all required OWNERS files.
cncf-cla: yes
Indicates the PR's author has signed the CNCF CLA.
size/L
Denotes a PR that changes 100-499 lines, ignoring generated files.
labels
Aug 30, 2026
@davidgamero
davidgamero
marked this pull request as draft
August 30, 2026 02:42
@kubernetes-prowkubernetes-prowBot
added
the 
do-not-merge/work-in-progress
Indicates that a PR should not merge because it is a work in progress.
label
Aug 30, 2026
@davidgamero
davidgamero
commented
Aug 30, 2026

Copy link


Copy Markdown

ContributorAuthor
/hold waiting for CI fix
@kubernetes-prowkubernetes-prowBot
added
the 
do-not-merge/hold
Indicates that a PR should not merge because someone has issued a /hold command.
label
Aug 30, 2026
@mstruebing
mstruebing
commented
Aug 31, 2026

Copy link


Copy Markdown

Member
@davidgamero CI should have been landed.
@davidgamero
fix(cache): match client-go's reconnect backoff algorithm
8f23ee7
The reconnect delay doubled from a 1s base with no jitter, so every
client disconnected by the same event (an apiserver rollout, a load
balancer cycling) retried at the same instants and kept colliding.
Adopt client-go's defaultBackoff* values from the reflector's
ExponentialBackoffManager: an 800ms base, a factor of 2, a 30s cap, full
jitter, and a reset once the backoff has gone unused for two minutes.
Full jitter spreads each retry over [level, 2*level), which is what
breaks up the lockstep, and the idle reset means a client that recovered
stops paying for failures it has already outlived.
Only the delay computation changes. When a reconnect backs off at all is
untouched: the first reconnect is still free, and receiving a watch
event or a TimeoutError still clears the delay.
nowFn and randFn join delayFn as injectable options so the backoff can
be asserted deterministically instead of by sleeping.
@davidgamero
davidgameroforce-pushed
the
fix/watch-backoff-jitter
branch
from
925a4f2 to
8f23ee7CompareAugust 31, 2026 03:41
@brendandburns
brendandburns
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
/lgtm

/approve
@kubernetes-prowkubernetes-prowBot
added
the 
lgtm
"Looks good to me", indicates that a PR is ready to be merged.
label
Aug 31, 2026
@brendandburns
brendandburns
marked this pull request as ready for review
August 31, 2026 18:01
@kubernetes-prowkubernetes-prowBot
removed
the 
do-not-merge/work-in-progress
Indicates that a PR should not merge because it is a work in progress.
label
Aug 31, 2026
@kubernetes-prow
kubernetes-prowBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: brendandburns, davidgamero
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Details
Needs approval from an approver in each of these files:

  • OWNERS [brendandburns,davidgamero]
Approvers can indicate their approval by writing /approve in a comment

Approvers can cancel approval by writing /approve cancel in a comment
@davidgamero
davidgamero
commented
Sep 1, 2026

Copy link


Copy Markdown

ContributorAuthor
/unhold
@kubernetes-prowkubernetes-prowBot
removed
the 
do-not-merge/hold
Indicates that a PR should not merge because someone has issued a /hold command.
label
Sep 1, 2026
@kubernetes-prow
kubernetes-prowBot
merged commit 202ce1b
into
kubernetes-client:mainSep 1, 2026

10 checks passed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@brendandburnsbrendandburnsbrendandburns approved these changes
@cjihrigcjihrigAwaiting requested review from cjihrig
@mstruebingmstruebingAwaiting requested review from mstruebing

Assignees

@brendandburnsbrendandburns

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.cncf-cla: yesIndicates the PR's author has signed the CNCF CLA.lgtm"Looks good to me", indicates that a PR is ready to be merged.size/LDenotes a PR that changes 100-499 lines, ignoring generated files.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@davidgamero@mstruebing@brendandburns

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
fix(cache): match client-go's reconnect backoff algorithm - #3026
Merged
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom 
davidgamero:fix/watch-backoff-jitter
Sep 1, 2026
Merged
fix(cache): match client-go's reconnect backoff algorithm#3026
kubernetes-prow[bot] merged 1 commit into
kubernetes-client:mainfrom 
davidgamero:fix/watch-backoff-jitter
Conversation
@davidgamero
@davidgamerodavidgamero
commented
Aug 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
  • Swapped the backoff constants for client-go's defaultBackoff*: 800ms base, ×2, 30s cap, full jitter, 2min reset. 30s cap is unchanged.
  • Added full jitter — delays now land in [level, 2*level),
  • Added a 2-minute idle reset, so a recovered client stops paying for old failures. New lastBackoffAt field.
  • Added nowFn/randFn injection for deterministic tests; updated 3 existing tests to 800/1600/3200, added 2 for jitter spread and cap-then-reset
followup on #3022
ai disclosure: used opencode + claude
@kubernetes-prowkubernetes-prowBot
added

approved
Indicates a PR has been approved by an approver from all required OWNERS files.
cncf-cla: yes
Indicates the PR's author has signed the CNCF CLA.
size/L
Denotes a PR that changes 100-499 lines, ignoring generated files.
labels
Aug 30, 2026
@davidgamero
davidgamero
marked this pull request as draft
August 30, 2026 02:42
@kubernetes-prowkubernetes-prowBot
added
the 
do-not-merge/work-in-progress
Indicates that a PR should not merge because it is a work in progress.
label
Aug 30, 2026
@davidgamero
davidgamero
commented
Aug 30, 2026

Copy link


Copy Markdown

ContributorAuthor
/hold waiting for CI fix
@kubernetes-prowkubernetes-prowBot
added
the 
do-not-merge/hold
Indicates that a PR should not merge because someone has issued a /hold command.
label
Aug 30, 2026
@mstruebing
mstruebing
commented
Aug 31, 2026

Copy link


Copy Markdown

Member
@davidgamero CI should have been landed.
@davidgamero
fix(cache): match client-go's reconnect backoff algorithm
8f23ee7
The reconnect delay doubled from a 1s base with no jitter, so every
client disconnected by the same event (an apiserver rollout, a load
balancer cycling) retried at the same instants and kept colliding.
Adopt client-go's defaultBackoff* values from the reflector's
ExponentialBackoffManager: an 800ms base, a factor of 2, a 30s cap, full
jitter, and a reset once the backoff has gone unused for two minutes.
Full jitter spreads each retry over [level, 2*level), which is what
breaks up the lockstep, and the idle reset means a client that recovered
stops paying for failures it has already outlived.
Only the delay computation changes. When a reconnect backs off at all is
untouched: the first reconnect is still free, and receiving a watch
event or a TimeoutError still clears the delay.
nowFn and randFn join delayFn as injectable options so the backoff can
be asserted deterministically instead of by sleeping.
@davidgamero
davidgameroforce-pushed
the
fix/watch-backoff-jitter
branch
from
925a4f2 to
8f23ee7CompareAugust 31, 2026 03:41
@brendandburns
brendandburns
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
/lgtm

/approve
@kubernetes-prowkubernetes-prowBot
added
the 
lgtm
"Looks good to me", indicates that a PR is ready to be merged.
label
Aug 31, 2026
@brendandburns
brendandburns
marked this pull request as ready for review
August 31, 2026 18:01
@kubernetes-prowkubernetes-prowBot
removed
the 
do-not-merge/work-in-progress
Indicates that a PR should not merge because it is a work in progress.
label
Aug 31, 2026
@kubernetes-prow
kubernetes-prowBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: brendandburns, davidgamero
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Details
Needs approval from an approver in each of these files:

  • OWNERS [brendandburns,davidgamero]
Approvers can indicate their approval by writing /approve in a comment

Approvers can cancel approval by writing /approve cancel in a comment
@davidgamero
davidgamero
commented
Sep 1, 2026

Copy link


Copy Markdown

ContributorAuthor
/unhold
@kubernetes-prowkubernetes-prowBot
removed
the 
do-not-merge/hold
Indicates that a PR should not merge because someone has issued a /hold command.
label
Sep 1, 2026
@kubernetes-prow
kubernetes-prowBot
merged commit 202ce1b
into
kubernetes-client:mainSep 1, 2026

10 checks passed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@brendandburnsbrendandburnsbrendandburns approved these changes
@cjihrigcjihrigAwaiting requested review from cjihrig
@mstruebingmstruebingAwaiting requested review from mstruebing

Assignees

@brendandburnsbrendandburns

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.cncf-cla: yesIndicates the PR's author has signed the CNCF CLA.lgtm"Looks good to me", indicates that a PR is ready to be merged.size/LDenotes a PR that changes 100-499 lines, ignoring generated files.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@davidgamero@mstruebing@brendandburns