Bound buffer for reading stats - #14523

Merged
knative-prow[bot] merged 1 commit into
knative:mainfrom
evankanderson:scaling-bounds
Oct 16, 2023
Merged

Bound buffer for reading stats#14523
knative-prow[bot] merged 1 commit into
knative:mainfrom
evankanderson:scaling-bounds

Conversation

@evankanderson

@evankandersonevankanderson commented Oct 16, 2023

Copy link
Copy Markdown
Member

Proposed Changes

  • Bound the bytes read from the queue-proxy by the autoscaler.

Release Note

Applied an upper bound to the statistics data read from the queue-proxy by the autoscaler.

@knative-prowknative-prowBot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. area/autoscale labels Oct 16, 2023
@codecov

codecovBot commented Oct 16, 2023•
edited
Loading

Copy link
Copy Markdown

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (6703812) 86.08% compared to head (588ea26) 86.13%.
Report is 7 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #14523 +/- ##
==========================================
+ Coverage 86.08% 86.13% +0.04% 
==========================================
Files 196 196 Lines 14887 14889 +2 ==========================================
+ Hits 12815 12824 +9 + Misses 1761 1756 -5 + Partials 311 309 -2
FilesCoverage Δ
pkg/autoscaler/metrics/http_scrape_client.go93.02% <100.00%> (+7.65%)⬆️

... and 4 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

dprotaso
dprotaso reviewed Oct 16, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client_test.go
ProcessUptime: 12345.678,
Timestamp: 1697431278,
},
expectedErr: "unmarshalling failed: unexpected EOF",

@dprotasodprotasoOct 16, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: any way to distinguish we hit the limit vs EOF on the wire?

@evankandersonevankandersonOct 18, 2023

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not easily. Technically, we could also end up skipping a bunch of extra garbage after a valid proto, but since the proto contains a length at the start, we can tell whether we read the whole proto or not just by reading the beginning. (In fact, it would be nice if the generated proto code had a method that read from an io.Reader, but it doesn't seem to.)

dprotaso
dprotaso approved these changes Oct 16, 2023
View reviewed changes

@dprotasodprotaso left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 16, 2023
@knative-prow

knative-prowBot commented Oct 16, 2023

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dprotaso, evankanderson

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prowknative-prowBot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 16, 2023
@knative-prow
knative-prowBot merged commit fff40ef into knative:mainOct 16, 2023
@evankanderson

evankanderson commented Oct 18, 2023

Copy link
Copy Markdown
MemberAuthor

/cherrypick release-1.10
/cherrypick release-1.11

@knative-prow-robotknative-prow-robot mentioned this pull request Oct 18, 2023
Merged
@knative-prow-robot

knative-prow-robot commented Oct 18, 2023

Copy link
Copy Markdown
Contributor

@evankanderson: new pull request created: #14541

Details

In response to this:

/cherrypick release-1.10
/cherrypick release-1.11

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@evankanderson

evankanderson commented Oct 18, 2023

Copy link
Copy Markdown
MemberAuthor

/cherrypick release-1.11

@knative-prow-robotknative-prow-robot mentioned this pull request Oct 18, 2023
Merged
@knative-prow-robot

knative-prow-robot commented Oct 18, 2023

Copy link
Copy Markdown
Contributor

@evankanderson: new pull request created: #14542

Details

In response to this:

/cherrypick release-1.11

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

skonto
skonto reviewed Oct 19, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client.go
defer pool.Put(b)
_, err := b.ReadFrom(body)
// 6 8-byte fields (+2 bytes marshalling), one hostname, 20 bytes extra space
r := io.LimitedReader{R: body, N: 6*10 + 256 + 20}

@skontoskontoOct 19, 2023•
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Should not this be 255 for the max hostname? 🤔

@evankandersonevankandersonOct 19, 2023

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra-nit: you might need two bytes for the tag + length if the string is actually 256 bytes long, plus another possible tag + length for the entire message.

Proto doesn't have a way to express / enforce that a string is less than a certain size. The goal here is simply to avoid accidentally reading e.g. 5MB or more into an in-memory buffer when we know the actual report should be less than 1KB.

@skontoskontoOct 20, 2023•
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity (haven't checked), what about using grpc instead?

This was referenced Jan 24, 2024
Merged
Open
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@skontoskontoskonto left review comments
@dprotasodprotasodprotaso approved these changes
@jsanin-vmwjsanin-vmwAwaiting requested review from jsanin-vmw
@ReToCodeReToCodeAwaiting requested review from ReToCode

Assignees

@dprotasodprotaso

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/autoscalelgtmIndicates that a PR is ready to be merged.size/SDenotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@evankanderson@knative-prow-robot@dprotaso@skonto
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Bound buffer for reading stats - #14523
Merged
knative-prow[bot] merged 1 commit into
knative:mainfrom 
evankanderson:scaling-bounds
Oct 16, 2023
Merged
Bound buffer for reading stats#14523
knative-prow[bot] merged 1 commit into
knative:mainfrom 
evankanderson:scaling-bounds
Conversation
@evankanderson
@evankandersonevankanderson
commented
Oct 16, 2023

Copy link


Copy Markdown

Member
Proposed Changes
  • Bound the bytes read from the queue-proxy by the autoscaler.
Release Note
Applied an upper bound to the statistics data read from the queue-proxy by the autoscaler.

@knative-prowknative-prowBot
added

size/S
Denotes a PR that changes 10-29 lines, ignoring generated files.
area/autoscale

labels
Oct 16, 2023
@codecov
codecovBot
commented
Oct 16, 2023•

edited

Loading

Copy link


Copy Markdown

Codecov Report
All modified lines are covered by tests ✅
Comparison is base (6703812) 86.08% compared to head (588ea26) 86.13%.

Report is 7 commits behind head on main.
Additional details and impacted files
@@ Coverage Diff @@## main #14523 +/- ##
==========================================
+ Coverage 86.08% 86.13% +0.04% 
==========================================
Files 196 196 Lines 14887 14889 +2 ==========================================
+ Hits 12815 12824 +9 + Misses 1761 1756 -5 + Partials 311 309 -2 
FilesCoverage Δ
pkg/autoscaler/metrics/http_scrape_client.go93.02% <100.00%> (+7.65%)⬆️
... and 4 files with indirect coverage changes
☔ View full report in Codecov by Sentry.

📢 Have feedback on the report? Share it here.
dprotaso
dprotaso
reviewed
Oct 16, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client_test.go
 ProcessUptime: 12345.678,
 Timestamp: 1697431278,
 },
 expectedErr: "unmarshalling failed: unexpected EOF",
@dprotasodprotasoOct 16, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: any way to distinguish we hit the limit vs EOF on the wire?
@evankandersonevankandersonOct 18, 2023

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not easily. Technically, we could also end up skipping a bunch of extra garbage after a valid proto, but since the proto contains a length at the start, we can tell whether we read the whole proto or not just by reading the beginning. (In fact, it would be nice if the generated proto code had a method that read from an io.Reader, but it doesn't seem to.)
dprotaso
dprotaso
approved these changes
Oct 16, 2023
View reviewed changes
@dprotasodprotaso
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

/approve
@knative-prowknative-prowBot
added
the 
lgtm
Indicates that a PR is ready to be merged.
label
Oct 16, 2023
@knative-prow
knative-prowBot
commented
Oct 16, 2023

Copy link


Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: dprotaso, evankanderson
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Details
Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment

Approvers can cancel approval by writing /approve cancel in a comment
@knative-prowknative-prowBot
added
the 
approved
Indicates a PR has been approved by an approver from all required OWNERS files.
label
Oct 16, 2023
@knative-prow
knative-prowBot
merged commit fff40ef
into
knative:mainOct 16, 2023
@evankanderson
evankanderson
commented
Oct 18, 2023

Copy link


Copy Markdown

MemberAuthor
/cherrypick release-1.10

/cherrypick release-1.11
@knative-prow-robotknative-prow-robot
mentioned this pull request
Oct 18, 2023
 Merged

@knative-prow-robot
knative-prow-robot
commented
Oct 18, 2023

Copy link


Copy Markdown

Contributor
@evankanderson: new pull request created: #14541
Details
In response to this:
/cherrypick release-1.10

/cherrypick release-1.11
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@evankanderson
evankanderson
commented
Oct 18, 2023

Copy link


Copy Markdown

MemberAuthor
/cherrypick release-1.11
@knative-prow-robotknative-prow-robot
mentioned this pull request
Oct 18, 2023
 Merged

@knative-prow-robot
knative-prow-robot
commented
Oct 18, 2023

Copy link


Copy Markdown

Contributor
@evankanderson: new pull request created: #14542
Details
In response to this:
/cherrypick release-1.11
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
skonto
skonto
reviewed
Oct 19, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client.go
	defer pool.Put(b)
	_, err := b.ReadFrom(body)
	// 6 8-byte fields (+2 bytes marshalling), one hostname, 20 bytes extra space
	r := io.LimitedReader{R: body, N: 6*10 + 256 + 20}
@skontoskontoOct 19, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Should not this be 255 for the max hostname? 🤔
@evankandersonevankandersonOct 19, 2023

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra-nit: you might need two bytes for the tag + length if the string is actually 256 bytes long, plus another possible tag + length for the entire message.
Proto doesn't have a way to express / enforce that a string is less than a certain size. The goal here is simply to avoid accidentally reading e.g. 5MB or more into an in-memory buffer when we know the actual report should be less than 1KB.
@skontoskontoOct 20, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity (haven't checked), what about using grpc instead?

This was referenced Jan 24, 2024
 Merged


Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@skontoskontoskonto left review comments
@dprotasodprotasodprotaso approved these changes
@jsanin-vmwjsanin-vmwAwaiting requested review from jsanin-vmw
@ReToCodeReToCodeAwaiting requested review from ReToCode

Assignees

@dprotasodprotaso

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/autoscalelgtmIndicates that a PR is ready to be merged.size/SDenotes a PR that changes 10-29 lines, ignoring generated files.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@evankanderson@knative-prow-robot@dprotaso@skonto

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Bound buffer for reading stats - #14523
Merged
knative-prow[bot] merged 1 commit into
knative:mainfrom 
evankanderson:scaling-bounds
Oct 16, 2023
Merged
Bound buffer for reading stats#14523
knative-prow[bot] merged 1 commit into
knative:mainfrom 
evankanderson:scaling-bounds
Conversation
@evankanderson
@evankandersonevankanderson
commented
Oct 16, 2023

Copy link


Copy Markdown

Member
Proposed Changes
  • Bound the bytes read from the queue-proxy by the autoscaler.
Release Note
Applied an upper bound to the statistics data read from the queue-proxy by the autoscaler.

@knative-prowknative-prowBot
added

size/S
Denotes a PR that changes 10-29 lines, ignoring generated files.
area/autoscale

labels
Oct 16, 2023
@codecov
codecovBot
commented
Oct 16, 2023•

edited

Loading

Copy link


Copy Markdown

Codecov Report
All modified lines are covered by tests ✅
Comparison is base (6703812) 86.08% compared to head (588ea26) 86.13%.

Report is 7 commits behind head on main.
Additional details and impacted files
@@ Coverage Diff @@## main #14523 +/- ##
==========================================
+ Coverage 86.08% 86.13% +0.04% 
==========================================
Files 196 196 Lines 14887 14889 +2 ==========================================
+ Hits 12815 12824 +9 + Misses 1761 1756 -5 + Partials 311 309 -2 
FilesCoverage Δ
pkg/autoscaler/metrics/http_scrape_client.go93.02% <100.00%> (+7.65%)⬆️
... and 4 files with indirect coverage changes
☔ View full report in Codecov by Sentry.

📢 Have feedback on the report? Share it here.
dprotaso
dprotaso
reviewed
Oct 16, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client_test.go
 ProcessUptime: 12345.678,
 Timestamp: 1697431278,
 },
 expectedErr: "unmarshalling failed: unexpected EOF",
@dprotasodprotasoOct 16, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: any way to distinguish we hit the limit vs EOF on the wire?
@evankandersonevankandersonOct 18, 2023

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not easily. Technically, we could also end up skipping a bunch of extra garbage after a valid proto, but since the proto contains a length at the start, we can tell whether we read the whole proto or not just by reading the beginning. (In fact, it would be nice if the generated proto code had a method that read from an io.Reader, but it doesn't seem to.)
dprotaso
dprotaso
approved these changes
Oct 16, 2023
View reviewed changes
@dprotasodprotaso
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

/approve
@knative-prowknative-prowBot
added
the 
lgtm
Indicates that a PR is ready to be merged.
label
Oct 16, 2023
@knative-prow
knative-prowBot
commented
Oct 16, 2023

Copy link


Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: dprotaso, evankanderson
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Details
Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment

Approvers can cancel approval by writing /approve cancel in a comment
@knative-prowknative-prowBot
added
the 
approved
Indicates a PR has been approved by an approver from all required OWNERS files.
label
Oct 16, 2023
@knative-prow
knative-prowBot
merged commit fff40ef
into
knative:mainOct 16, 2023
@evankanderson
evankanderson
commented
Oct 18, 2023

Copy link


Copy Markdown

MemberAuthor
/cherrypick release-1.10

/cherrypick release-1.11
@knative-prow-robotknative-prow-robot
mentioned this pull request
Oct 18, 2023
 Merged

@knative-prow-robot
knative-prow-robot
commented
Oct 18, 2023

Copy link


Copy Markdown

Contributor
@evankanderson: new pull request created: #14541
Details
In response to this:
/cherrypick release-1.10

/cherrypick release-1.11
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@evankanderson
evankanderson
commented
Oct 18, 2023

Copy link


Copy Markdown

MemberAuthor
/cherrypick release-1.11
@knative-prow-robotknative-prow-robot
mentioned this pull request
Oct 18, 2023
 Merged

@knative-prow-robot
knative-prow-robot
commented
Oct 18, 2023

Copy link


Copy Markdown

Contributor
@evankanderson: new pull request created: #14542
Details
In response to this:
/cherrypick release-1.11
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
skonto
skonto
reviewed
Oct 19, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client.go
	defer pool.Put(b)
	_, err := b.ReadFrom(body)
	// 6 8-byte fields (+2 bytes marshalling), one hostname, 20 bytes extra space
	r := io.LimitedReader{R: body, N: 6*10 + 256 + 20}
@skontoskontoOct 19, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Should not this be 255 for the max hostname? 🤔
@evankandersonevankandersonOct 19, 2023

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra-nit: you might need two bytes for the tag + length if the string is actually 256 bytes long, plus another possible tag + length for the entire message.
Proto doesn't have a way to express / enforce that a string is less than a certain size. The goal here is simply to avoid accidentally reading e.g. 5MB or more into an in-memory buffer when we know the actual report should be less than 1KB.
@skontoskontoOct 20, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity (haven't checked), what about using grpc instead?

This was referenced Jan 24, 2024
 Merged


Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@skontoskontoskonto left review comments
@dprotasodprotasodprotaso approved these changes
@jsanin-vmwjsanin-vmwAwaiting requested review from jsanin-vmw
@ReToCodeReToCodeAwaiting requested review from ReToCode

Assignees

@dprotasodprotaso

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/autoscalelgtmIndicates that a PR is ready to be merged.size/SDenotes a PR that changes 10-29 lines, ignoring generated files.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@evankanderson@knative-prow-robot@dprotaso@skonto

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Bound buffer for reading stats - #14523
Merged
knative-prow[bot] merged 1 commit into
knative:mainfrom 
evankanderson:scaling-bounds
Oct 16, 2023
Merged
Bound buffer for reading stats#14523
knative-prow[bot] merged 1 commit into
knative:mainfrom 
evankanderson:scaling-bounds
Conversation
@evankanderson
@evankandersonevankanderson
commented
Oct 16, 2023

Copy link


Copy Markdown

Member
Proposed Changes
  • Bound the bytes read from the queue-proxy by the autoscaler.
Release Note
Applied an upper bound to the statistics data read from the queue-proxy by the autoscaler.

@knative-prowknative-prowBot
added

size/S
Denotes a PR that changes 10-29 lines, ignoring generated files.
area/autoscale

labels
Oct 16, 2023
@codecov
codecovBot
commented
Oct 16, 2023•

edited

Loading

Copy link


Copy Markdown

Codecov Report
All modified lines are covered by tests ✅
Comparison is base (6703812) 86.08% compared to head (588ea26) 86.13%.

Report is 7 commits behind head on main.
Additional details and impacted files
@@ Coverage Diff @@## main #14523 +/- ##
==========================================
+ Coverage 86.08% 86.13% +0.04% 
==========================================
Files 196 196 Lines 14887 14889 +2 ==========================================
+ Hits 12815 12824 +9 + Misses 1761 1756 -5 + Partials 311 309 -2 
FilesCoverage Δ
pkg/autoscaler/metrics/http_scrape_client.go93.02% <100.00%> (+7.65%)⬆️
... and 4 files with indirect coverage changes
☔ View full report in Codecov by Sentry.

📢 Have feedback on the report? Share it here.
dprotaso
dprotaso
reviewed
Oct 16, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client_test.go
 ProcessUptime: 12345.678,
 Timestamp: 1697431278,
 },
 expectedErr: "unmarshalling failed: unexpected EOF",
@dprotasodprotasoOct 16, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: any way to distinguish we hit the limit vs EOF on the wire?
@evankandersonevankandersonOct 18, 2023

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not easily. Technically, we could also end up skipping a bunch of extra garbage after a valid proto, but since the proto contains a length at the start, we can tell whether we read the whole proto or not just by reading the beginning. (In fact, it would be nice if the generated proto code had a method that read from an io.Reader, but it doesn't seem to.)
dprotaso
dprotaso
approved these changes
Oct 16, 2023
View reviewed changes
@dprotasodprotaso
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

/approve
@knative-prowknative-prowBot
added
the 
lgtm
Indicates that a PR is ready to be merged.
label
Oct 16, 2023
@knative-prow
knative-prowBot
commented
Oct 16, 2023

Copy link


Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: dprotaso, evankanderson
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Details
Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment

Approvers can cancel approval by writing /approve cancel in a comment
@knative-prowknative-prowBot
added
the 
approved
Indicates a PR has been approved by an approver from all required OWNERS files.
label
Oct 16, 2023
@knative-prow
knative-prowBot
merged commit fff40ef
into
knative:mainOct 16, 2023
@evankanderson
evankanderson
commented
Oct 18, 2023

Copy link


Copy Markdown

MemberAuthor
/cherrypick release-1.10

/cherrypick release-1.11
@knative-prow-robotknative-prow-robot
mentioned this pull request
Oct 18, 2023
 Merged

@knative-prow-robot
knative-prow-robot
commented
Oct 18, 2023

Copy link


Copy Markdown

Contributor
@evankanderson: new pull request created: #14541
Details
In response to this:
/cherrypick release-1.10

/cherrypick release-1.11
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@evankanderson
evankanderson
commented
Oct 18, 2023

Copy link


Copy Markdown

MemberAuthor
/cherrypick release-1.11
@knative-prow-robotknative-prow-robot
mentioned this pull request
Oct 18, 2023
 Merged

@knative-prow-robot
knative-prow-robot
commented
Oct 18, 2023

Copy link


Copy Markdown

Contributor
@evankanderson: new pull request created: #14542
Details
In response to this:
/cherrypick release-1.11
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
skonto
skonto
reviewed
Oct 19, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client.go
	defer pool.Put(b)
	_, err := b.ReadFrom(body)
	// 6 8-byte fields (+2 bytes marshalling), one hostname, 20 bytes extra space
	r := io.LimitedReader{R: body, N: 6*10 + 256 + 20}
@skontoskontoOct 19, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Should not this be 255 for the max hostname? 🤔
@evankandersonevankandersonOct 19, 2023

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra-nit: you might need two bytes for the tag + length if the string is actually 256 bytes long, plus another possible tag + length for the entire message.
Proto doesn't have a way to express / enforce that a string is less than a certain size. The goal here is simply to avoid accidentally reading e.g. 5MB or more into an in-memory buffer when we know the actual report should be less than 1KB.
@skontoskontoOct 20, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity (haven't checked), what about using grpc instead?

This was referenced Jan 24, 2024
 Merged


Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@skontoskontoskonto left review comments
@dprotasodprotasodprotaso approved these changes
@jsanin-vmwjsanin-vmwAwaiting requested review from jsanin-vmw
@ReToCodeReToCodeAwaiting requested review from ReToCode

Assignees

@dprotasodprotaso

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/autoscalelgtmIndicates that a PR is ready to be merged.size/SDenotes a PR that changes 10-29 lines, ignoring generated files.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@evankanderson@knative-prow-robot@dprotaso@skonto

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Bound buffer for reading stats - #14523
Merged
knative-prow[bot] merged 1 commit into
knative:mainfrom 
evankanderson:scaling-bounds
Oct 16, 2023
Merged
Bound buffer for reading stats#14523
knative-prow[bot] merged 1 commit into
knative:mainfrom 
evankanderson:scaling-bounds
Conversation
@evankanderson
@evankandersonevankanderson
commented
Oct 16, 2023

Copy link


Copy Markdown

Member
Proposed Changes
  • Bound the bytes read from the queue-proxy by the autoscaler.
Release Note
Applied an upper bound to the statistics data read from the queue-proxy by the autoscaler.

@knative-prowknative-prowBot
added

size/S
Denotes a PR that changes 10-29 lines, ignoring generated files.
area/autoscale

labels
Oct 16, 2023
@codecov
codecovBot
commented
Oct 16, 2023•

edited

Loading

Copy link


Copy Markdown

Codecov Report
All modified lines are covered by tests ✅
Comparison is base (6703812) 86.08% compared to head (588ea26) 86.13%.

Report is 7 commits behind head on main.
Additional details and impacted files
@@ Coverage Diff @@## main #14523 +/- ##
==========================================
+ Coverage 86.08% 86.13% +0.04% 
==========================================
Files 196 196 Lines 14887 14889 +2 ==========================================
+ Hits 12815 12824 +9 + Misses 1761 1756 -5 + Partials 311 309 -2 
FilesCoverage Δ
pkg/autoscaler/metrics/http_scrape_client.go93.02% <100.00%> (+7.65%)⬆️
... and 4 files with indirect coverage changes
☔ View full report in Codecov by Sentry.

📢 Have feedback on the report? Share it here.
dprotaso
dprotaso
reviewed
Oct 16, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client_test.go
 ProcessUptime: 12345.678,
 Timestamp: 1697431278,
 },
 expectedErr: "unmarshalling failed: unexpected EOF",
@dprotasodprotasoOct 16, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: any way to distinguish we hit the limit vs EOF on the wire?
@evankandersonevankandersonOct 18, 2023

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not easily. Technically, we could also end up skipping a bunch of extra garbage after a valid proto, but since the proto contains a length at the start, we can tell whether we read the whole proto or not just by reading the beginning. (In fact, it would be nice if the generated proto code had a method that read from an io.Reader, but it doesn't seem to.)
dprotaso
dprotaso
approved these changes
Oct 16, 2023
View reviewed changes
@dprotasodprotaso
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

/approve
@knative-prowknative-prowBot
added
the 
lgtm
Indicates that a PR is ready to be merged.
label
Oct 16, 2023
@knative-prow
knative-prowBot
commented
Oct 16, 2023

Copy link


Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: dprotaso, evankanderson
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Details
Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment

Approvers can cancel approval by writing /approve cancel in a comment
@knative-prowknative-prowBot
added
the 
approved
Indicates a PR has been approved by an approver from all required OWNERS files.
label
Oct 16, 2023
@knative-prow
knative-prowBot
merged commit fff40ef
into
knative:mainOct 16, 2023
@evankanderson
evankanderson
commented
Oct 18, 2023

Copy link


Copy Markdown

MemberAuthor
/cherrypick release-1.10

/cherrypick release-1.11
@knative-prow-robotknative-prow-robot
mentioned this pull request
Oct 18, 2023
 Merged

@knative-prow-robot
knative-prow-robot
commented
Oct 18, 2023

Copy link


Copy Markdown

Contributor
@evankanderson: new pull request created: #14541
Details
In response to this:
/cherrypick release-1.10

/cherrypick release-1.11
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@evankanderson
evankanderson
commented
Oct 18, 2023

Copy link


Copy Markdown

MemberAuthor
/cherrypick release-1.11
@knative-prow-robotknative-prow-robot
mentioned this pull request
Oct 18, 2023
 Merged

@knative-prow-robot
knative-prow-robot
commented
Oct 18, 2023

Copy link


Copy Markdown

Contributor
@evankanderson: new pull request created: #14542
Details
In response to this:
/cherrypick release-1.11
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
skonto
skonto
reviewed
Oct 19, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client.go
	defer pool.Put(b)
	_, err := b.ReadFrom(body)
	// 6 8-byte fields (+2 bytes marshalling), one hostname, 20 bytes extra space
	r := io.LimitedReader{R: body, N: 6*10 + 256 + 20}
@skontoskontoOct 19, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Should not this be 255 for the max hostname? 🤔
@evankandersonevankandersonOct 19, 2023

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra-nit: you might need two bytes for the tag + length if the string is actually 256 bytes long, plus another possible tag + length for the entire message.
Proto doesn't have a way to express / enforce that a string is less than a certain size. The goal here is simply to avoid accidentally reading e.g. 5MB or more into an in-memory buffer when we know the actual report should be less than 1KB.
@skontoskontoOct 20, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity (haven't checked), what about using grpc instead?

This was referenced Jan 24, 2024
 Merged


Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@skontoskontoskonto left review comments
@dprotasodprotasodprotaso approved these changes
@jsanin-vmwjsanin-vmwAwaiting requested review from jsanin-vmw
@ReToCodeReToCodeAwaiting requested review from ReToCode

Assignees

@dprotasodprotaso

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/autoscalelgtmIndicates that a PR is ready to be merged.size/SDenotes a PR that changes 10-29 lines, ignoring generated files.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@evankanderson@knative-prow-robot@dprotaso@skonto

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Bound buffer for reading stats - #14523
Merged
knative-prow[bot] merged 1 commit into
knative:mainfrom 
evankanderson:scaling-bounds
Oct 16, 2023
Merged
Bound buffer for reading stats#14523
knative-prow[bot] merged 1 commit into
knative:mainfrom 
evankanderson:scaling-bounds
Conversation
@evankanderson
@evankandersonevankanderson
commented
Oct 16, 2023

Copy link


Copy Markdown

Member
Proposed Changes
  • Bound the bytes read from the queue-proxy by the autoscaler.
Release Note
Applied an upper bound to the statistics data read from the queue-proxy by the autoscaler.

@knative-prowknative-prowBot
added

size/S
Denotes a PR that changes 10-29 lines, ignoring generated files.
area/autoscale

labels
Oct 16, 2023
@codecov
codecovBot
commented
Oct 16, 2023•

edited

Loading

Copy link


Copy Markdown

Codecov Report
All modified lines are covered by tests ✅
Comparison is base (6703812) 86.08% compared to head (588ea26) 86.13%.

Report is 7 commits behind head on main.
Additional details and impacted files
@@ Coverage Diff @@## main #14523 +/- ##
==========================================
+ Coverage 86.08% 86.13% +0.04% 
==========================================
Files 196 196 Lines 14887 14889 +2 ==========================================
+ Hits 12815 12824 +9 + Misses 1761 1756 -5 + Partials 311 309 -2 
FilesCoverage Δ
pkg/autoscaler/metrics/http_scrape_client.go93.02% <100.00%> (+7.65%)⬆️
... and 4 files with indirect coverage changes
☔ View full report in Codecov by Sentry.

📢 Have feedback on the report? Share it here.
dprotaso
dprotaso
reviewed
Oct 16, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client_test.go
 ProcessUptime: 12345.678,
 Timestamp: 1697431278,
 },
 expectedErr: "unmarshalling failed: unexpected EOF",
@dprotasodprotasoOct 16, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: any way to distinguish we hit the limit vs EOF on the wire?
@evankandersonevankandersonOct 18, 2023

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not easily. Technically, we could also end up skipping a bunch of extra garbage after a valid proto, but since the proto contains a length at the start, we can tell whether we read the whole proto or not just by reading the beginning. (In fact, it would be nice if the generated proto code had a method that read from an io.Reader, but it doesn't seem to.)
dprotaso
dprotaso
approved these changes
Oct 16, 2023
View reviewed changes
@dprotasodprotaso
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

/approve
@knative-prowknative-prowBot
added
the 
lgtm
Indicates that a PR is ready to be merged.
label
Oct 16, 2023
@knative-prow
knative-prowBot
commented
Oct 16, 2023

Copy link


Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: dprotaso, evankanderson
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Details
Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment

Approvers can cancel approval by writing /approve cancel in a comment
@knative-prowknative-prowBot
added
the 
approved
Indicates a PR has been approved by an approver from all required OWNERS files.
label
Oct 16, 2023
@knative-prow
knative-prowBot
merged commit fff40ef
into
knative:mainOct 16, 2023
@evankanderson
evankanderson
commented
Oct 18, 2023

Copy link


Copy Markdown

MemberAuthor
/cherrypick release-1.10

/cherrypick release-1.11
@knative-prow-robotknative-prow-robot
mentioned this pull request
Oct 18, 2023
 Merged

@knative-prow-robot
knative-prow-robot
commented
Oct 18, 2023

Copy link


Copy Markdown

Contributor
@evankanderson: new pull request created: #14541
Details
In response to this:
/cherrypick release-1.10

/cherrypick release-1.11
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@evankanderson
evankanderson
commented
Oct 18, 2023

Copy link


Copy Markdown

MemberAuthor
/cherrypick release-1.11
@knative-prow-robotknative-prow-robot
mentioned this pull request
Oct 18, 2023
 Merged

@knative-prow-robot
knative-prow-robot
commented
Oct 18, 2023

Copy link


Copy Markdown

Contributor
@evankanderson: new pull request created: #14542
Details
In response to this:
/cherrypick release-1.11
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
skonto
skonto
reviewed
Oct 19, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client.go
	defer pool.Put(b)
	_, err := b.ReadFrom(body)
	// 6 8-byte fields (+2 bytes marshalling), one hostname, 20 bytes extra space
	r := io.LimitedReader{R: body, N: 6*10 + 256 + 20}
@skontoskontoOct 19, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Should not this be 255 for the max hostname? 🤔
@evankandersonevankandersonOct 19, 2023

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra-nit: you might need two bytes for the tag + length if the string is actually 256 bytes long, plus another possible tag + length for the entire message.
Proto doesn't have a way to express / enforce that a string is less than a certain size. The goal here is simply to avoid accidentally reading e.g. 5MB or more into an in-memory buffer when we know the actual report should be less than 1KB.
@skontoskontoOct 20, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity (haven't checked), what about using grpc instead?

This was referenced Jan 24, 2024
 Merged


Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@skontoskontoskonto left review comments
@dprotasodprotasodprotaso approved these changes
@jsanin-vmwjsanin-vmwAwaiting requested review from jsanin-vmw
@ReToCodeReToCodeAwaiting requested review from ReToCode

Assignees

@dprotasodprotaso

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/autoscalelgtmIndicates that a PR is ready to be merged.size/SDenotes a PR that changes 10-29 lines, ignoring generated files.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@evankanderson@knative-prow-robot@dprotaso@skonto

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Bound buffer for reading stats - #14523
Merged
knative-prow[bot] merged 1 commit into
knative:mainfrom 
evankanderson:scaling-bounds
Oct 16, 2023
Merged
Bound buffer for reading stats#14523
knative-prow[bot] merged 1 commit into
knative:mainfrom 
evankanderson:scaling-bounds
Conversation
@evankanderson
@evankandersonevankanderson
commented
Oct 16, 2023

Copy link


Copy Markdown

Member
Proposed Changes
  • Bound the bytes read from the queue-proxy by the autoscaler.
Release Note
Applied an upper bound to the statistics data read from the queue-proxy by the autoscaler.

@knative-prowknative-prowBot
added

size/S
Denotes a PR that changes 10-29 lines, ignoring generated files.
area/autoscale

labels
Oct 16, 2023
@codecov
codecovBot
commented
Oct 16, 2023•

edited

Loading

Copy link


Copy Markdown

Codecov Report
All modified lines are covered by tests ✅
Comparison is base (6703812) 86.08% compared to head (588ea26) 86.13%.

Report is 7 commits behind head on main.
Additional details and impacted files
@@ Coverage Diff @@## main #14523 +/- ##
==========================================
+ Coverage 86.08% 86.13% +0.04% 
==========================================
Files 196 196 Lines 14887 14889 +2 ==========================================
+ Hits 12815 12824 +9 + Misses 1761 1756 -5 + Partials 311 309 -2 
FilesCoverage Δ
pkg/autoscaler/metrics/http_scrape_client.go93.02% <100.00%> (+7.65%)⬆️
... and 4 files with indirect coverage changes
☔ View full report in Codecov by Sentry.

📢 Have feedback on the report? Share it here.
dprotaso
dprotaso
reviewed
Oct 16, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client_test.go
 ProcessUptime: 12345.678,
 Timestamp: 1697431278,
 },
 expectedErr: "unmarshalling failed: unexpected EOF",
@dprotasodprotasoOct 16, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: any way to distinguish we hit the limit vs EOF on the wire?
@evankandersonevankandersonOct 18, 2023

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not easily. Technically, we could also end up skipping a bunch of extra garbage after a valid proto, but since the proto contains a length at the start, we can tell whether we read the whole proto or not just by reading the beginning. (In fact, it would be nice if the generated proto code had a method that read from an io.Reader, but it doesn't seem to.)
dprotaso
dprotaso
approved these changes
Oct 16, 2023
View reviewed changes
@dprotasodprotaso
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

/approve
@knative-prowknative-prowBot
added
the 
lgtm
Indicates that a PR is ready to be merged.
label
Oct 16, 2023
@knative-prow
knative-prowBot
commented
Oct 16, 2023

Copy link


Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: dprotaso, evankanderson
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Details
Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment

Approvers can cancel approval by writing /approve cancel in a comment
@knative-prowknative-prowBot
added
the 
approved
Indicates a PR has been approved by an approver from all required OWNERS files.
label
Oct 16, 2023
@knative-prow
knative-prowBot
merged commit fff40ef
into
knative:mainOct 16, 2023
@evankanderson
evankanderson
commented
Oct 18, 2023

Copy link


Copy Markdown

MemberAuthor
/cherrypick release-1.10

/cherrypick release-1.11
@knative-prow-robotknative-prow-robot
mentioned this pull request
Oct 18, 2023
 Merged

@knative-prow-robot
knative-prow-robot
commented
Oct 18, 2023

Copy link


Copy Markdown

Contributor
@evankanderson: new pull request created: #14541
Details
In response to this:
/cherrypick release-1.10

/cherrypick release-1.11
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@evankanderson
evankanderson
commented
Oct 18, 2023

Copy link


Copy Markdown

MemberAuthor
/cherrypick release-1.11
@knative-prow-robotknative-prow-robot
mentioned this pull request
Oct 18, 2023
 Merged

@knative-prow-robot
knative-prow-robot
commented
Oct 18, 2023

Copy link


Copy Markdown

Contributor
@evankanderson: new pull request created: #14542
Details
In response to this:
/cherrypick release-1.11
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
skonto
skonto
reviewed
Oct 19, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client.go
	defer pool.Put(b)
	_, err := b.ReadFrom(body)
	// 6 8-byte fields (+2 bytes marshalling), one hostname, 20 bytes extra space
	r := io.LimitedReader{R: body, N: 6*10 + 256 + 20}
@skontoskontoOct 19, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Should not this be 255 for the max hostname? 🤔
@evankandersonevankandersonOct 19, 2023

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra-nit: you might need two bytes for the tag + length if the string is actually 256 bytes long, plus another possible tag + length for the entire message.
Proto doesn't have a way to express / enforce that a string is less than a certain size. The goal here is simply to avoid accidentally reading e.g. 5MB or more into an in-memory buffer when we know the actual report should be less than 1KB.
@skontoskontoOct 20, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity (haven't checked), what about using grpc instead?

This was referenced Jan 24, 2024
 Merged


Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@skontoskontoskonto left review comments
@dprotasodprotasodprotaso approved these changes
@jsanin-vmwjsanin-vmwAwaiting requested review from jsanin-vmw
@ReToCodeReToCodeAwaiting requested review from ReToCode

Assignees

@dprotasodprotaso

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/autoscalelgtmIndicates that a PR is ready to be merged.size/SDenotes a PR that changes 10-29 lines, ignoring generated files.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@evankanderson@knative-prow-robot@dprotaso@skonto

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Bound buffer for reading stats - #14523
Merged
knative-prow[bot] merged 1 commit into
knative:mainfrom 
evankanderson:scaling-bounds
Oct 16, 2023
Merged
Bound buffer for reading stats#14523
knative-prow[bot] merged 1 commit into
knative:mainfrom 
evankanderson:scaling-bounds
Conversation
@evankanderson
@evankandersonevankanderson
commented
Oct 16, 2023

Copy link


Copy Markdown

Member
Proposed Changes
  • Bound the bytes read from the queue-proxy by the autoscaler.
Release Note
Applied an upper bound to the statistics data read from the queue-proxy by the autoscaler.

@knative-prowknative-prowBot
added

size/S
Denotes a PR that changes 10-29 lines, ignoring generated files.
area/autoscale

labels
Oct 16, 2023
@codecov
codecovBot
commented
Oct 16, 2023•

edited

Loading

Copy link


Copy Markdown

Codecov Report
All modified lines are covered by tests ✅
Comparison is base (6703812) 86.08% compared to head (588ea26) 86.13%.

Report is 7 commits behind head on main.
Additional details and impacted files
@@ Coverage Diff @@## main #14523 +/- ##
==========================================
+ Coverage 86.08% 86.13% +0.04% 
==========================================
Files 196 196 Lines 14887 14889 +2 ==========================================
+ Hits 12815 12824 +9 + Misses 1761 1756 -5 + Partials 311 309 -2 
FilesCoverage Δ
pkg/autoscaler/metrics/http_scrape_client.go93.02% <100.00%> (+7.65%)⬆️
... and 4 files with indirect coverage changes
☔ View full report in Codecov by Sentry.

📢 Have feedback on the report? Share it here.
dprotaso
dprotaso
reviewed
Oct 16, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client_test.go
 ProcessUptime: 12345.678,
 Timestamp: 1697431278,
 },
 expectedErr: "unmarshalling failed: unexpected EOF",
@dprotasodprotasoOct 16, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: any way to distinguish we hit the limit vs EOF on the wire?
@evankandersonevankandersonOct 18, 2023

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not easily. Technically, we could also end up skipping a bunch of extra garbage after a valid proto, but since the proto contains a length at the start, we can tell whether we read the whole proto or not just by reading the beginning. (In fact, it would be nice if the generated proto code had a method that read from an io.Reader, but it doesn't seem to.)
dprotaso
dprotaso
approved these changes
Oct 16, 2023
View reviewed changes
@dprotasodprotaso
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

/approve
@knative-prowknative-prowBot
added
the 
lgtm
Indicates that a PR is ready to be merged.
label
Oct 16, 2023
@knative-prow
knative-prowBot
commented
Oct 16, 2023

Copy link


Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: dprotaso, evankanderson
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Details
Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment

Approvers can cancel approval by writing /approve cancel in a comment
@knative-prowknative-prowBot
added
the 
approved
Indicates a PR has been approved by an approver from all required OWNERS files.
label
Oct 16, 2023
@knative-prow
knative-prowBot
merged commit fff40ef
into
knative:mainOct 16, 2023
@evankanderson
evankanderson
commented
Oct 18, 2023

Copy link


Copy Markdown

MemberAuthor
/cherrypick release-1.10

/cherrypick release-1.11
@knative-prow-robotknative-prow-robot
mentioned this pull request
Oct 18, 2023
 Merged

@knative-prow-robot
knative-prow-robot
commented
Oct 18, 2023

Copy link


Copy Markdown

Contributor
@evankanderson: new pull request created: #14541
Details
In response to this:
/cherrypick release-1.10

/cherrypick release-1.11
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@evankanderson
evankanderson
commented
Oct 18, 2023

Copy link


Copy Markdown

MemberAuthor
/cherrypick release-1.11
@knative-prow-robotknative-prow-robot
mentioned this pull request
Oct 18, 2023
 Merged

@knative-prow-robot
knative-prow-robot
commented
Oct 18, 2023

Copy link


Copy Markdown

Contributor
@evankanderson: new pull request created: #14542
Details
In response to this:
/cherrypick release-1.11
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
skonto
skonto
reviewed
Oct 19, 2023
View reviewed changes
Comment threadpkg/autoscaler/metrics/http_scrape_client.go
	defer pool.Put(b)
	_, err := b.ReadFrom(body)
	// 6 8-byte fields (+2 bytes marshalling), one hostname, 20 bytes extra space
	r := io.LimitedReader{R: body, N: 6*10 + 256 + 20}
@skontoskontoOct 19, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Should not this be 255 for the max hostname? 🤔
@evankandersonevankandersonOct 19, 2023

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra-nit: you might need two bytes for the tag + length if the string is actually 256 bytes long, plus another possible tag + length for the entire message.
Proto doesn't have a way to express / enforce that a string is less than a certain size. The goal here is simply to avoid accidentally reading e.g. 5MB or more into an in-memory buffer when we know the actual report should be less than 1KB.
@skontoskontoOct 20, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity (haven't checked), what about using grpc instead?

This was referenced Jan 24, 2024
 Merged


Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@skontoskontoskonto left review comments
@dprotasodprotasodprotaso approved these changes
@jsanin-vmwjsanin-vmwAwaiting requested review from jsanin-vmw
@ReToCodeReToCodeAwaiting requested review from ReToCode

Assignees

@dprotasodprotaso

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/autoscalelgtmIndicates that a PR is ready to be merged.size/SDenotes a PR that changes 10-29 lines, ignoring generated files.

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@evankanderson@knative-prow-robot@dprotaso@skonto