Run queue proxy with restricted profile - #13376

Merged
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue
Oct 18, 2022
Merged

Run queue proxy with restricted profile#13376
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue

Conversation

@skonto

@skontoskonto commented Oct 10, 2022

Copy link
Copy Markdown
Contributor

Fixes partially #13308

Proposed Changes

  • Allows to run a ksvc in a namespace with restricted profile enabled. Without this patch we get the following issue on 1.25 (see instructions next):

Warning FailedCreate replicaset/helloworld-go-00001-deployment-584759b77b (combined from similar events): Error creating: pods "helloworld-go-00001-deployment-584759b77b-tmxxd" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "user-container" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "queue-proxy" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or containers "user-container", "queue-proxy" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

  • To test follow instructions here.
  • Discussed here.

Release Note

Queue proxy explicit set `SeccompProfile` to `RunTimeDefault` to be able to run under restricted PSP policy by default.

@knative-prowknative-prowBot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. area/API API objects and controllers labels Oct 10, 2022

// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

@skontoskontoOct 10, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't allow the user to set the following two properties. So we enforce the defaults here to make it pass the PSP auditing.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I sent #13401 to allow setting this in the Revision Spec (e.g. in Service).

@knative-prowknative-prowBot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 10, 2022
@codecov

codecovBot commented Oct 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 86.47% // Head: 86.47% // No change to project coverage 👍

Coverage data is based on head (f3360cf) compared to base (6264c1b).
Patch has no changes to coverable lines.

Additional details and impacted files
@@ Coverage Diff @@## main #13376 +/- ##
=======================================
Coverage 86.47% 86.47% =======================================
Files 196 196 Lines 14551 14551 =======================================
Hits 12583 12583 Misses 1669 1669 Partials 299 299 
Impacted FilesCoverage Δ
pkg/reconciler/revision/resources/queue.go98.23% <ø> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@skontoskonto changed the title [wip] Allow user workloads to run with restricted profileAllow user workloads to run with restricted profileOct 11, 2022
@knative-prowknative-prowBot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 11, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minor things, but otherwise this looks good to me:

  • could you add a release note?
  • this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

If you mean the gateway yes. I will take a look to fix it. Downstream we already run without root.

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

@psschwei gentle ping, added the release note and created knative-extensions/net-kourier#934.

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve
/hold

to give @dprotaso a chance to weigh in

@knative-prowknative-prowBot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 13, 2022
@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 13, 2022
@knative-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: psschwei, skonto

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prowknative-prowBot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 13, 2022

@evankandersonevankanderson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may want to pass-through the allowPrivilegeEscalation and seccompProfile fields, and then add a feature flag to default them to safe values if not provided.

I hadn't hit on this until the other day seeming @mattmoor 's #13395 , but I think I'd feel most happy long term with "didn't set anything" meaning secure, but allowing people to explicitly be insecure when needed (maybe with warnings)


// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

@skonto

skonto commented Oct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

@evankanderson I also mentioned on slack that allowing users to configure stuff on their own was more user friendly than having everything explicitly set without ability to change on demand. Also as you said users can choose their security level. For some properties though you can be opinionated and set explicitly the defaults instead of optionally applying them as in #13398 as long as users can remove them. This can be done, given our experience of what users need in practice eg. usually you dont want a service to escalate privileges. Another approach is to define security application as a downstream issue only. We have options depending on the strategy, do we want to be secure by default? My understanding is that this concept applies to other features like internal-encryption etc. Btw I am not sure if @mattmoor was aware of this PR or discussion. Should I close this PR and enable all stuff in #13398 or keep this PR with the queue proxy changes only?

(maybe with warnings)

You get the warning anyways from psp no need to add more imho.

Drop: []corev1.Capability{"all"},
Drop: []corev1.Capability{"ALL"},
},
SeccompProfile: &corev1.SeccompProfile{

@skontoskontoOct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this takes precedence compared to podsecurityContext.SeccompProfile.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it does.

@evankanderson

Copy link
Copy Markdown
Member

Can users see the PSP / PSA warnings when they apply a Knative Service? If so, then I'd agree about not needing a second set of warnings that settings are possibly dangerous.

In #13399, I tried to add a warning because our future behavior might change, which could break a small number of applications.

Upon reflection, I think giving the defaulting of security properties time to sit rather than rushing it for release is probably the right idea, particularly given the issues hit by #13399.

@evankanderson

Copy link
Copy Markdown
Member

I'm still willing to approve the queue-proxy side of this, but I don't want to add a breaking behavioral change without some sort of "out" for users.

@knative-prowknative-prowBot removed the lgtm Indicates that a PR is ready to be merged. label Oct 17, 2022
@knative-prowknative-prowBot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 17, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

/test istio-latest-no-mesh

@psschweipsschwei changed the title Allow user workloads to run with restricted profileRun queue proxy with restricted profileOct 18, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @skonto I updated the PR title / release note since these changes are just on QP now.
/lgtm

@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 18, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

@psschwei thank you, ok to unhold?

@psschwei

Copy link
Copy Markdown
Member

/hold cancel

@knative-prowknative-prowBot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 18, 2022
@knative-prow
knative-prowBot merged commit 388128b into knative:mainOct 18, 2022
skonto pushed a commit to skonto/serving that referenced this pull request Oct 19, 2022
* allow user workloads to run with restricted profile
* only change queue proxy
openshift-merge-robot pushed a commit to openshift/knative-serving that referenced this pull request Oct 27, 2022
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
skonto pushed a commit to skonto/serving that referenced this pull request Nov 15, 2022
…ve#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 16, 2022
#13)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/knative-serving that referenced this pull request Nov 16, 2022
…hift#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 18, 2022
#19)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
Co-authored-by: Stavros Kontopoulos <skontopo@redhat.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/APIAPI objects and controllerslgtmIndicates that a PR is ready to be merged.size/XSDenotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@skonto@evankanderson@psschwei
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Run queue proxy with restricted profile - #13376

Merged
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue
Oct 18, 2022
Merged

Run queue proxy with restricted profile#13376
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue

Conversation

@skonto

@skontoskonto commented Oct 10, 2022

Copy link
Copy Markdown
Contributor

Fixes partially #13308

Proposed Changes

  • Allows to run a ksvc in a namespace with restricted profile enabled. Without this patch we get the following issue on 1.25 (see instructions next):

Warning FailedCreate replicaset/helloworld-go-00001-deployment-584759b77b (combined from similar events): Error creating: pods "helloworld-go-00001-deployment-584759b77b-tmxxd" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "user-container" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "queue-proxy" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or containers "user-container", "queue-proxy" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

  • To test follow instructions here.
  • Discussed here.

Release Note

Queue proxy explicit set `SeccompProfile` to `RunTimeDefault` to be able to run under restricted PSP policy by default.

@knative-prowknative-prowBot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. area/API API objects and controllers labels Oct 10, 2022

// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

@skontoskontoOct 10, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't allow the user to set the following two properties. So we enforce the defaults here to make it pass the PSP auditing.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I sent #13401 to allow setting this in the Revision Spec (e.g. in Service).

@knative-prowknative-prowBot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 10, 2022
@codecov

codecovBot commented Oct 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 86.47% // Head: 86.47% // No change to project coverage 👍

Coverage data is based on head (f3360cf) compared to base (6264c1b).
Patch has no changes to coverable lines.

Additional details and impacted files
@@ Coverage Diff @@## main #13376 +/- ##
=======================================
Coverage 86.47% 86.47% =======================================
Files 196 196 Lines 14551 14551 =======================================
Hits 12583 12583 Misses 1669 1669 Partials 299 299 
Impacted FilesCoverage Δ
pkg/reconciler/revision/resources/queue.go98.23% <ø> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@skontoskonto changed the title [wip] Allow user workloads to run with restricted profileAllow user workloads to run with restricted profileOct 11, 2022
@knative-prowknative-prowBot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 11, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minor things, but otherwise this looks good to me:

  • could you add a release note?
  • this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

If you mean the gateway yes. I will take a look to fix it. Downstream we already run without root.

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

@psschwei gentle ping, added the release note and created knative-extensions/net-kourier#934.

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve
/hold

to give @dprotaso a chance to weigh in

@knative-prowknative-prowBot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 13, 2022
@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 13, 2022
@knative-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: psschwei, skonto

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prowknative-prowBot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 13, 2022

@evankandersonevankanderson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may want to pass-through the allowPrivilegeEscalation and seccompProfile fields, and then add a feature flag to default them to safe values if not provided.

I hadn't hit on this until the other day seeming @mattmoor 's #13395 , but I think I'd feel most happy long term with "didn't set anything" meaning secure, but allowing people to explicitly be insecure when needed (maybe with warnings)


// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

@skonto

skonto commented Oct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

@evankanderson I also mentioned on slack that allowing users to configure stuff on their own was more user friendly than having everything explicitly set without ability to change on demand. Also as you said users can choose their security level. For some properties though you can be opinionated and set explicitly the defaults instead of optionally applying them as in #13398 as long as users can remove them. This can be done, given our experience of what users need in practice eg. usually you dont want a service to escalate privileges. Another approach is to define security application as a downstream issue only. We have options depending on the strategy, do we want to be secure by default? My understanding is that this concept applies to other features like internal-encryption etc. Btw I am not sure if @mattmoor was aware of this PR or discussion. Should I close this PR and enable all stuff in #13398 or keep this PR with the queue proxy changes only?

(maybe with warnings)

You get the warning anyways from psp no need to add more imho.

Drop: []corev1.Capability{"all"},
Drop: []corev1.Capability{"ALL"},
},
SeccompProfile: &corev1.SeccompProfile{

@skontoskontoOct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this takes precedence compared to podsecurityContext.SeccompProfile.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it does.

@evankanderson

Copy link
Copy Markdown
Member

Can users see the PSP / PSA warnings when they apply a Knative Service? If so, then I'd agree about not needing a second set of warnings that settings are possibly dangerous.

In #13399, I tried to add a warning because our future behavior might change, which could break a small number of applications.

Upon reflection, I think giving the defaulting of security properties time to sit rather than rushing it for release is probably the right idea, particularly given the issues hit by #13399.

@evankanderson

Copy link
Copy Markdown
Member

I'm still willing to approve the queue-proxy side of this, but I don't want to add a breaking behavioral change without some sort of "out" for users.

@knative-prowknative-prowBot removed the lgtm Indicates that a PR is ready to be merged. label Oct 17, 2022
@knative-prowknative-prowBot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 17, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

/test istio-latest-no-mesh

@psschweipsschwei changed the title Allow user workloads to run with restricted profileRun queue proxy with restricted profileOct 18, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @skonto I updated the PR title / release note since these changes are just on QP now.
/lgtm

@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 18, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

@psschwei thank you, ok to unhold?

@psschwei

Copy link
Copy Markdown
Member

/hold cancel

@knative-prowknative-prowBot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 18, 2022
@knative-prow
knative-prowBot merged commit 388128b into knative:mainOct 18, 2022
skonto pushed a commit to skonto/serving that referenced this pull request Oct 19, 2022
* allow user workloads to run with restricted profile
* only change queue proxy
openshift-merge-robot pushed a commit to openshift/knative-serving that referenced this pull request Oct 27, 2022
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
skonto pushed a commit to skonto/serving that referenced this pull request Nov 15, 2022
…ve#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 16, 2022
#13)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/knative-serving that referenced this pull request Nov 16, 2022
…hift#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 18, 2022
#19)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
Co-authored-by: Stavros Kontopoulos <skontopo@redhat.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/APIAPI objects and controllerslgtmIndicates that a PR is ready to be merged.size/XSDenotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@skonto@evankanderson@psschwei
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Run queue proxy with restricted profile - #13376

Merged
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue
Oct 18, 2022
Merged

Run queue proxy with restricted profile#13376
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue

Conversation

@skonto

@skontoskonto commented Oct 10, 2022

Copy link
Copy Markdown
Contributor

Fixes partially #13308

Proposed Changes

  • Allows to run a ksvc in a namespace with restricted profile enabled. Without this patch we get the following issue on 1.25 (see instructions next):

Warning FailedCreate replicaset/helloworld-go-00001-deployment-584759b77b (combined from similar events): Error creating: pods "helloworld-go-00001-deployment-584759b77b-tmxxd" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "user-container" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "queue-proxy" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or containers "user-container", "queue-proxy" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

  • To test follow instructions here.
  • Discussed here.

Release Note

Queue proxy explicit set `SeccompProfile` to `RunTimeDefault` to be able to run under restricted PSP policy by default.

@knative-prowknative-prowBot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. area/API API objects and controllers labels Oct 10, 2022

// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

@skontoskontoOct 10, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't allow the user to set the following two properties. So we enforce the defaults here to make it pass the PSP auditing.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I sent #13401 to allow setting this in the Revision Spec (e.g. in Service).

@knative-prowknative-prowBot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 10, 2022
@codecov

codecovBot commented Oct 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 86.47% // Head: 86.47% // No change to project coverage 👍

Coverage data is based on head (f3360cf) compared to base (6264c1b).
Patch has no changes to coverable lines.

Additional details and impacted files
@@ Coverage Diff @@## main #13376 +/- ##
=======================================
Coverage 86.47% 86.47% =======================================
Files 196 196 Lines 14551 14551 =======================================
Hits 12583 12583 Misses 1669 1669 Partials 299 299 
Impacted FilesCoverage Δ
pkg/reconciler/revision/resources/queue.go98.23% <ø> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@skontoskonto changed the title [wip] Allow user workloads to run with restricted profileAllow user workloads to run with restricted profileOct 11, 2022
@knative-prowknative-prowBot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 11, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minor things, but otherwise this looks good to me:

  • could you add a release note?
  • this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

If you mean the gateway yes. I will take a look to fix it. Downstream we already run without root.

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

@psschwei gentle ping, added the release note and created knative-extensions/net-kourier#934.

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve
/hold

to give @dprotaso a chance to weigh in

@knative-prowknative-prowBot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 13, 2022
@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 13, 2022
@knative-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: psschwei, skonto

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prowknative-prowBot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 13, 2022

@evankandersonevankanderson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may want to pass-through the allowPrivilegeEscalation and seccompProfile fields, and then add a feature flag to default them to safe values if not provided.

I hadn't hit on this until the other day seeming @mattmoor 's #13395 , but I think I'd feel most happy long term with "didn't set anything" meaning secure, but allowing people to explicitly be insecure when needed (maybe with warnings)


// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

@skonto

skonto commented Oct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

@evankanderson I also mentioned on slack that allowing users to configure stuff on their own was more user friendly than having everything explicitly set without ability to change on demand. Also as you said users can choose their security level. For some properties though you can be opinionated and set explicitly the defaults instead of optionally applying them as in #13398 as long as users can remove them. This can be done, given our experience of what users need in practice eg. usually you dont want a service to escalate privileges. Another approach is to define security application as a downstream issue only. We have options depending on the strategy, do we want to be secure by default? My understanding is that this concept applies to other features like internal-encryption etc. Btw I am not sure if @mattmoor was aware of this PR or discussion. Should I close this PR and enable all stuff in #13398 or keep this PR with the queue proxy changes only?

(maybe with warnings)

You get the warning anyways from psp no need to add more imho.

Drop: []corev1.Capability{"all"},
Drop: []corev1.Capability{"ALL"},
},
SeccompProfile: &corev1.SeccompProfile{

@skontoskontoOct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this takes precedence compared to podsecurityContext.SeccompProfile.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it does.

@evankanderson

Copy link
Copy Markdown
Member

Can users see the PSP / PSA warnings when they apply a Knative Service? If so, then I'd agree about not needing a second set of warnings that settings are possibly dangerous.

In #13399, I tried to add a warning because our future behavior might change, which could break a small number of applications.

Upon reflection, I think giving the defaulting of security properties time to sit rather than rushing it for release is probably the right idea, particularly given the issues hit by #13399.

@evankanderson

Copy link
Copy Markdown
Member

I'm still willing to approve the queue-proxy side of this, but I don't want to add a breaking behavioral change without some sort of "out" for users.

@knative-prowknative-prowBot removed the lgtm Indicates that a PR is ready to be merged. label Oct 17, 2022
@knative-prowknative-prowBot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 17, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

/test istio-latest-no-mesh

@psschweipsschwei changed the title Allow user workloads to run with restricted profileRun queue proxy with restricted profileOct 18, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @skonto I updated the PR title / release note since these changes are just on QP now.
/lgtm

@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 18, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

@psschwei thank you, ok to unhold?

@psschwei

Copy link
Copy Markdown
Member

/hold cancel

@knative-prowknative-prowBot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 18, 2022
@knative-prow
knative-prowBot merged commit 388128b into knative:mainOct 18, 2022
skonto pushed a commit to skonto/serving that referenced this pull request Oct 19, 2022
* allow user workloads to run with restricted profile
* only change queue proxy
openshift-merge-robot pushed a commit to openshift/knative-serving that referenced this pull request Oct 27, 2022
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
skonto pushed a commit to skonto/serving that referenced this pull request Nov 15, 2022
…ve#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 16, 2022
#13)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/knative-serving that referenced this pull request Nov 16, 2022
…hift#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 18, 2022
#19)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
Co-authored-by: Stavros Kontopoulos <skontopo@redhat.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/APIAPI objects and controllerslgtmIndicates that a PR is ready to be merged.size/XSDenotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@skonto@evankanderson@psschwei
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Run queue proxy with restricted profile - #13376

Merged
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue
Oct 18, 2022
Merged

Run queue proxy with restricted profile#13376
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue

Conversation

@skonto

@skontoskonto commented Oct 10, 2022

Copy link
Copy Markdown
Contributor

Fixes partially #13308

Proposed Changes

  • Allows to run a ksvc in a namespace with restricted profile enabled. Without this patch we get the following issue on 1.25 (see instructions next):

Warning FailedCreate replicaset/helloworld-go-00001-deployment-584759b77b (combined from similar events): Error creating: pods "helloworld-go-00001-deployment-584759b77b-tmxxd" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "user-container" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "queue-proxy" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or containers "user-container", "queue-proxy" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

  • To test follow instructions here.
  • Discussed here.

Release Note

Queue proxy explicit set `SeccompProfile` to `RunTimeDefault` to be able to run under restricted PSP policy by default.

@knative-prowknative-prowBot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. area/API API objects and controllers labels Oct 10, 2022

// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

@skontoskontoOct 10, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't allow the user to set the following two properties. So we enforce the defaults here to make it pass the PSP auditing.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I sent #13401 to allow setting this in the Revision Spec (e.g. in Service).

@knative-prowknative-prowBot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 10, 2022
@codecov

codecovBot commented Oct 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 86.47% // Head: 86.47% // No change to project coverage 👍

Coverage data is based on head (f3360cf) compared to base (6264c1b).
Patch has no changes to coverable lines.

Additional details and impacted files
@@ Coverage Diff @@## main #13376 +/- ##
=======================================
Coverage 86.47% 86.47% =======================================
Files 196 196 Lines 14551 14551 =======================================
Hits 12583 12583 Misses 1669 1669 Partials 299 299 
Impacted FilesCoverage Δ
pkg/reconciler/revision/resources/queue.go98.23% <ø> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@skontoskonto changed the title [wip] Allow user workloads to run with restricted profileAllow user workloads to run with restricted profileOct 11, 2022
@knative-prowknative-prowBot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 11, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minor things, but otherwise this looks good to me:

  • could you add a release note?
  • this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

If you mean the gateway yes. I will take a look to fix it. Downstream we already run without root.

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

@psschwei gentle ping, added the release note and created knative-extensions/net-kourier#934.

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve
/hold

to give @dprotaso a chance to weigh in

@knative-prowknative-prowBot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 13, 2022
@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 13, 2022
@knative-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: psschwei, skonto

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prowknative-prowBot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 13, 2022

@evankandersonevankanderson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may want to pass-through the allowPrivilegeEscalation and seccompProfile fields, and then add a feature flag to default them to safe values if not provided.

I hadn't hit on this until the other day seeming @mattmoor 's #13395 , but I think I'd feel most happy long term with "didn't set anything" meaning secure, but allowing people to explicitly be insecure when needed (maybe with warnings)


// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

@skonto

skonto commented Oct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

@evankanderson I also mentioned on slack that allowing users to configure stuff on their own was more user friendly than having everything explicitly set without ability to change on demand. Also as you said users can choose their security level. For some properties though you can be opinionated and set explicitly the defaults instead of optionally applying them as in #13398 as long as users can remove them. This can be done, given our experience of what users need in practice eg. usually you dont want a service to escalate privileges. Another approach is to define security application as a downstream issue only. We have options depending on the strategy, do we want to be secure by default? My understanding is that this concept applies to other features like internal-encryption etc. Btw I am not sure if @mattmoor was aware of this PR or discussion. Should I close this PR and enable all stuff in #13398 or keep this PR with the queue proxy changes only?

(maybe with warnings)

You get the warning anyways from psp no need to add more imho.

Drop: []corev1.Capability{"all"},
Drop: []corev1.Capability{"ALL"},
},
SeccompProfile: &corev1.SeccompProfile{

@skontoskontoOct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this takes precedence compared to podsecurityContext.SeccompProfile.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it does.

@evankanderson

Copy link
Copy Markdown
Member

Can users see the PSP / PSA warnings when they apply a Knative Service? If so, then I'd agree about not needing a second set of warnings that settings are possibly dangerous.

In #13399, I tried to add a warning because our future behavior might change, which could break a small number of applications.

Upon reflection, I think giving the defaulting of security properties time to sit rather than rushing it for release is probably the right idea, particularly given the issues hit by #13399.

@evankanderson

Copy link
Copy Markdown
Member

I'm still willing to approve the queue-proxy side of this, but I don't want to add a breaking behavioral change without some sort of "out" for users.

@knative-prowknative-prowBot removed the lgtm Indicates that a PR is ready to be merged. label Oct 17, 2022
@knative-prowknative-prowBot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 17, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

/test istio-latest-no-mesh

@psschweipsschwei changed the title Allow user workloads to run with restricted profileRun queue proxy with restricted profileOct 18, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @skonto I updated the PR title / release note since these changes are just on QP now.
/lgtm

@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 18, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

@psschwei thank you, ok to unhold?

@psschwei

Copy link
Copy Markdown
Member

/hold cancel

@knative-prowknative-prowBot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 18, 2022
@knative-prow
knative-prowBot merged commit 388128b into knative:mainOct 18, 2022
skonto pushed a commit to skonto/serving that referenced this pull request Oct 19, 2022
* allow user workloads to run with restricted profile
* only change queue proxy
openshift-merge-robot pushed a commit to openshift/knative-serving that referenced this pull request Oct 27, 2022
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
skonto pushed a commit to skonto/serving that referenced this pull request Nov 15, 2022
…ve#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 16, 2022
#13)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/knative-serving that referenced this pull request Nov 16, 2022
…hift#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 18, 2022
#19)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
Co-authored-by: Stavros Kontopoulos <skontopo@redhat.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/APIAPI objects and controllerslgtmIndicates that a PR is ready to be merged.size/XSDenotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@skonto@evankanderson@psschwei
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Run queue proxy with restricted profile - #13376

Merged
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue
Oct 18, 2022
Merged

Run queue proxy with restricted profile#13376
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue

Conversation

@skonto

@skontoskonto commented Oct 10, 2022

Copy link
Copy Markdown
Contributor

Fixes partially #13308

Proposed Changes

  • Allows to run a ksvc in a namespace with restricted profile enabled. Without this patch we get the following issue on 1.25 (see instructions next):

Warning FailedCreate replicaset/helloworld-go-00001-deployment-584759b77b (combined from similar events): Error creating: pods "helloworld-go-00001-deployment-584759b77b-tmxxd" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "user-container" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "queue-proxy" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or containers "user-container", "queue-proxy" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

  • To test follow instructions here.
  • Discussed here.

Release Note

Queue proxy explicit set `SeccompProfile` to `RunTimeDefault` to be able to run under restricted PSP policy by default.

@knative-prowknative-prowBot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. area/API API objects and controllers labels Oct 10, 2022

// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

@skontoskontoOct 10, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't allow the user to set the following two properties. So we enforce the defaults here to make it pass the PSP auditing.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I sent #13401 to allow setting this in the Revision Spec (e.g. in Service).

@knative-prowknative-prowBot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 10, 2022
@codecov

codecovBot commented Oct 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 86.47% // Head: 86.47% // No change to project coverage 👍

Coverage data is based on head (f3360cf) compared to base (6264c1b).
Patch has no changes to coverable lines.

Additional details and impacted files
@@ Coverage Diff @@## main #13376 +/- ##
=======================================
Coverage 86.47% 86.47% =======================================
Files 196 196 Lines 14551 14551 =======================================
Hits 12583 12583 Misses 1669 1669 Partials 299 299 
Impacted FilesCoverage Δ
pkg/reconciler/revision/resources/queue.go98.23% <ø> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@skontoskonto changed the title [wip] Allow user workloads to run with restricted profileAllow user workloads to run with restricted profileOct 11, 2022
@knative-prowknative-prowBot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 11, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minor things, but otherwise this looks good to me:

  • could you add a release note?
  • this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

If you mean the gateway yes. I will take a look to fix it. Downstream we already run without root.

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

@psschwei gentle ping, added the release note and created knative-extensions/net-kourier#934.

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve
/hold

to give @dprotaso a chance to weigh in

@knative-prowknative-prowBot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 13, 2022
@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 13, 2022
@knative-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: psschwei, skonto

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prowknative-prowBot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 13, 2022

@evankandersonevankanderson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may want to pass-through the allowPrivilegeEscalation and seccompProfile fields, and then add a feature flag to default them to safe values if not provided.

I hadn't hit on this until the other day seeming @mattmoor 's #13395 , but I think I'd feel most happy long term with "didn't set anything" meaning secure, but allowing people to explicitly be insecure when needed (maybe with warnings)


// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

@skonto

skonto commented Oct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

@evankanderson I also mentioned on slack that allowing users to configure stuff on their own was more user friendly than having everything explicitly set without ability to change on demand. Also as you said users can choose their security level. For some properties though you can be opinionated and set explicitly the defaults instead of optionally applying them as in #13398 as long as users can remove them. This can be done, given our experience of what users need in practice eg. usually you dont want a service to escalate privileges. Another approach is to define security application as a downstream issue only. We have options depending on the strategy, do we want to be secure by default? My understanding is that this concept applies to other features like internal-encryption etc. Btw I am not sure if @mattmoor was aware of this PR or discussion. Should I close this PR and enable all stuff in #13398 or keep this PR with the queue proxy changes only?

(maybe with warnings)

You get the warning anyways from psp no need to add more imho.

Drop: []corev1.Capability{"all"},
Drop: []corev1.Capability{"ALL"},
},
SeccompProfile: &corev1.SeccompProfile{

@skontoskontoOct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this takes precedence compared to podsecurityContext.SeccompProfile.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it does.

@evankanderson

Copy link
Copy Markdown
Member

Can users see the PSP / PSA warnings when they apply a Knative Service? If so, then I'd agree about not needing a second set of warnings that settings are possibly dangerous.

In #13399, I tried to add a warning because our future behavior might change, which could break a small number of applications.

Upon reflection, I think giving the defaulting of security properties time to sit rather than rushing it for release is probably the right idea, particularly given the issues hit by #13399.

@evankanderson

Copy link
Copy Markdown
Member

I'm still willing to approve the queue-proxy side of this, but I don't want to add a breaking behavioral change without some sort of "out" for users.

@knative-prowknative-prowBot removed the lgtm Indicates that a PR is ready to be merged. label Oct 17, 2022
@knative-prowknative-prowBot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 17, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

/test istio-latest-no-mesh

@psschweipsschwei changed the title Allow user workloads to run with restricted profileRun queue proxy with restricted profileOct 18, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @skonto I updated the PR title / release note since these changes are just on QP now.
/lgtm

@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 18, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

@psschwei thank you, ok to unhold?

@psschwei

Copy link
Copy Markdown
Member

/hold cancel

@knative-prowknative-prowBot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 18, 2022
@knative-prow
knative-prowBot merged commit 388128b into knative:mainOct 18, 2022
skonto pushed a commit to skonto/serving that referenced this pull request Oct 19, 2022
* allow user workloads to run with restricted profile
* only change queue proxy
openshift-merge-robot pushed a commit to openshift/knative-serving that referenced this pull request Oct 27, 2022
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
skonto pushed a commit to skonto/serving that referenced this pull request Nov 15, 2022
…ve#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 16, 2022
#13)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/knative-serving that referenced this pull request Nov 16, 2022
…hift#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 18, 2022
#19)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
Co-authored-by: Stavros Kontopoulos <skontopo@redhat.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/APIAPI objects and controllerslgtmIndicates that a PR is ready to be merged.size/XSDenotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@skonto@evankanderson@psschwei
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Run queue proxy with restricted profile - #13376

Merged
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue
Oct 18, 2022
Merged

Run queue proxy with restricted profile#13376
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue

Conversation

@skonto

@skontoskonto commented Oct 10, 2022

Copy link
Copy Markdown
Contributor

Fixes partially #13308

Proposed Changes

  • Allows to run a ksvc in a namespace with restricted profile enabled. Without this patch we get the following issue on 1.25 (see instructions next):

Warning FailedCreate replicaset/helloworld-go-00001-deployment-584759b77b (combined from similar events): Error creating: pods "helloworld-go-00001-deployment-584759b77b-tmxxd" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "user-container" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "queue-proxy" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or containers "user-container", "queue-proxy" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

  • To test follow instructions here.
  • Discussed here.

Release Note

Queue proxy explicit set `SeccompProfile` to `RunTimeDefault` to be able to run under restricted PSP policy by default.

@knative-prowknative-prowBot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. area/API API objects and controllers labels Oct 10, 2022

// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

@skontoskontoOct 10, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't allow the user to set the following two properties. So we enforce the defaults here to make it pass the PSP auditing.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I sent #13401 to allow setting this in the Revision Spec (e.g. in Service).

@knative-prowknative-prowBot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 10, 2022
@codecov

codecovBot commented Oct 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 86.47% // Head: 86.47% // No change to project coverage 👍

Coverage data is based on head (f3360cf) compared to base (6264c1b).
Patch has no changes to coverable lines.

Additional details and impacted files
@@ Coverage Diff @@## main #13376 +/- ##
=======================================
Coverage 86.47% 86.47% =======================================
Files 196 196 Lines 14551 14551 =======================================
Hits 12583 12583 Misses 1669 1669 Partials 299 299 
Impacted FilesCoverage Δ
pkg/reconciler/revision/resources/queue.go98.23% <ø> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@skontoskonto changed the title [wip] Allow user workloads to run with restricted profileAllow user workloads to run with restricted profileOct 11, 2022
@knative-prowknative-prowBot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 11, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minor things, but otherwise this looks good to me:

  • could you add a release note?
  • this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

If you mean the gateway yes. I will take a look to fix it. Downstream we already run without root.

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

@psschwei gentle ping, added the release note and created knative-extensions/net-kourier#934.

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve
/hold

to give @dprotaso a chance to weigh in

@knative-prowknative-prowBot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 13, 2022
@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 13, 2022
@knative-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: psschwei, skonto

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prowknative-prowBot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 13, 2022

@evankandersonevankanderson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may want to pass-through the allowPrivilegeEscalation and seccompProfile fields, and then add a feature flag to default them to safe values if not provided.

I hadn't hit on this until the other day seeming @mattmoor 's #13395 , but I think I'd feel most happy long term with "didn't set anything" meaning secure, but allowing people to explicitly be insecure when needed (maybe with warnings)


// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

@skonto

skonto commented Oct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

@evankanderson I also mentioned on slack that allowing users to configure stuff on their own was more user friendly than having everything explicitly set without ability to change on demand. Also as you said users can choose their security level. For some properties though you can be opinionated and set explicitly the defaults instead of optionally applying them as in #13398 as long as users can remove them. This can be done, given our experience of what users need in practice eg. usually you dont want a service to escalate privileges. Another approach is to define security application as a downstream issue only. We have options depending on the strategy, do we want to be secure by default? My understanding is that this concept applies to other features like internal-encryption etc. Btw I am not sure if @mattmoor was aware of this PR or discussion. Should I close this PR and enable all stuff in #13398 or keep this PR with the queue proxy changes only?

(maybe with warnings)

You get the warning anyways from psp no need to add more imho.

Drop: []corev1.Capability{"all"},
Drop: []corev1.Capability{"ALL"},
},
SeccompProfile: &corev1.SeccompProfile{

@skontoskontoOct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this takes precedence compared to podsecurityContext.SeccompProfile.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it does.

@evankanderson

Copy link
Copy Markdown
Member

Can users see the PSP / PSA warnings when they apply a Knative Service? If so, then I'd agree about not needing a second set of warnings that settings are possibly dangerous.

In #13399, I tried to add a warning because our future behavior might change, which could break a small number of applications.

Upon reflection, I think giving the defaulting of security properties time to sit rather than rushing it for release is probably the right idea, particularly given the issues hit by #13399.

@evankanderson

Copy link
Copy Markdown
Member

I'm still willing to approve the queue-proxy side of this, but I don't want to add a breaking behavioral change without some sort of "out" for users.

@knative-prowknative-prowBot removed the lgtm Indicates that a PR is ready to be merged. label Oct 17, 2022
@knative-prowknative-prowBot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 17, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

/test istio-latest-no-mesh

@psschweipsschwei changed the title Allow user workloads to run with restricted profileRun queue proxy with restricted profileOct 18, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @skonto I updated the PR title / release note since these changes are just on QP now.
/lgtm

@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 18, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

@psschwei thank you, ok to unhold?

@psschwei

Copy link
Copy Markdown
Member

/hold cancel

@knative-prowknative-prowBot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 18, 2022
@knative-prow
knative-prowBot merged commit 388128b into knative:mainOct 18, 2022
skonto pushed a commit to skonto/serving that referenced this pull request Oct 19, 2022
* allow user workloads to run with restricted profile
* only change queue proxy
openshift-merge-robot pushed a commit to openshift/knative-serving that referenced this pull request Oct 27, 2022
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
skonto pushed a commit to skonto/serving that referenced this pull request Nov 15, 2022
…ve#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 16, 2022
#13)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/knative-serving that referenced this pull request Nov 16, 2022
…hift#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 18, 2022
#19)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
Co-authored-by: Stavros Kontopoulos <skontopo@redhat.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/APIAPI objects and controllerslgtmIndicates that a PR is ready to be merged.size/XSDenotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@skonto@evankanderson@psschwei
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Run queue proxy with restricted profile - #13376

Merged
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue
Oct 18, 2022
Merged

Run queue proxy with restricted profile#13376
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue

Conversation

@skonto

@skontoskonto commented Oct 10, 2022

Copy link
Copy Markdown
Contributor

Fixes partially #13308

Proposed Changes

  • Allows to run a ksvc in a namespace with restricted profile enabled. Without this patch we get the following issue on 1.25 (see instructions next):

Warning FailedCreate replicaset/helloworld-go-00001-deployment-584759b77b (combined from similar events): Error creating: pods "helloworld-go-00001-deployment-584759b77b-tmxxd" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "user-container" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "queue-proxy" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or containers "user-container", "queue-proxy" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

  • To test follow instructions here.
  • Discussed here.

Release Note

Queue proxy explicit set `SeccompProfile` to `RunTimeDefault` to be able to run under restricted PSP policy by default.

@knative-prowknative-prowBot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. area/API API objects and controllers labels Oct 10, 2022

// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

@skontoskontoOct 10, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't allow the user to set the following two properties. So we enforce the defaults here to make it pass the PSP auditing.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I sent #13401 to allow setting this in the Revision Spec (e.g. in Service).

@knative-prowknative-prowBot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 10, 2022
@codecov

codecovBot commented Oct 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 86.47% // Head: 86.47% // No change to project coverage 👍

Coverage data is based on head (f3360cf) compared to base (6264c1b).
Patch has no changes to coverable lines.

Additional details and impacted files
@@ Coverage Diff @@## main #13376 +/- ##
=======================================
Coverage 86.47% 86.47% =======================================
Files 196 196 Lines 14551 14551 =======================================
Hits 12583 12583 Misses 1669 1669 Partials 299 299 
Impacted FilesCoverage Δ
pkg/reconciler/revision/resources/queue.go98.23% <ø> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@skontoskonto changed the title [wip] Allow user workloads to run with restricted profileAllow user workloads to run with restricted profileOct 11, 2022
@knative-prowknative-prowBot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 11, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minor things, but otherwise this looks good to me:

  • could you add a release note?
  • this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

If you mean the gateway yes. I will take a look to fix it. Downstream we already run without root.

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

@psschwei gentle ping, added the release note and created knative-extensions/net-kourier#934.

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve
/hold

to give @dprotaso a chance to weigh in

@knative-prowknative-prowBot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 13, 2022
@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 13, 2022
@knative-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: psschwei, skonto

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prowknative-prowBot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 13, 2022

@evankandersonevankanderson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may want to pass-through the allowPrivilegeEscalation and seccompProfile fields, and then add a feature flag to default them to safe values if not provided.

I hadn't hit on this until the other day seeming @mattmoor 's #13395 , but I think I'd feel most happy long term with "didn't set anything" meaning secure, but allowing people to explicitly be insecure when needed (maybe with warnings)


// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

@skonto

skonto commented Oct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

@evankanderson I also mentioned on slack that allowing users to configure stuff on their own was more user friendly than having everything explicitly set without ability to change on demand. Also as you said users can choose their security level. For some properties though you can be opinionated and set explicitly the defaults instead of optionally applying them as in #13398 as long as users can remove them. This can be done, given our experience of what users need in practice eg. usually you dont want a service to escalate privileges. Another approach is to define security application as a downstream issue only. We have options depending on the strategy, do we want to be secure by default? My understanding is that this concept applies to other features like internal-encryption etc. Btw I am not sure if @mattmoor was aware of this PR or discussion. Should I close this PR and enable all stuff in #13398 or keep this PR with the queue proxy changes only?

(maybe with warnings)

You get the warning anyways from psp no need to add more imho.

Drop: []corev1.Capability{"all"},
Drop: []corev1.Capability{"ALL"},
},
SeccompProfile: &corev1.SeccompProfile{

@skontoskontoOct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this takes precedence compared to podsecurityContext.SeccompProfile.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it does.

@evankanderson

Copy link
Copy Markdown
Member

Can users see the PSP / PSA warnings when they apply a Knative Service? If so, then I'd agree about not needing a second set of warnings that settings are possibly dangerous.

In #13399, I tried to add a warning because our future behavior might change, which could break a small number of applications.

Upon reflection, I think giving the defaulting of security properties time to sit rather than rushing it for release is probably the right idea, particularly given the issues hit by #13399.

@evankanderson

Copy link
Copy Markdown
Member

I'm still willing to approve the queue-proxy side of this, but I don't want to add a breaking behavioral change without some sort of "out" for users.

@knative-prowknative-prowBot removed the lgtm Indicates that a PR is ready to be merged. label Oct 17, 2022
@knative-prowknative-prowBot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 17, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

/test istio-latest-no-mesh

@psschweipsschwei changed the title Allow user workloads to run with restricted profileRun queue proxy with restricted profileOct 18, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @skonto I updated the PR title / release note since these changes are just on QP now.
/lgtm

@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 18, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

@psschwei thank you, ok to unhold?

@psschwei

Copy link
Copy Markdown
Member

/hold cancel

@knative-prowknative-prowBot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 18, 2022
@knative-prow
knative-prowBot merged commit 388128b into knative:mainOct 18, 2022
skonto pushed a commit to skonto/serving that referenced this pull request Oct 19, 2022
* allow user workloads to run with restricted profile
* only change queue proxy
openshift-merge-robot pushed a commit to openshift/knative-serving that referenced this pull request Oct 27, 2022
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
skonto pushed a commit to skonto/serving that referenced this pull request Nov 15, 2022
…ve#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 16, 2022
#13)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/knative-serving that referenced this pull request Nov 16, 2022
…hift#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 18, 2022
#19)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
Co-authored-by: Stavros Kontopoulos <skontopo@redhat.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/APIAPI objects and controllerslgtmIndicates that a PR is ready to be merged.size/XSDenotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@skonto@evankanderson@psschwei
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Run queue proxy with restricted profile - #13376

Merged
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue
Oct 18, 2022
Merged

Run queue proxy with restricted profile#13376
knative-prow[bot] merged 2 commits into
knative:mainfrom
skonto:fix_psp_queue

Conversation

@skonto

@skontoskonto commented Oct 10, 2022

Copy link
Copy Markdown
Contributor

Fixes partially #13308

Proposed Changes

  • Allows to run a ksvc in a namespace with restricted profile enabled. Without this patch we get the following issue on 1.25 (see instructions next):

Warning FailedCreate replicaset/helloworld-go-00001-deployment-584759b77b (combined from similar events): Error creating: pods "helloworld-go-00001-deployment-584759b77b-tmxxd" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "user-container" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "queue-proxy" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or containers "user-container", "queue-proxy" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

  • To test follow instructions here.
  • Discussed here.

Release Note

Queue proxy explicit set `SeccompProfile` to `RunTimeDefault` to be able to run under restricted PSP policy by default.

@knative-prowknative-prowBot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. area/API API objects and controllers labels Oct 10, 2022

// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

@skontoskontoOct 10, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't allow the user to set the following two properties. So we enforce the defaults here to make it pass the PSP auditing.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I sent #13401 to allow setting this in the Revision Spec (e.g. in Service).

@knative-prowknative-prowBot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 10, 2022
@codecov

codecovBot commented Oct 10, 2022

Copy link
Copy Markdown

Codecov Report

Base: 86.47% // Head: 86.47% // No change to project coverage 👍

Coverage data is based on head (f3360cf) compared to base (6264c1b).
Patch has no changes to coverable lines.

Additional details and impacted files
@@ Coverage Diff @@## main #13376 +/- ##
=======================================
Coverage 86.47% 86.47% =======================================
Files 196 196 Lines 14551 14551 =======================================
Hits 12583 12583 Misses 1669 1669 Partials 299 299 
Impacted FilesCoverage Δ
pkg/reconciler/revision/resources/queue.go98.23% <ø> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@skontoskonto changed the title [wip] Allow user workloads to run with restricted profileAllow user workloads to run with restricted profileOct 11, 2022
@knative-prowknative-prowBot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 11, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minor things, but otherwise this looks good to me:

  • could you add a release note?
  • this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

this won't work with kourier (it doesn't set runAsNotRoot = true)... don't think that blocks us here, just wanted to make a note of it

If you mean the gateway yes. I will take a look to fix it. Downstream we already run without root.

@skonto

skonto commented Oct 13, 2022

Copy link
Copy Markdown
ContributorAuthor

@psschwei gentle ping, added the release note and created knative-extensions/net-kourier#934.

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve
/hold

to give @dprotaso a chance to weigh in

@knative-prowknative-prowBot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 13, 2022
@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 13, 2022
@knative-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: psschwei, skonto

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prowknative-prowBot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 13, 2022

@evankandersonevankanderson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may want to pass-through the allowPrivilegeEscalation and seccompProfile fields, and then add a feature flag to default them to safe values if not provided.

I hadn't hit on this until the other day seeming @mattmoor 's #13395 , but I think I'd feel most happy long term with "didn't set anything" meaning secure, but allowing people to explicitly be insecure when needed (maybe with warnings)


// Set PSP requirements explicitly to avoid failures in case `pod-security.kubernetes.io/enforce=restricted` is used
// at the user workload namespace
container.SecurityContext.AllowPrivilegeEscalation = ptr.Bool(false)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattmoor just made allowPrivilegeEscalation pass-through in #13395 , though we could force it to false as we do here instead.

@skonto

skonto commented Oct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

@evankanderson I also mentioned on slack that allowing users to configure stuff on their own was more user friendly than having everything explicitly set without ability to change on demand. Also as you said users can choose their security level. For some properties though you can be opinionated and set explicitly the defaults instead of optionally applying them as in #13398 as long as users can remove them. This can be done, given our experience of what users need in practice eg. usually you dont want a service to escalate privileges. Another approach is to define security application as a downstream issue only. We have options depending on the strategy, do we want to be secure by default? My understanding is that this concept applies to other features like internal-encryption etc. Btw I am not sure if @mattmoor was aware of this PR or discussion. Should I close this PR and enable all stuff in #13398 or keep this PR with the queue proxy changes only?

(maybe with warnings)

You get the warning anyways from psp no need to add more imho.

Drop: []corev1.Capability{"all"},
Drop: []corev1.Capability{"ALL"},
},
SeccompProfile: &corev1.SeccompProfile{

@skontoskontoOct 17, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this takes precedence compared to podsecurityContext.SeccompProfile.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it does.

@evankanderson

Copy link
Copy Markdown
Member

Can users see the PSP / PSA warnings when they apply a Knative Service? If so, then I'd agree about not needing a second set of warnings that settings are possibly dangerous.

In #13399, I tried to add a warning because our future behavior might change, which could break a small number of applications.

Upon reflection, I think giving the defaulting of security properties time to sit rather than rushing it for release is probably the right idea, particularly given the issues hit by #13399.

@evankanderson

Copy link
Copy Markdown
Member

I'm still willing to approve the queue-proxy side of this, but I don't want to add a breaking behavioral change without some sort of "out" for users.

@knative-prowknative-prowBot removed the lgtm Indicates that a PR is ready to be merged. label Oct 17, 2022
@knative-prowknative-prowBot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 17, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

/test istio-latest-no-mesh

@psschweipsschwei changed the title Allow user workloads to run with restricted profileRun queue proxy with restricted profileOct 18, 2022

@psschweipsschwei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @skonto I updated the PR title / release note since these changes are just on QP now.
/lgtm

@knative-prowknative-prowBot added the lgtm Indicates that a PR is ready to be merged. label Oct 18, 2022
@skonto

Copy link
Copy Markdown
ContributorAuthor

@psschwei thank you, ok to unhold?

@psschwei

Copy link
Copy Markdown
Member

/hold cancel

@knative-prowknative-prowBot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 18, 2022
@knative-prow
knative-prowBot merged commit 388128b into knative:mainOct 18, 2022
skonto pushed a commit to skonto/serving that referenced this pull request Oct 19, 2022
* allow user workloads to run with restricted profile
* only change queue proxy
openshift-merge-robot pushed a commit to openshift/knative-serving that referenced this pull request Oct 27, 2022
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
skonto pushed a commit to skonto/serving that referenced this pull request Nov 15, 2022
…ve#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 16, 2022
#13)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/knative-serving that referenced this pull request Nov 16, 2022
…hift#1283)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
openshift-merge-robot pushed a commit to openshift-knative/serving that referenced this pull request Nov 18, 2022
#19)
* Run queue proxy with restricted profile (knative#13376)
* allow user workloads to run with restricted profile
* only change queue proxy
* remove seccomp
Co-authored-by: Stavros Kontopoulos <skontopo@redhat.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by an approver from all required OWNERS files.area/APIAPI objects and controllerslgtmIndicates that a PR is ready to be merged.size/XSDenotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@skonto@evankanderson@psschwei