Use exec_attr to gain proc_lock_memory priv on illumos - #104292

Closed
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock
Closed

Use exec_attr to gain proc_lock_memory priv on illumos#104292
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock

Conversation

@gwr

@gwrgwr commented Jul 2, 2024

Copy link
Copy Markdown
Contributor

On Solaris and illumos, calling mlock(2) requires proc_lock_memory privilege.
Without that privilege, mlock() returns EPERM.

We can use fine-grained privileges(7) on these systems to grant this privilege.
See the comments in the code on how one sets that up.

This is helpful until we find a way to get rid of the mlock() page strategy,
either implementing somethings like membarrier(2) or something else.
Investigation of that is underway but may take a while.

While I'm here, fix the procfs.h include. Applications should use <procfs.h> not <sys/procfs.h>
That's why the temporary #define _KERNEL mess was there. With that fix, the mess is cleaned up.

gwr added 2 commits July 1, 2024 22:17
Or else define _STRUCTURED_PROC = 1 as described in sys/procfs.h
Also untangle from the NetBSD ifdef section.
@ghostghost added the area-PAL-coreclr only for closed issues label Jul 2, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Jul 2, 2024
@gwr

gwr commented Jul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

@am11 please have a look. I need this before #104275 for testing.

// PAL_InitializeCoreCLR / InitializeFlushProcessWriteBuffers
//
// This privilege is added to the "inheritable" set via the exec_attr(5)
// by adding a file like: /etc/security/exec_attr.d/dotnet containing:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expecting user to create this entry feels like a similar workaround as asking them to run sudo usermod -K defaultpriv='basic,proc_lock_memory' $(whoami) (current workaround). I think we should find a better way to avoid calling mlock(), as we do for macOS and other membarrier() supported platforms.

We were discussing about fork et al. based solution, did it not work?

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am looking at better options, but want to get through self hosting before I invest more time in that.
The exec_attr method is a bit easier for me to deal with for testing, but I guess I could just keep this change locally for now.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets keep this patch in local branch. I don't think it is a good idea to trade one semi-finished solution with another of a kind. We had a long discussion here: #36632.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's just close this then.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we can mark it as "draft" (there is a button visible for you on this page). In case we can't avoid mlock() this solution is better for packaging as you explained on IRC.

}
#endif // __APPLE__

#if defined(__sun)

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be able to use TARGET_SUNOS here too. (All __sun in this file)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's ok to leave it as __sun if nearby context has the similar usage.

#endif
#include <sys/procfs.h>
#ifdef UNDEF_KERNEL
#undef _KERNEL

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are three places where we had to use this workaround for a reason (I think some missing definition or behavior difference):

$ git grep -n "undef _KERNEL"
src/coreclr/pal/src/init/pal.cpp:86:#undef _KERNEL
src/coreclr/pal/src/thread/thread.cpp:45:#undef _KERNEL
src/native/libs/System.Native/pal_io.c:69:#undef _KERNEL

Maybe we should revisit them as well if you are sure that _KERNEL is not needed now?

Please make sure cross-build scenario is working with this change, otherwise it gets very complicated as we don't have the illumos SDK.

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other locations should have the same fix. All are due to the mistake of using <sys/procfs.h> where just <procfs.h> should be used. Would it be easier to have that change in a separate PR and fix all three places? I'm happy to try that and put up a PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Note that I'm not a maintainer. So I'm just giving what "I think" makes sense, which may or may not be the final word when maintainers will review these changes. :)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, also related to #44417

@gwrgwr closed this Jul 2, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Aug 2, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-PAL-coreclronly for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@gwr@am11
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Use exec_attr to gain proc_lock_memory priv on illumos - #104292

Closed
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock
Closed

Use exec_attr to gain proc_lock_memory priv on illumos#104292
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock

Conversation

@gwr

@gwrgwr commented Jul 2, 2024

Copy link
Copy Markdown
Contributor

On Solaris and illumos, calling mlock(2) requires proc_lock_memory privilege.
Without that privilege, mlock() returns EPERM.

We can use fine-grained privileges(7) on these systems to grant this privilege.
See the comments in the code on how one sets that up.

This is helpful until we find a way to get rid of the mlock() page strategy,
either implementing somethings like membarrier(2) or something else.
Investigation of that is underway but may take a while.

While I'm here, fix the procfs.h include. Applications should use <procfs.h> not <sys/procfs.h>
That's why the temporary #define _KERNEL mess was there. With that fix, the mess is cleaned up.

gwr added 2 commits July 1, 2024 22:17
Or else define _STRUCTURED_PROC = 1 as described in sys/procfs.h
Also untangle from the NetBSD ifdef section.
@ghostghost added the area-PAL-coreclr only for closed issues label Jul 2, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Jul 2, 2024
@gwr

gwr commented Jul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

@am11 please have a look. I need this before #104275 for testing.

// PAL_InitializeCoreCLR / InitializeFlushProcessWriteBuffers
//
// This privilege is added to the "inheritable" set via the exec_attr(5)
// by adding a file like: /etc/security/exec_attr.d/dotnet containing:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expecting user to create this entry feels like a similar workaround as asking them to run sudo usermod -K defaultpriv='basic,proc_lock_memory' $(whoami) (current workaround). I think we should find a better way to avoid calling mlock(), as we do for macOS and other membarrier() supported platforms.

We were discussing about fork et al. based solution, did it not work?

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am looking at better options, but want to get through self hosting before I invest more time in that.
The exec_attr method is a bit easier for me to deal with for testing, but I guess I could just keep this change locally for now.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets keep this patch in local branch. I don't think it is a good idea to trade one semi-finished solution with another of a kind. We had a long discussion here: #36632.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's just close this then.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we can mark it as "draft" (there is a button visible for you on this page). In case we can't avoid mlock() this solution is better for packaging as you explained on IRC.

}
#endif // __APPLE__

#if defined(__sun)

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be able to use TARGET_SUNOS here too. (All __sun in this file)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's ok to leave it as __sun if nearby context has the similar usage.

#endif
#include <sys/procfs.h>
#ifdef UNDEF_KERNEL
#undef _KERNEL

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are three places where we had to use this workaround for a reason (I think some missing definition or behavior difference):

$ git grep -n "undef _KERNEL"
src/coreclr/pal/src/init/pal.cpp:86:#undef _KERNEL
src/coreclr/pal/src/thread/thread.cpp:45:#undef _KERNEL
src/native/libs/System.Native/pal_io.c:69:#undef _KERNEL

Maybe we should revisit them as well if you are sure that _KERNEL is not needed now?

Please make sure cross-build scenario is working with this change, otherwise it gets very complicated as we don't have the illumos SDK.

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other locations should have the same fix. All are due to the mistake of using <sys/procfs.h> where just <procfs.h> should be used. Would it be easier to have that change in a separate PR and fix all three places? I'm happy to try that and put up a PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Note that I'm not a maintainer. So I'm just giving what "I think" makes sense, which may or may not be the final word when maintainers will review these changes. :)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, also related to #44417

@gwrgwr closed this Jul 2, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Aug 2, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-PAL-coreclronly for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@gwr@am11
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Use exec_attr to gain proc_lock_memory priv on illumos - #104292

Closed
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock
Closed

Use exec_attr to gain proc_lock_memory priv on illumos#104292
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock

Conversation

@gwr

@gwrgwr commented Jul 2, 2024

Copy link
Copy Markdown
Contributor

On Solaris and illumos, calling mlock(2) requires proc_lock_memory privilege.
Without that privilege, mlock() returns EPERM.

We can use fine-grained privileges(7) on these systems to grant this privilege.
See the comments in the code on how one sets that up.

This is helpful until we find a way to get rid of the mlock() page strategy,
either implementing somethings like membarrier(2) or something else.
Investigation of that is underway but may take a while.

While I'm here, fix the procfs.h include. Applications should use <procfs.h> not <sys/procfs.h>
That's why the temporary #define _KERNEL mess was there. With that fix, the mess is cleaned up.

gwr added 2 commits July 1, 2024 22:17
Or else define _STRUCTURED_PROC = 1 as described in sys/procfs.h
Also untangle from the NetBSD ifdef section.
@ghostghost added the area-PAL-coreclr only for closed issues label Jul 2, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Jul 2, 2024
@gwr

gwr commented Jul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

@am11 please have a look. I need this before #104275 for testing.

// PAL_InitializeCoreCLR / InitializeFlushProcessWriteBuffers
//
// This privilege is added to the "inheritable" set via the exec_attr(5)
// by adding a file like: /etc/security/exec_attr.d/dotnet containing:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expecting user to create this entry feels like a similar workaround as asking them to run sudo usermod -K defaultpriv='basic,proc_lock_memory' $(whoami) (current workaround). I think we should find a better way to avoid calling mlock(), as we do for macOS and other membarrier() supported platforms.

We were discussing about fork et al. based solution, did it not work?

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am looking at better options, but want to get through self hosting before I invest more time in that.
The exec_attr method is a bit easier for me to deal with for testing, but I guess I could just keep this change locally for now.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets keep this patch in local branch. I don't think it is a good idea to trade one semi-finished solution with another of a kind. We had a long discussion here: #36632.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's just close this then.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we can mark it as "draft" (there is a button visible for you on this page). In case we can't avoid mlock() this solution is better for packaging as you explained on IRC.

}
#endif // __APPLE__

#if defined(__sun)

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be able to use TARGET_SUNOS here too. (All __sun in this file)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's ok to leave it as __sun if nearby context has the similar usage.

#endif
#include <sys/procfs.h>
#ifdef UNDEF_KERNEL
#undef _KERNEL

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are three places where we had to use this workaround for a reason (I think some missing definition or behavior difference):

$ git grep -n "undef _KERNEL"
src/coreclr/pal/src/init/pal.cpp:86:#undef _KERNEL
src/coreclr/pal/src/thread/thread.cpp:45:#undef _KERNEL
src/native/libs/System.Native/pal_io.c:69:#undef _KERNEL

Maybe we should revisit them as well if you are sure that _KERNEL is not needed now?

Please make sure cross-build scenario is working with this change, otherwise it gets very complicated as we don't have the illumos SDK.

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other locations should have the same fix. All are due to the mistake of using <sys/procfs.h> where just <procfs.h> should be used. Would it be easier to have that change in a separate PR and fix all three places? I'm happy to try that and put up a PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Note that I'm not a maintainer. So I'm just giving what "I think" makes sense, which may or may not be the final word when maintainers will review these changes. :)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, also related to #44417

@gwrgwr closed this Jul 2, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Aug 2, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-PAL-coreclronly for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@gwr@am11
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Use exec_attr to gain proc_lock_memory priv on illumos - #104292

Closed
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock
Closed

Use exec_attr to gain proc_lock_memory priv on illumos#104292
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock

Conversation

@gwr

@gwrgwr commented Jul 2, 2024

Copy link
Copy Markdown
Contributor

On Solaris and illumos, calling mlock(2) requires proc_lock_memory privilege.
Without that privilege, mlock() returns EPERM.

We can use fine-grained privileges(7) on these systems to grant this privilege.
See the comments in the code on how one sets that up.

This is helpful until we find a way to get rid of the mlock() page strategy,
either implementing somethings like membarrier(2) or something else.
Investigation of that is underway but may take a while.

While I'm here, fix the procfs.h include. Applications should use <procfs.h> not <sys/procfs.h>
That's why the temporary #define _KERNEL mess was there. With that fix, the mess is cleaned up.

gwr added 2 commits July 1, 2024 22:17
Or else define _STRUCTURED_PROC = 1 as described in sys/procfs.h
Also untangle from the NetBSD ifdef section.
@ghostghost added the area-PAL-coreclr only for closed issues label Jul 2, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Jul 2, 2024
@gwr

gwr commented Jul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

@am11 please have a look. I need this before #104275 for testing.

// PAL_InitializeCoreCLR / InitializeFlushProcessWriteBuffers
//
// This privilege is added to the "inheritable" set via the exec_attr(5)
// by adding a file like: /etc/security/exec_attr.d/dotnet containing:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expecting user to create this entry feels like a similar workaround as asking them to run sudo usermod -K defaultpriv='basic,proc_lock_memory' $(whoami) (current workaround). I think we should find a better way to avoid calling mlock(), as we do for macOS and other membarrier() supported platforms.

We were discussing about fork et al. based solution, did it not work?

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am looking at better options, but want to get through self hosting before I invest more time in that.
The exec_attr method is a bit easier for me to deal with for testing, but I guess I could just keep this change locally for now.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets keep this patch in local branch. I don't think it is a good idea to trade one semi-finished solution with another of a kind. We had a long discussion here: #36632.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's just close this then.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we can mark it as "draft" (there is a button visible for you on this page). In case we can't avoid mlock() this solution is better for packaging as you explained on IRC.

}
#endif // __APPLE__

#if defined(__sun)

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be able to use TARGET_SUNOS here too. (All __sun in this file)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's ok to leave it as __sun if nearby context has the similar usage.

#endif
#include <sys/procfs.h>
#ifdef UNDEF_KERNEL
#undef _KERNEL

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are three places where we had to use this workaround for a reason (I think some missing definition or behavior difference):

$ git grep -n "undef _KERNEL"
src/coreclr/pal/src/init/pal.cpp:86:#undef _KERNEL
src/coreclr/pal/src/thread/thread.cpp:45:#undef _KERNEL
src/native/libs/System.Native/pal_io.c:69:#undef _KERNEL

Maybe we should revisit them as well if you are sure that _KERNEL is not needed now?

Please make sure cross-build scenario is working with this change, otherwise it gets very complicated as we don't have the illumos SDK.

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other locations should have the same fix. All are due to the mistake of using <sys/procfs.h> where just <procfs.h> should be used. Would it be easier to have that change in a separate PR and fix all three places? I'm happy to try that and put up a PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Note that I'm not a maintainer. So I'm just giving what "I think" makes sense, which may or may not be the final word when maintainers will review these changes. :)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, also related to #44417

@gwrgwr closed this Jul 2, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Aug 2, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-PAL-coreclronly for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@gwr@am11
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Use exec_attr to gain proc_lock_memory priv on illumos - #104292

Closed
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock
Closed

Use exec_attr to gain proc_lock_memory priv on illumos#104292
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock

Conversation

@gwr

@gwrgwr commented Jul 2, 2024

Copy link
Copy Markdown
Contributor

On Solaris and illumos, calling mlock(2) requires proc_lock_memory privilege.
Without that privilege, mlock() returns EPERM.

We can use fine-grained privileges(7) on these systems to grant this privilege.
See the comments in the code on how one sets that up.

This is helpful until we find a way to get rid of the mlock() page strategy,
either implementing somethings like membarrier(2) or something else.
Investigation of that is underway but may take a while.

While I'm here, fix the procfs.h include. Applications should use <procfs.h> not <sys/procfs.h>
That's why the temporary #define _KERNEL mess was there. With that fix, the mess is cleaned up.

gwr added 2 commits July 1, 2024 22:17
Or else define _STRUCTURED_PROC = 1 as described in sys/procfs.h
Also untangle from the NetBSD ifdef section.
@ghostghost added the area-PAL-coreclr only for closed issues label Jul 2, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Jul 2, 2024
@gwr

gwr commented Jul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

@am11 please have a look. I need this before #104275 for testing.

// PAL_InitializeCoreCLR / InitializeFlushProcessWriteBuffers
//
// This privilege is added to the "inheritable" set via the exec_attr(5)
// by adding a file like: /etc/security/exec_attr.d/dotnet containing:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expecting user to create this entry feels like a similar workaround as asking them to run sudo usermod -K defaultpriv='basic,proc_lock_memory' $(whoami) (current workaround). I think we should find a better way to avoid calling mlock(), as we do for macOS and other membarrier() supported platforms.

We were discussing about fork et al. based solution, did it not work?

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am looking at better options, but want to get through self hosting before I invest more time in that.
The exec_attr method is a bit easier for me to deal with for testing, but I guess I could just keep this change locally for now.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets keep this patch in local branch. I don't think it is a good idea to trade one semi-finished solution with another of a kind. We had a long discussion here: #36632.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's just close this then.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we can mark it as "draft" (there is a button visible for you on this page). In case we can't avoid mlock() this solution is better for packaging as you explained on IRC.

}
#endif // __APPLE__

#if defined(__sun)

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be able to use TARGET_SUNOS here too. (All __sun in this file)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's ok to leave it as __sun if nearby context has the similar usage.

#endif
#include <sys/procfs.h>
#ifdef UNDEF_KERNEL
#undef _KERNEL

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are three places where we had to use this workaround for a reason (I think some missing definition or behavior difference):

$ git grep -n "undef _KERNEL"
src/coreclr/pal/src/init/pal.cpp:86:#undef _KERNEL
src/coreclr/pal/src/thread/thread.cpp:45:#undef _KERNEL
src/native/libs/System.Native/pal_io.c:69:#undef _KERNEL

Maybe we should revisit them as well if you are sure that _KERNEL is not needed now?

Please make sure cross-build scenario is working with this change, otherwise it gets very complicated as we don't have the illumos SDK.

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other locations should have the same fix. All are due to the mistake of using <sys/procfs.h> where just <procfs.h> should be used. Would it be easier to have that change in a separate PR and fix all three places? I'm happy to try that and put up a PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Note that I'm not a maintainer. So I'm just giving what "I think" makes sense, which may or may not be the final word when maintainers will review these changes. :)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, also related to #44417

@gwrgwr closed this Jul 2, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Aug 2, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-PAL-coreclronly for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@gwr@am11
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Use exec_attr to gain proc_lock_memory priv on illumos - #104292

Closed
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock
Closed

Use exec_attr to gain proc_lock_memory priv on illumos#104292
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock

Conversation

@gwr

@gwrgwr commented Jul 2, 2024

Copy link
Copy Markdown
Contributor

On Solaris and illumos, calling mlock(2) requires proc_lock_memory privilege.
Without that privilege, mlock() returns EPERM.

We can use fine-grained privileges(7) on these systems to grant this privilege.
See the comments in the code on how one sets that up.

This is helpful until we find a way to get rid of the mlock() page strategy,
either implementing somethings like membarrier(2) or something else.
Investigation of that is underway but may take a while.

While I'm here, fix the procfs.h include. Applications should use <procfs.h> not <sys/procfs.h>
That's why the temporary #define _KERNEL mess was there. With that fix, the mess is cleaned up.

gwr added 2 commits July 1, 2024 22:17
Or else define _STRUCTURED_PROC = 1 as described in sys/procfs.h
Also untangle from the NetBSD ifdef section.
@ghostghost added the area-PAL-coreclr only for closed issues label Jul 2, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Jul 2, 2024
@gwr

gwr commented Jul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

@am11 please have a look. I need this before #104275 for testing.

// PAL_InitializeCoreCLR / InitializeFlushProcessWriteBuffers
//
// This privilege is added to the "inheritable" set via the exec_attr(5)
// by adding a file like: /etc/security/exec_attr.d/dotnet containing:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expecting user to create this entry feels like a similar workaround as asking them to run sudo usermod -K defaultpriv='basic,proc_lock_memory' $(whoami) (current workaround). I think we should find a better way to avoid calling mlock(), as we do for macOS and other membarrier() supported platforms.

We were discussing about fork et al. based solution, did it not work?

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am looking at better options, but want to get through self hosting before I invest more time in that.
The exec_attr method is a bit easier for me to deal with for testing, but I guess I could just keep this change locally for now.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets keep this patch in local branch. I don't think it is a good idea to trade one semi-finished solution with another of a kind. We had a long discussion here: #36632.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's just close this then.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we can mark it as "draft" (there is a button visible for you on this page). In case we can't avoid mlock() this solution is better for packaging as you explained on IRC.

}
#endif // __APPLE__

#if defined(__sun)

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be able to use TARGET_SUNOS here too. (All __sun in this file)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's ok to leave it as __sun if nearby context has the similar usage.

#endif
#include <sys/procfs.h>
#ifdef UNDEF_KERNEL
#undef _KERNEL

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are three places where we had to use this workaround for a reason (I think some missing definition or behavior difference):

$ git grep -n "undef _KERNEL"
src/coreclr/pal/src/init/pal.cpp:86:#undef _KERNEL
src/coreclr/pal/src/thread/thread.cpp:45:#undef _KERNEL
src/native/libs/System.Native/pal_io.c:69:#undef _KERNEL

Maybe we should revisit them as well if you are sure that _KERNEL is not needed now?

Please make sure cross-build scenario is working with this change, otherwise it gets very complicated as we don't have the illumos SDK.

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other locations should have the same fix. All are due to the mistake of using <sys/procfs.h> where just <procfs.h> should be used. Would it be easier to have that change in a separate PR and fix all three places? I'm happy to try that and put up a PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Note that I'm not a maintainer. So I'm just giving what "I think" makes sense, which may or may not be the final word when maintainers will review these changes. :)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, also related to #44417

@gwrgwr closed this Jul 2, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Aug 2, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-PAL-coreclronly for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@gwr@am11
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Use exec_attr to gain proc_lock_memory priv on illumos - #104292

Closed
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock
Closed

Use exec_attr to gain proc_lock_memory priv on illumos#104292
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock

Conversation

@gwr

@gwrgwr commented Jul 2, 2024

Copy link
Copy Markdown
Contributor

On Solaris and illumos, calling mlock(2) requires proc_lock_memory privilege.
Without that privilege, mlock() returns EPERM.

We can use fine-grained privileges(7) on these systems to grant this privilege.
See the comments in the code on how one sets that up.

This is helpful until we find a way to get rid of the mlock() page strategy,
either implementing somethings like membarrier(2) or something else.
Investigation of that is underway but may take a while.

While I'm here, fix the procfs.h include. Applications should use <procfs.h> not <sys/procfs.h>
That's why the temporary #define _KERNEL mess was there. With that fix, the mess is cleaned up.

gwr added 2 commits July 1, 2024 22:17
Or else define _STRUCTURED_PROC = 1 as described in sys/procfs.h
Also untangle from the NetBSD ifdef section.
@ghostghost added the area-PAL-coreclr only for closed issues label Jul 2, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Jul 2, 2024
@gwr

gwr commented Jul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

@am11 please have a look. I need this before #104275 for testing.

// PAL_InitializeCoreCLR / InitializeFlushProcessWriteBuffers
//
// This privilege is added to the "inheritable" set via the exec_attr(5)
// by adding a file like: /etc/security/exec_attr.d/dotnet containing:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expecting user to create this entry feels like a similar workaround as asking them to run sudo usermod -K defaultpriv='basic,proc_lock_memory' $(whoami) (current workaround). I think we should find a better way to avoid calling mlock(), as we do for macOS and other membarrier() supported platforms.

We were discussing about fork et al. based solution, did it not work?

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am looking at better options, but want to get through self hosting before I invest more time in that.
The exec_attr method is a bit easier for me to deal with for testing, but I guess I could just keep this change locally for now.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets keep this patch in local branch. I don't think it is a good idea to trade one semi-finished solution with another of a kind. We had a long discussion here: #36632.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's just close this then.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we can mark it as "draft" (there is a button visible for you on this page). In case we can't avoid mlock() this solution is better for packaging as you explained on IRC.

}
#endif // __APPLE__

#if defined(__sun)

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be able to use TARGET_SUNOS here too. (All __sun in this file)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's ok to leave it as __sun if nearby context has the similar usage.

#endif
#include <sys/procfs.h>
#ifdef UNDEF_KERNEL
#undef _KERNEL

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are three places where we had to use this workaround for a reason (I think some missing definition or behavior difference):

$ git grep -n "undef _KERNEL"
src/coreclr/pal/src/init/pal.cpp:86:#undef _KERNEL
src/coreclr/pal/src/thread/thread.cpp:45:#undef _KERNEL
src/native/libs/System.Native/pal_io.c:69:#undef _KERNEL

Maybe we should revisit them as well if you are sure that _KERNEL is not needed now?

Please make sure cross-build scenario is working with this change, otherwise it gets very complicated as we don't have the illumos SDK.

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other locations should have the same fix. All are due to the mistake of using <sys/procfs.h> where just <procfs.h> should be used. Would it be easier to have that change in a separate PR and fix all three places? I'm happy to try that and put up a PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Note that I'm not a maintainer. So I'm just giving what "I think" makes sense, which may or may not be the final word when maintainers will review these changes. :)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, also related to #44417

@gwrgwr closed this Jul 2, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Aug 2, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-PAL-coreclronly for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@gwr@am11
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Use exec_attr to gain proc_lock_memory priv on illumos - #104292

Closed
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock
Closed

Use exec_attr to gain proc_lock_memory priv on illumos#104292
gwr wants to merge 2 commits into
dotnet:mainfrom
gwr:pr-mlock

Conversation

@gwr

@gwrgwr commented Jul 2, 2024

Copy link
Copy Markdown
Contributor

On Solaris and illumos, calling mlock(2) requires proc_lock_memory privilege.
Without that privilege, mlock() returns EPERM.

We can use fine-grained privileges(7) on these systems to grant this privilege.
See the comments in the code on how one sets that up.

This is helpful until we find a way to get rid of the mlock() page strategy,
either implementing somethings like membarrier(2) or something else.
Investigation of that is underway but may take a while.

While I'm here, fix the procfs.h include. Applications should use <procfs.h> not <sys/procfs.h>
That's why the temporary #define _KERNEL mess was there. With that fix, the mess is cleaned up.

gwr added 2 commits July 1, 2024 22:17
Or else define _STRUCTURED_PROC = 1 as described in sys/procfs.h
Also untangle from the NetBSD ifdef section.
@ghostghost added the area-PAL-coreclr only for closed issues label Jul 2, 2024
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Jul 2, 2024
@gwr

gwr commented Jul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

@am11 please have a look. I need this before #104275 for testing.

// PAL_InitializeCoreCLR / InitializeFlushProcessWriteBuffers
//
// This privilege is added to the "inheritable" set via the exec_attr(5)
// by adding a file like: /etc/security/exec_attr.d/dotnet containing:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expecting user to create this entry feels like a similar workaround as asking them to run sudo usermod -K defaultpriv='basic,proc_lock_memory' $(whoami) (current workaround). I think we should find a better way to avoid calling mlock(), as we do for macOS and other membarrier() supported platforms.

We were discussing about fork et al. based solution, did it not work?

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am looking at better options, but want to get through self hosting before I invest more time in that.
The exec_attr method is a bit easier for me to deal with for testing, but I guess I could just keep this change locally for now.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets keep this patch in local branch. I don't think it is a good idea to trade one semi-finished solution with another of a kind. We had a long discussion here: #36632.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's just close this then.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we can mark it as "draft" (there is a button visible for you on this page). In case we can't avoid mlock() this solution is better for packaging as you explained on IRC.

}
#endif // __APPLE__

#if defined(__sun)

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be able to use TARGET_SUNOS here too. (All __sun in this file)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's ok to leave it as __sun if nearby context has the similar usage.

#endif
#include <sys/procfs.h>
#ifdef UNDEF_KERNEL
#undef _KERNEL

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are three places where we had to use this workaround for a reason (I think some missing definition or behavior difference):

$ git grep -n "undef _KERNEL"
src/coreclr/pal/src/init/pal.cpp:86:#undef _KERNEL
src/coreclr/pal/src/thread/thread.cpp:45:#undef _KERNEL
src/native/libs/System.Native/pal_io.c:69:#undef _KERNEL

Maybe we should revisit them as well if you are sure that _KERNEL is not needed now?

Please make sure cross-build scenario is working with this change, otherwise it gets very complicated as we don't have the illumos SDK.

@gwrgwrJul 2, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other locations should have the same fix. All are due to the mistake of using <sys/procfs.h> where just <procfs.h> should be used. Would it be easier to have that change in a separate PR and fix all three places? I'm happy to try that and put up a PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Note that I'm not a maintainer. So I'm just giving what "I think" makes sense, which may or may not be the final word when maintainers will review these changes. :)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, also related to #44417

@gwrgwr closed this Jul 2, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Aug 2, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-PAL-coreclronly for closed issuescommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@gwr@am11