Adding Linux support to System.DirectoryServices.Protocols - #35380

Merged
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP
May 7, 2020
Merged

Adding Linux support to System.DirectoryServices.Protocols#35380
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP

Conversation

@joperezr

Copy link
Copy Markdown
Member

Fixes#23944

Creating this as a draft PR first as I expect a lot of feedback plus some things will still need to happen in order to get tests working in Linux, but I want to start getting feedback so I can address it while I finish setting up running tests in Linux. This adds support to Linux for the System.DirectoryServices.Protocols namespace, by simply creating a PAL layer that abstracts the calls between wldap32 and openldap. Given both native libraries are just implementing ldap protocol, most of their APIs are the same with very similar or same structures which makes it so that managed code didn't actually have to change that much, we just change the native calls we do underneath. There are some open questions, like for example, there are some settings that are specific to wldap32 and not supported on any other library, the question is, should they be No-Ops for compatibility with previous apps working in Windows or should they instead throw PNSE?

One thing that is missing from this PR is the support for Default Credentials. That means that if you have your code running on a Kestrel Server which is domain-joined to an AD and have a valid session token already authenticated in the machine, then you wouldn't need to pass in credentials again when creating an LDAPConnection since you would just reuse that token. That works today in Windows, and I'm working on fixing the native calls on Bind to make sure that it also works on Linux. That support will come either as part of this PR later, or it will be added as a subsequent PR.

cc: @ericstj@tarekgh@bartonjs@stephentoub@davidsh

FYI: @JunTaoLuo@Tratcher

@tarekgh

tarekgh commented Apr 23, 2020

Copy link
Copy Markdown
Member

CC @tquerec

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CharSet = CharSet.Unicode [](start = 41, length = 25)

why this needed here?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure, this was the way this struct already existed, I just moved them to a new file.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disposeHandle here seems to be redundant with the one from the base class... which you're always giving the literal true.

If it means "false to never call ReleaseHandle", that's exactly what the base bool ownsHandle is for.

fPerformRelease = ((oldState & (SH_State_RefCount | SH_State_Closed)) == SH_RefCountOne) && m_ownsHandle;
if (fPerformRelease)
{
GCPROTECT_BEGIN(sh);
CLR_BOOLfIsInvalid = FALSE;
DECLARE_ARGHOLDER_ARRAY(args, 1);
args[ARGNUM_0] = OBJECTREF_TO_ARGHOLDER(sh);
PREPARE_SIMPLE_VIRTUAL_CALLSITE_USING_SLOT(s_IsInvalidHandleMethodSlot, sh);
CRITICAL_CALLSITE;
CALL_MANAGED_METHOD(fIsInvalid, CLR_BOOL, args);
if (fIsInvalid)
{
fPerformRelease = false;
}
GCPROTECT_END();
}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is your suggestion to basically instead of having a field for _needDispose and checking this in ReleaseHandle, I should just call base passing in disposeHandle value and not need to check in ReleaseHandle as it would not get called in the case it was set to false right? Just making sure I understood this comment..

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually we do have one problem with doing this. The one in the base I believe can't be overwritten after creation, and we have some (internal) methods in LdapConnection that can flip the _needsDispose, so I guess the reason why the original design was like this, was so that ReleaseHandle would get called always, and we would only actually dispose if we need to.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value, (maybe I'm wrong and it just finds a ctor that looks like one it knows how to call? or maybe a default is required and I'm having a very bad recollection day). I'm not sure this code actually prevents invalid handles.

A more canonical representation of this would be to declare the ldap_init call to return a SafeConnectionHandle (instead of IntPtr) and the caller should do

SafeConnectionHandlehandle=LdapPal.Initialize(null,389);

// In LdapPal.Windows.cs

internalstaticSafeConnectionHandleInitialize(stringhost,intport){SafeConnectionHandlehandle=Interop.wldap32.ldap_init(host,port);if(handle.IsInvalid){handle.Dispose();// this error handling code}returnhandle;}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value

Nope :)

e.g. try this:

usingSystem;usingSystem.Runtime.InteropServices;classProgram{staticvoidMain()=>GetStdHandle(-10);[DllImport("kernel32.dll",SetLastError=true)]privatestaticexternMySafeHandleGetStdHandle(intnStdHandle);}classMySafeHandle:SafeHandle{publicMySafeHandle(stringhello,stringworld):base(IntPtr.Zero,ownsHandle:false){}publicoverrideboolIsInvalid=>true;protectedoverrideboolReleaseHandle()=>true;}

You'll get an exception like this:

Unhandled Exception: System.MissingMethodException: .ctor
at Program.GetStdHandle(Int32 nStdHandle)
at Program.Main()


namespace System.DirectoryServices.Protocols
{
internal sealed class ConnectionHandle : SafeHandleZeroOrMinusOneIsInvalid

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is used by the P/Invoke declarations in Interop, I'd put this file in that same directory (similar to the Windows one). e.g. https://github.com/dotnet/runtime/blob/master/src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.OCSP.cs#L103-L134

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Following the OCSP example, you could just nestle the appropriate SafeHandle types in the bottom of the Interop.Ber.cs and Interop.Ldap.cs files, so that if some other library needs them it's just one include)

Comment threadsrc/libraries/System.DirectoryServices.Protocols/tests/BerConverterTests.cs Outdated
public static IEnumerable<object[]> Ctor_BeforeCount_AfterCount_ByteArrayTarget_Data()
{
yield return new object[] { 0, 0, null, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 18, 2, 1, 0, 2, 1, 0, 160, 132, 0, 0, 0, 6, 2, 1, 0, 2, 1, 0 } : new byte[] { 48, 14, 2, 1, 0, 2, 1, 0, 160, 6, 2, 1, 0, 2, 1, 0 } };
yield return new object[] { 10, 10, new byte[] { 1, 2, 3 }, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } : new byte[] { 48, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } };

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For long-term test-building maintenance, consider a helper. E.g.

yieldreturnnewobject[]{0,0,null,FormatLengths("30{0}020100020100A0{1}020100020100",18,6)};
...private static byte[]FormatLengths(stringhexFormat,paramsint[]lengths){string[]hexLengths=newstring[lengths.Length];for(inti=0;i<lengths.Length;i++){intcurLen=lengths[i];if(PlatformDetection.IsWindows){hexLengths[i]=$"84{curLen:X8}";}elseif(curLen<0x80){hexLengths[i]=curLen.ToString("X2");}elseif(curLen<=byte.MaxValue){hexLengths[i]=$"81{curLen:X2}";}elseif(curLen<=ushort.MaxValue){hexLengths[i]=$"82{curLen:X4}";}elseif(curLen<=0xFFFFFF){hexLengths[i]=$"83{curLen:X6}";}else{hexLengths[i]=$"84{curLen:X8}";}}stringhex=string.Format(hexFormat,hexLengths);// This presupposes something like the crypto tests have:returnhex.HexToByteArray();}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this would be valuable, but if that's ok with you I would rather do that in a subsequent PR so to not add more complexity to this PR for now. Once I have tests running on Mono and OSX and passing, then I think it would be a great time to address this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferring the test helper is totally fine.

Comment threadsrc/libraries/Common/src/Interop/Interop.Ldap.cs Outdated
@joperezr
joperezr merged commit 55d3260 into dotnet:masterMay 7, 2020
@joperezr
joperezr deleted the SDSP branch May 7, 2020 20:37
@joperezr

Copy link
Copy Markdown
MemberAuthor

Thanks a lot to all for the reviews! 😄

@danmoseley

Copy link
Copy Markdown
Contributor

Nice! 🔥

@JunTaoLuo

Copy link
Copy Markdown
Contributor

🎆

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support System.DirectoryServices.Protocols on Linux/Mac

9 participants

@joperezr@tarekgh@danmoseley@JunTaoLuo@abatishchev@stephentoub@gfoidl@bartonjs@Dotnet-GitSync-Bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Adding Linux support to System.DirectoryServices.Protocols - #35380

Merged
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP
May 7, 2020
Merged

Adding Linux support to System.DirectoryServices.Protocols#35380
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP

Conversation

@joperezr

Copy link
Copy Markdown
Member

Fixes#23944

Creating this as a draft PR first as I expect a lot of feedback plus some things will still need to happen in order to get tests working in Linux, but I want to start getting feedback so I can address it while I finish setting up running tests in Linux. This adds support to Linux for the System.DirectoryServices.Protocols namespace, by simply creating a PAL layer that abstracts the calls between wldap32 and openldap. Given both native libraries are just implementing ldap protocol, most of their APIs are the same with very similar or same structures which makes it so that managed code didn't actually have to change that much, we just change the native calls we do underneath. There are some open questions, like for example, there are some settings that are specific to wldap32 and not supported on any other library, the question is, should they be No-Ops for compatibility with previous apps working in Windows or should they instead throw PNSE?

One thing that is missing from this PR is the support for Default Credentials. That means that if you have your code running on a Kestrel Server which is domain-joined to an AD and have a valid session token already authenticated in the machine, then you wouldn't need to pass in credentials again when creating an LDAPConnection since you would just reuse that token. That works today in Windows, and I'm working on fixing the native calls on Bind to make sure that it also works on Linux. That support will come either as part of this PR later, or it will be added as a subsequent PR.

cc: @ericstj@tarekgh@bartonjs@stephentoub@davidsh

FYI: @JunTaoLuo@Tratcher

@tarekgh

tarekgh commented Apr 23, 2020

Copy link
Copy Markdown
Member

CC @tquerec

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CharSet = CharSet.Unicode [](start = 41, length = 25)

why this needed here?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure, this was the way this struct already existed, I just moved them to a new file.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disposeHandle here seems to be redundant with the one from the base class... which you're always giving the literal true.

If it means "false to never call ReleaseHandle", that's exactly what the base bool ownsHandle is for.

fPerformRelease = ((oldState & (SH_State_RefCount | SH_State_Closed)) == SH_RefCountOne) && m_ownsHandle;
if (fPerformRelease)
{
GCPROTECT_BEGIN(sh);
CLR_BOOLfIsInvalid = FALSE;
DECLARE_ARGHOLDER_ARRAY(args, 1);
args[ARGNUM_0] = OBJECTREF_TO_ARGHOLDER(sh);
PREPARE_SIMPLE_VIRTUAL_CALLSITE_USING_SLOT(s_IsInvalidHandleMethodSlot, sh);
CRITICAL_CALLSITE;
CALL_MANAGED_METHOD(fIsInvalid, CLR_BOOL, args);
if (fIsInvalid)
{
fPerformRelease = false;
}
GCPROTECT_END();
}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is your suggestion to basically instead of having a field for _needDispose and checking this in ReleaseHandle, I should just call base passing in disposeHandle value and not need to check in ReleaseHandle as it would not get called in the case it was set to false right? Just making sure I understood this comment..

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually we do have one problem with doing this. The one in the base I believe can't be overwritten after creation, and we have some (internal) methods in LdapConnection that can flip the _needsDispose, so I guess the reason why the original design was like this, was so that ReleaseHandle would get called always, and we would only actually dispose if we need to.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value, (maybe I'm wrong and it just finds a ctor that looks like one it knows how to call? or maybe a default is required and I'm having a very bad recollection day). I'm not sure this code actually prevents invalid handles.

A more canonical representation of this would be to declare the ldap_init call to return a SafeConnectionHandle (instead of IntPtr) and the caller should do

SafeConnectionHandlehandle=LdapPal.Initialize(null,389);

// In LdapPal.Windows.cs

internalstaticSafeConnectionHandleInitialize(stringhost,intport){SafeConnectionHandlehandle=Interop.wldap32.ldap_init(host,port);if(handle.IsInvalid){handle.Dispose();// this error handling code}returnhandle;}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value

Nope :)

e.g. try this:

usingSystem;usingSystem.Runtime.InteropServices;classProgram{staticvoidMain()=>GetStdHandle(-10);[DllImport("kernel32.dll",SetLastError=true)]privatestaticexternMySafeHandleGetStdHandle(intnStdHandle);}classMySafeHandle:SafeHandle{publicMySafeHandle(stringhello,stringworld):base(IntPtr.Zero,ownsHandle:false){}publicoverrideboolIsInvalid=>true;protectedoverrideboolReleaseHandle()=>true;}

You'll get an exception like this:

Unhandled Exception: System.MissingMethodException: .ctor
at Program.GetStdHandle(Int32 nStdHandle)
at Program.Main()


namespace System.DirectoryServices.Protocols
{
internal sealed class ConnectionHandle : SafeHandleZeroOrMinusOneIsInvalid

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is used by the P/Invoke declarations in Interop, I'd put this file in that same directory (similar to the Windows one). e.g. https://github.com/dotnet/runtime/blob/master/src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.OCSP.cs#L103-L134

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Following the OCSP example, you could just nestle the appropriate SafeHandle types in the bottom of the Interop.Ber.cs and Interop.Ldap.cs files, so that if some other library needs them it's just one include)

Comment threadsrc/libraries/System.DirectoryServices.Protocols/tests/BerConverterTests.cs Outdated
public static IEnumerable<object[]> Ctor_BeforeCount_AfterCount_ByteArrayTarget_Data()
{
yield return new object[] { 0, 0, null, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 18, 2, 1, 0, 2, 1, 0, 160, 132, 0, 0, 0, 6, 2, 1, 0, 2, 1, 0 } : new byte[] { 48, 14, 2, 1, 0, 2, 1, 0, 160, 6, 2, 1, 0, 2, 1, 0 } };
yield return new object[] { 10, 10, new byte[] { 1, 2, 3 }, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } : new byte[] { 48, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } };

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For long-term test-building maintenance, consider a helper. E.g.

yieldreturnnewobject[]{0,0,null,FormatLengths("30{0}020100020100A0{1}020100020100",18,6)};
...private static byte[]FormatLengths(stringhexFormat,paramsint[]lengths){string[]hexLengths=newstring[lengths.Length];for(inti=0;i<lengths.Length;i++){intcurLen=lengths[i];if(PlatformDetection.IsWindows){hexLengths[i]=$"84{curLen:X8}";}elseif(curLen<0x80){hexLengths[i]=curLen.ToString("X2");}elseif(curLen<=byte.MaxValue){hexLengths[i]=$"81{curLen:X2}";}elseif(curLen<=ushort.MaxValue){hexLengths[i]=$"82{curLen:X4}";}elseif(curLen<=0xFFFFFF){hexLengths[i]=$"83{curLen:X6}";}else{hexLengths[i]=$"84{curLen:X8}";}}stringhex=string.Format(hexFormat,hexLengths);// This presupposes something like the crypto tests have:returnhex.HexToByteArray();}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this would be valuable, but if that's ok with you I would rather do that in a subsequent PR so to not add more complexity to this PR for now. Once I have tests running on Mono and OSX and passing, then I think it would be a great time to address this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferring the test helper is totally fine.

Comment threadsrc/libraries/Common/src/Interop/Interop.Ldap.cs Outdated
@joperezr
joperezr merged commit 55d3260 into dotnet:masterMay 7, 2020
@joperezr
joperezr deleted the SDSP branch May 7, 2020 20:37
@joperezr

Copy link
Copy Markdown
MemberAuthor

Thanks a lot to all for the reviews! 😄

@danmoseley

Copy link
Copy Markdown
Contributor

Nice! 🔥

@JunTaoLuo

Copy link
Copy Markdown
Contributor

🎆

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support System.DirectoryServices.Protocols on Linux/Mac

9 participants

@joperezr@tarekgh@danmoseley@JunTaoLuo@abatishchev@stephentoub@gfoidl@bartonjs@Dotnet-GitSync-Bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Adding Linux support to System.DirectoryServices.Protocols - #35380

Merged
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP
May 7, 2020
Merged

Adding Linux support to System.DirectoryServices.Protocols#35380
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP

Conversation

@joperezr

Copy link
Copy Markdown
Member

Fixes#23944

Creating this as a draft PR first as I expect a lot of feedback plus some things will still need to happen in order to get tests working in Linux, but I want to start getting feedback so I can address it while I finish setting up running tests in Linux. This adds support to Linux for the System.DirectoryServices.Protocols namespace, by simply creating a PAL layer that abstracts the calls between wldap32 and openldap. Given both native libraries are just implementing ldap protocol, most of their APIs are the same with very similar or same structures which makes it so that managed code didn't actually have to change that much, we just change the native calls we do underneath. There are some open questions, like for example, there are some settings that are specific to wldap32 and not supported on any other library, the question is, should they be No-Ops for compatibility with previous apps working in Windows or should they instead throw PNSE?

One thing that is missing from this PR is the support for Default Credentials. That means that if you have your code running on a Kestrel Server which is domain-joined to an AD and have a valid session token already authenticated in the machine, then you wouldn't need to pass in credentials again when creating an LDAPConnection since you would just reuse that token. That works today in Windows, and I'm working on fixing the native calls on Bind to make sure that it also works on Linux. That support will come either as part of this PR later, or it will be added as a subsequent PR.

cc: @ericstj@tarekgh@bartonjs@stephentoub@davidsh

FYI: @JunTaoLuo@Tratcher

@tarekgh

tarekgh commented Apr 23, 2020

Copy link
Copy Markdown
Member

CC @tquerec

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CharSet = CharSet.Unicode [](start = 41, length = 25)

why this needed here?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure, this was the way this struct already existed, I just moved them to a new file.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disposeHandle here seems to be redundant with the one from the base class... which you're always giving the literal true.

If it means "false to never call ReleaseHandle", that's exactly what the base bool ownsHandle is for.

fPerformRelease = ((oldState & (SH_State_RefCount | SH_State_Closed)) == SH_RefCountOne) && m_ownsHandle;
if (fPerformRelease)
{
GCPROTECT_BEGIN(sh);
CLR_BOOLfIsInvalid = FALSE;
DECLARE_ARGHOLDER_ARRAY(args, 1);
args[ARGNUM_0] = OBJECTREF_TO_ARGHOLDER(sh);
PREPARE_SIMPLE_VIRTUAL_CALLSITE_USING_SLOT(s_IsInvalidHandleMethodSlot, sh);
CRITICAL_CALLSITE;
CALL_MANAGED_METHOD(fIsInvalid, CLR_BOOL, args);
if (fIsInvalid)
{
fPerformRelease = false;
}
GCPROTECT_END();
}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is your suggestion to basically instead of having a field for _needDispose and checking this in ReleaseHandle, I should just call base passing in disposeHandle value and not need to check in ReleaseHandle as it would not get called in the case it was set to false right? Just making sure I understood this comment..

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually we do have one problem with doing this. The one in the base I believe can't be overwritten after creation, and we have some (internal) methods in LdapConnection that can flip the _needsDispose, so I guess the reason why the original design was like this, was so that ReleaseHandle would get called always, and we would only actually dispose if we need to.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value, (maybe I'm wrong and it just finds a ctor that looks like one it knows how to call? or maybe a default is required and I'm having a very bad recollection day). I'm not sure this code actually prevents invalid handles.

A more canonical representation of this would be to declare the ldap_init call to return a SafeConnectionHandle (instead of IntPtr) and the caller should do

SafeConnectionHandlehandle=LdapPal.Initialize(null,389);

// In LdapPal.Windows.cs

internalstaticSafeConnectionHandleInitialize(stringhost,intport){SafeConnectionHandlehandle=Interop.wldap32.ldap_init(host,port);if(handle.IsInvalid){handle.Dispose();// this error handling code}returnhandle;}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value

Nope :)

e.g. try this:

usingSystem;usingSystem.Runtime.InteropServices;classProgram{staticvoidMain()=>GetStdHandle(-10);[DllImport("kernel32.dll",SetLastError=true)]privatestaticexternMySafeHandleGetStdHandle(intnStdHandle);}classMySafeHandle:SafeHandle{publicMySafeHandle(stringhello,stringworld):base(IntPtr.Zero,ownsHandle:false){}publicoverrideboolIsInvalid=>true;protectedoverrideboolReleaseHandle()=>true;}

You'll get an exception like this:

Unhandled Exception: System.MissingMethodException: .ctor
at Program.GetStdHandle(Int32 nStdHandle)
at Program.Main()


namespace System.DirectoryServices.Protocols
{
internal sealed class ConnectionHandle : SafeHandleZeroOrMinusOneIsInvalid

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is used by the P/Invoke declarations in Interop, I'd put this file in that same directory (similar to the Windows one). e.g. https://github.com/dotnet/runtime/blob/master/src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.OCSP.cs#L103-L134

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Following the OCSP example, you could just nestle the appropriate SafeHandle types in the bottom of the Interop.Ber.cs and Interop.Ldap.cs files, so that if some other library needs them it's just one include)

Comment threadsrc/libraries/System.DirectoryServices.Protocols/tests/BerConverterTests.cs Outdated
public static IEnumerable<object[]> Ctor_BeforeCount_AfterCount_ByteArrayTarget_Data()
{
yield return new object[] { 0, 0, null, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 18, 2, 1, 0, 2, 1, 0, 160, 132, 0, 0, 0, 6, 2, 1, 0, 2, 1, 0 } : new byte[] { 48, 14, 2, 1, 0, 2, 1, 0, 160, 6, 2, 1, 0, 2, 1, 0 } };
yield return new object[] { 10, 10, new byte[] { 1, 2, 3 }, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } : new byte[] { 48, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } };

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For long-term test-building maintenance, consider a helper. E.g.

yieldreturnnewobject[]{0,0,null,FormatLengths("30{0}020100020100A0{1}020100020100",18,6)};
...private static byte[]FormatLengths(stringhexFormat,paramsint[]lengths){string[]hexLengths=newstring[lengths.Length];for(inti=0;i<lengths.Length;i++){intcurLen=lengths[i];if(PlatformDetection.IsWindows){hexLengths[i]=$"84{curLen:X8}";}elseif(curLen<0x80){hexLengths[i]=curLen.ToString("X2");}elseif(curLen<=byte.MaxValue){hexLengths[i]=$"81{curLen:X2}";}elseif(curLen<=ushort.MaxValue){hexLengths[i]=$"82{curLen:X4}";}elseif(curLen<=0xFFFFFF){hexLengths[i]=$"83{curLen:X6}";}else{hexLengths[i]=$"84{curLen:X8}";}}stringhex=string.Format(hexFormat,hexLengths);// This presupposes something like the crypto tests have:returnhex.HexToByteArray();}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this would be valuable, but if that's ok with you I would rather do that in a subsequent PR so to not add more complexity to this PR for now. Once I have tests running on Mono and OSX and passing, then I think it would be a great time to address this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferring the test helper is totally fine.

Comment threadsrc/libraries/Common/src/Interop/Interop.Ldap.cs Outdated
@joperezr
joperezr merged commit 55d3260 into dotnet:masterMay 7, 2020
@joperezr
joperezr deleted the SDSP branch May 7, 2020 20:37
@joperezr

Copy link
Copy Markdown
MemberAuthor

Thanks a lot to all for the reviews! 😄

@danmoseley

Copy link
Copy Markdown
Contributor

Nice! 🔥

@JunTaoLuo

Copy link
Copy Markdown
Contributor

🎆

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support System.DirectoryServices.Protocols on Linux/Mac

9 participants

@joperezr@tarekgh@danmoseley@JunTaoLuo@abatishchev@stephentoub@gfoidl@bartonjs@Dotnet-GitSync-Bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Adding Linux support to System.DirectoryServices.Protocols - #35380

Merged
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP
May 7, 2020
Merged

Adding Linux support to System.DirectoryServices.Protocols#35380
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP

Conversation

@joperezr

Copy link
Copy Markdown
Member

Fixes#23944

Creating this as a draft PR first as I expect a lot of feedback plus some things will still need to happen in order to get tests working in Linux, but I want to start getting feedback so I can address it while I finish setting up running tests in Linux. This adds support to Linux for the System.DirectoryServices.Protocols namespace, by simply creating a PAL layer that abstracts the calls between wldap32 and openldap. Given both native libraries are just implementing ldap protocol, most of their APIs are the same with very similar or same structures which makes it so that managed code didn't actually have to change that much, we just change the native calls we do underneath. There are some open questions, like for example, there are some settings that are specific to wldap32 and not supported on any other library, the question is, should they be No-Ops for compatibility with previous apps working in Windows or should they instead throw PNSE?

One thing that is missing from this PR is the support for Default Credentials. That means that if you have your code running on a Kestrel Server which is domain-joined to an AD and have a valid session token already authenticated in the machine, then you wouldn't need to pass in credentials again when creating an LDAPConnection since you would just reuse that token. That works today in Windows, and I'm working on fixing the native calls on Bind to make sure that it also works on Linux. That support will come either as part of this PR later, or it will be added as a subsequent PR.

cc: @ericstj@tarekgh@bartonjs@stephentoub@davidsh

FYI: @JunTaoLuo@Tratcher

@tarekgh

tarekgh commented Apr 23, 2020

Copy link
Copy Markdown
Member

CC @tquerec

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CharSet = CharSet.Unicode [](start = 41, length = 25)

why this needed here?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure, this was the way this struct already existed, I just moved them to a new file.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disposeHandle here seems to be redundant with the one from the base class... which you're always giving the literal true.

If it means "false to never call ReleaseHandle", that's exactly what the base bool ownsHandle is for.

fPerformRelease = ((oldState & (SH_State_RefCount | SH_State_Closed)) == SH_RefCountOne) && m_ownsHandle;
if (fPerformRelease)
{
GCPROTECT_BEGIN(sh);
CLR_BOOLfIsInvalid = FALSE;
DECLARE_ARGHOLDER_ARRAY(args, 1);
args[ARGNUM_0] = OBJECTREF_TO_ARGHOLDER(sh);
PREPARE_SIMPLE_VIRTUAL_CALLSITE_USING_SLOT(s_IsInvalidHandleMethodSlot, sh);
CRITICAL_CALLSITE;
CALL_MANAGED_METHOD(fIsInvalid, CLR_BOOL, args);
if (fIsInvalid)
{
fPerformRelease = false;
}
GCPROTECT_END();
}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is your suggestion to basically instead of having a field for _needDispose and checking this in ReleaseHandle, I should just call base passing in disposeHandle value and not need to check in ReleaseHandle as it would not get called in the case it was set to false right? Just making sure I understood this comment..

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually we do have one problem with doing this. The one in the base I believe can't be overwritten after creation, and we have some (internal) methods in LdapConnection that can flip the _needsDispose, so I guess the reason why the original design was like this, was so that ReleaseHandle would get called always, and we would only actually dispose if we need to.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value, (maybe I'm wrong and it just finds a ctor that looks like one it knows how to call? or maybe a default is required and I'm having a very bad recollection day). I'm not sure this code actually prevents invalid handles.

A more canonical representation of this would be to declare the ldap_init call to return a SafeConnectionHandle (instead of IntPtr) and the caller should do

SafeConnectionHandlehandle=LdapPal.Initialize(null,389);

// In LdapPal.Windows.cs

internalstaticSafeConnectionHandleInitialize(stringhost,intport){SafeConnectionHandlehandle=Interop.wldap32.ldap_init(host,port);if(handle.IsInvalid){handle.Dispose();// this error handling code}returnhandle;}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value

Nope :)

e.g. try this:

usingSystem;usingSystem.Runtime.InteropServices;classProgram{staticvoidMain()=>GetStdHandle(-10);[DllImport("kernel32.dll",SetLastError=true)]privatestaticexternMySafeHandleGetStdHandle(intnStdHandle);}classMySafeHandle:SafeHandle{publicMySafeHandle(stringhello,stringworld):base(IntPtr.Zero,ownsHandle:false){}publicoverrideboolIsInvalid=>true;protectedoverrideboolReleaseHandle()=>true;}

You'll get an exception like this:

Unhandled Exception: System.MissingMethodException: .ctor
at Program.GetStdHandle(Int32 nStdHandle)
at Program.Main()


namespace System.DirectoryServices.Protocols
{
internal sealed class ConnectionHandle : SafeHandleZeroOrMinusOneIsInvalid

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is used by the P/Invoke declarations in Interop, I'd put this file in that same directory (similar to the Windows one). e.g. https://github.com/dotnet/runtime/blob/master/src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.OCSP.cs#L103-L134

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Following the OCSP example, you could just nestle the appropriate SafeHandle types in the bottom of the Interop.Ber.cs and Interop.Ldap.cs files, so that if some other library needs them it's just one include)

Comment threadsrc/libraries/System.DirectoryServices.Protocols/tests/BerConverterTests.cs Outdated
public static IEnumerable<object[]> Ctor_BeforeCount_AfterCount_ByteArrayTarget_Data()
{
yield return new object[] { 0, 0, null, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 18, 2, 1, 0, 2, 1, 0, 160, 132, 0, 0, 0, 6, 2, 1, 0, 2, 1, 0 } : new byte[] { 48, 14, 2, 1, 0, 2, 1, 0, 160, 6, 2, 1, 0, 2, 1, 0 } };
yield return new object[] { 10, 10, new byte[] { 1, 2, 3 }, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } : new byte[] { 48, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } };

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For long-term test-building maintenance, consider a helper. E.g.

yieldreturnnewobject[]{0,0,null,FormatLengths("30{0}020100020100A0{1}020100020100",18,6)};
...private static byte[]FormatLengths(stringhexFormat,paramsint[]lengths){string[]hexLengths=newstring[lengths.Length];for(inti=0;i<lengths.Length;i++){intcurLen=lengths[i];if(PlatformDetection.IsWindows){hexLengths[i]=$"84{curLen:X8}";}elseif(curLen<0x80){hexLengths[i]=curLen.ToString("X2");}elseif(curLen<=byte.MaxValue){hexLengths[i]=$"81{curLen:X2}";}elseif(curLen<=ushort.MaxValue){hexLengths[i]=$"82{curLen:X4}";}elseif(curLen<=0xFFFFFF){hexLengths[i]=$"83{curLen:X6}";}else{hexLengths[i]=$"84{curLen:X8}";}}stringhex=string.Format(hexFormat,hexLengths);// This presupposes something like the crypto tests have:returnhex.HexToByteArray();}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this would be valuable, but if that's ok with you I would rather do that in a subsequent PR so to not add more complexity to this PR for now. Once I have tests running on Mono and OSX and passing, then I think it would be a great time to address this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferring the test helper is totally fine.

Comment threadsrc/libraries/Common/src/Interop/Interop.Ldap.cs Outdated
@joperezr
joperezr merged commit 55d3260 into dotnet:masterMay 7, 2020
@joperezr
joperezr deleted the SDSP branch May 7, 2020 20:37
@joperezr

Copy link
Copy Markdown
MemberAuthor

Thanks a lot to all for the reviews! 😄

@danmoseley

Copy link
Copy Markdown
Contributor

Nice! 🔥

@JunTaoLuo

Copy link
Copy Markdown
Contributor

🎆

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support System.DirectoryServices.Protocols on Linux/Mac

9 participants

@joperezr@tarekgh@danmoseley@JunTaoLuo@abatishchev@stephentoub@gfoidl@bartonjs@Dotnet-GitSync-Bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Adding Linux support to System.DirectoryServices.Protocols - #35380

Merged
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP
May 7, 2020
Merged

Adding Linux support to System.DirectoryServices.Protocols#35380
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP

Conversation

@joperezr

Copy link
Copy Markdown
Member

Fixes#23944

Creating this as a draft PR first as I expect a lot of feedback plus some things will still need to happen in order to get tests working in Linux, but I want to start getting feedback so I can address it while I finish setting up running tests in Linux. This adds support to Linux for the System.DirectoryServices.Protocols namespace, by simply creating a PAL layer that abstracts the calls between wldap32 and openldap. Given both native libraries are just implementing ldap protocol, most of their APIs are the same with very similar or same structures which makes it so that managed code didn't actually have to change that much, we just change the native calls we do underneath. There are some open questions, like for example, there are some settings that are specific to wldap32 and not supported on any other library, the question is, should they be No-Ops for compatibility with previous apps working in Windows or should they instead throw PNSE?

One thing that is missing from this PR is the support for Default Credentials. That means that if you have your code running on a Kestrel Server which is domain-joined to an AD and have a valid session token already authenticated in the machine, then you wouldn't need to pass in credentials again when creating an LDAPConnection since you would just reuse that token. That works today in Windows, and I'm working on fixing the native calls on Bind to make sure that it also works on Linux. That support will come either as part of this PR later, or it will be added as a subsequent PR.

cc: @ericstj@tarekgh@bartonjs@stephentoub@davidsh

FYI: @JunTaoLuo@Tratcher

@tarekgh

tarekgh commented Apr 23, 2020

Copy link
Copy Markdown
Member

CC @tquerec

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CharSet = CharSet.Unicode [](start = 41, length = 25)

why this needed here?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure, this was the way this struct already existed, I just moved them to a new file.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disposeHandle here seems to be redundant with the one from the base class... which you're always giving the literal true.

If it means "false to never call ReleaseHandle", that's exactly what the base bool ownsHandle is for.

fPerformRelease = ((oldState & (SH_State_RefCount | SH_State_Closed)) == SH_RefCountOne) && m_ownsHandle;
if (fPerformRelease)
{
GCPROTECT_BEGIN(sh);
CLR_BOOLfIsInvalid = FALSE;
DECLARE_ARGHOLDER_ARRAY(args, 1);
args[ARGNUM_0] = OBJECTREF_TO_ARGHOLDER(sh);
PREPARE_SIMPLE_VIRTUAL_CALLSITE_USING_SLOT(s_IsInvalidHandleMethodSlot, sh);
CRITICAL_CALLSITE;
CALL_MANAGED_METHOD(fIsInvalid, CLR_BOOL, args);
if (fIsInvalid)
{
fPerformRelease = false;
}
GCPROTECT_END();
}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is your suggestion to basically instead of having a field for _needDispose and checking this in ReleaseHandle, I should just call base passing in disposeHandle value and not need to check in ReleaseHandle as it would not get called in the case it was set to false right? Just making sure I understood this comment..

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually we do have one problem with doing this. The one in the base I believe can't be overwritten after creation, and we have some (internal) methods in LdapConnection that can flip the _needsDispose, so I guess the reason why the original design was like this, was so that ReleaseHandle would get called always, and we would only actually dispose if we need to.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value, (maybe I'm wrong and it just finds a ctor that looks like one it knows how to call? or maybe a default is required and I'm having a very bad recollection day). I'm not sure this code actually prevents invalid handles.

A more canonical representation of this would be to declare the ldap_init call to return a SafeConnectionHandle (instead of IntPtr) and the caller should do

SafeConnectionHandlehandle=LdapPal.Initialize(null,389);

// In LdapPal.Windows.cs

internalstaticSafeConnectionHandleInitialize(stringhost,intport){SafeConnectionHandlehandle=Interop.wldap32.ldap_init(host,port);if(handle.IsInvalid){handle.Dispose();// this error handling code}returnhandle;}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value

Nope :)

e.g. try this:

usingSystem;usingSystem.Runtime.InteropServices;classProgram{staticvoidMain()=>GetStdHandle(-10);[DllImport("kernel32.dll",SetLastError=true)]privatestaticexternMySafeHandleGetStdHandle(intnStdHandle);}classMySafeHandle:SafeHandle{publicMySafeHandle(stringhello,stringworld):base(IntPtr.Zero,ownsHandle:false){}publicoverrideboolIsInvalid=>true;protectedoverrideboolReleaseHandle()=>true;}

You'll get an exception like this:

Unhandled Exception: System.MissingMethodException: .ctor
at Program.GetStdHandle(Int32 nStdHandle)
at Program.Main()


namespace System.DirectoryServices.Protocols
{
internal sealed class ConnectionHandle : SafeHandleZeroOrMinusOneIsInvalid

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is used by the P/Invoke declarations in Interop, I'd put this file in that same directory (similar to the Windows one). e.g. https://github.com/dotnet/runtime/blob/master/src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.OCSP.cs#L103-L134

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Following the OCSP example, you could just nestle the appropriate SafeHandle types in the bottom of the Interop.Ber.cs and Interop.Ldap.cs files, so that if some other library needs them it's just one include)

Comment threadsrc/libraries/System.DirectoryServices.Protocols/tests/BerConverterTests.cs Outdated
public static IEnumerable<object[]> Ctor_BeforeCount_AfterCount_ByteArrayTarget_Data()
{
yield return new object[] { 0, 0, null, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 18, 2, 1, 0, 2, 1, 0, 160, 132, 0, 0, 0, 6, 2, 1, 0, 2, 1, 0 } : new byte[] { 48, 14, 2, 1, 0, 2, 1, 0, 160, 6, 2, 1, 0, 2, 1, 0 } };
yield return new object[] { 10, 10, new byte[] { 1, 2, 3 }, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } : new byte[] { 48, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } };

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For long-term test-building maintenance, consider a helper. E.g.

yieldreturnnewobject[]{0,0,null,FormatLengths("30{0}020100020100A0{1}020100020100",18,6)};
...private static byte[]FormatLengths(stringhexFormat,paramsint[]lengths){string[]hexLengths=newstring[lengths.Length];for(inti=0;i<lengths.Length;i++){intcurLen=lengths[i];if(PlatformDetection.IsWindows){hexLengths[i]=$"84{curLen:X8}";}elseif(curLen<0x80){hexLengths[i]=curLen.ToString("X2");}elseif(curLen<=byte.MaxValue){hexLengths[i]=$"81{curLen:X2}";}elseif(curLen<=ushort.MaxValue){hexLengths[i]=$"82{curLen:X4}";}elseif(curLen<=0xFFFFFF){hexLengths[i]=$"83{curLen:X6}";}else{hexLengths[i]=$"84{curLen:X8}";}}stringhex=string.Format(hexFormat,hexLengths);// This presupposes something like the crypto tests have:returnhex.HexToByteArray();}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this would be valuable, but if that's ok with you I would rather do that in a subsequent PR so to not add more complexity to this PR for now. Once I have tests running on Mono and OSX and passing, then I think it would be a great time to address this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferring the test helper is totally fine.

Comment threadsrc/libraries/Common/src/Interop/Interop.Ldap.cs Outdated
@joperezr
joperezr merged commit 55d3260 into dotnet:masterMay 7, 2020
@joperezr
joperezr deleted the SDSP branch May 7, 2020 20:37
@joperezr

Copy link
Copy Markdown
MemberAuthor

Thanks a lot to all for the reviews! 😄

@danmoseley

Copy link
Copy Markdown
Contributor

Nice! 🔥

@JunTaoLuo

Copy link
Copy Markdown
Contributor

🎆

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support System.DirectoryServices.Protocols on Linux/Mac

9 participants

@joperezr@tarekgh@danmoseley@JunTaoLuo@abatishchev@stephentoub@gfoidl@bartonjs@Dotnet-GitSync-Bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Adding Linux support to System.DirectoryServices.Protocols - #35380

Merged
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP
May 7, 2020
Merged

Adding Linux support to System.DirectoryServices.Protocols#35380
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP

Conversation

@joperezr

Copy link
Copy Markdown
Member

Fixes#23944

Creating this as a draft PR first as I expect a lot of feedback plus some things will still need to happen in order to get tests working in Linux, but I want to start getting feedback so I can address it while I finish setting up running tests in Linux. This adds support to Linux for the System.DirectoryServices.Protocols namespace, by simply creating a PAL layer that abstracts the calls between wldap32 and openldap. Given both native libraries are just implementing ldap protocol, most of their APIs are the same with very similar or same structures which makes it so that managed code didn't actually have to change that much, we just change the native calls we do underneath. There are some open questions, like for example, there are some settings that are specific to wldap32 and not supported on any other library, the question is, should they be No-Ops for compatibility with previous apps working in Windows or should they instead throw PNSE?

One thing that is missing from this PR is the support for Default Credentials. That means that if you have your code running on a Kestrel Server which is domain-joined to an AD and have a valid session token already authenticated in the machine, then you wouldn't need to pass in credentials again when creating an LDAPConnection since you would just reuse that token. That works today in Windows, and I'm working on fixing the native calls on Bind to make sure that it also works on Linux. That support will come either as part of this PR later, or it will be added as a subsequent PR.

cc: @ericstj@tarekgh@bartonjs@stephentoub@davidsh

FYI: @JunTaoLuo@Tratcher

@tarekgh

tarekgh commented Apr 23, 2020

Copy link
Copy Markdown
Member

CC @tquerec

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CharSet = CharSet.Unicode [](start = 41, length = 25)

why this needed here?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure, this was the way this struct already existed, I just moved them to a new file.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disposeHandle here seems to be redundant with the one from the base class... which you're always giving the literal true.

If it means "false to never call ReleaseHandle", that's exactly what the base bool ownsHandle is for.

fPerformRelease = ((oldState & (SH_State_RefCount | SH_State_Closed)) == SH_RefCountOne) && m_ownsHandle;
if (fPerformRelease)
{
GCPROTECT_BEGIN(sh);
CLR_BOOLfIsInvalid = FALSE;
DECLARE_ARGHOLDER_ARRAY(args, 1);
args[ARGNUM_0] = OBJECTREF_TO_ARGHOLDER(sh);
PREPARE_SIMPLE_VIRTUAL_CALLSITE_USING_SLOT(s_IsInvalidHandleMethodSlot, sh);
CRITICAL_CALLSITE;
CALL_MANAGED_METHOD(fIsInvalid, CLR_BOOL, args);
if (fIsInvalid)
{
fPerformRelease = false;
}
GCPROTECT_END();
}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is your suggestion to basically instead of having a field for _needDispose and checking this in ReleaseHandle, I should just call base passing in disposeHandle value and not need to check in ReleaseHandle as it would not get called in the case it was set to false right? Just making sure I understood this comment..

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually we do have one problem with doing this. The one in the base I believe can't be overwritten after creation, and we have some (internal) methods in LdapConnection that can flip the _needsDispose, so I guess the reason why the original design was like this, was so that ReleaseHandle would get called always, and we would only actually dispose if we need to.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value, (maybe I'm wrong and it just finds a ctor that looks like one it knows how to call? or maybe a default is required and I'm having a very bad recollection day). I'm not sure this code actually prevents invalid handles.

A more canonical representation of this would be to declare the ldap_init call to return a SafeConnectionHandle (instead of IntPtr) and the caller should do

SafeConnectionHandlehandle=LdapPal.Initialize(null,389);

// In LdapPal.Windows.cs

internalstaticSafeConnectionHandleInitialize(stringhost,intport){SafeConnectionHandlehandle=Interop.wldap32.ldap_init(host,port);if(handle.IsInvalid){handle.Dispose();// this error handling code}returnhandle;}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value

Nope :)

e.g. try this:

usingSystem;usingSystem.Runtime.InteropServices;classProgram{staticvoidMain()=>GetStdHandle(-10);[DllImport("kernel32.dll",SetLastError=true)]privatestaticexternMySafeHandleGetStdHandle(intnStdHandle);}classMySafeHandle:SafeHandle{publicMySafeHandle(stringhello,stringworld):base(IntPtr.Zero,ownsHandle:false){}publicoverrideboolIsInvalid=>true;protectedoverrideboolReleaseHandle()=>true;}

You'll get an exception like this:

Unhandled Exception: System.MissingMethodException: .ctor
at Program.GetStdHandle(Int32 nStdHandle)
at Program.Main()


namespace System.DirectoryServices.Protocols
{
internal sealed class ConnectionHandle : SafeHandleZeroOrMinusOneIsInvalid

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is used by the P/Invoke declarations in Interop, I'd put this file in that same directory (similar to the Windows one). e.g. https://github.com/dotnet/runtime/blob/master/src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.OCSP.cs#L103-L134

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Following the OCSP example, you could just nestle the appropriate SafeHandle types in the bottom of the Interop.Ber.cs and Interop.Ldap.cs files, so that if some other library needs them it's just one include)

Comment threadsrc/libraries/System.DirectoryServices.Protocols/tests/BerConverterTests.cs Outdated
public static IEnumerable<object[]> Ctor_BeforeCount_AfterCount_ByteArrayTarget_Data()
{
yield return new object[] { 0, 0, null, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 18, 2, 1, 0, 2, 1, 0, 160, 132, 0, 0, 0, 6, 2, 1, 0, 2, 1, 0 } : new byte[] { 48, 14, 2, 1, 0, 2, 1, 0, 160, 6, 2, 1, 0, 2, 1, 0 } };
yield return new object[] { 10, 10, new byte[] { 1, 2, 3 }, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } : new byte[] { 48, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } };

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For long-term test-building maintenance, consider a helper. E.g.

yieldreturnnewobject[]{0,0,null,FormatLengths("30{0}020100020100A0{1}020100020100",18,6)};
...private static byte[]FormatLengths(stringhexFormat,paramsint[]lengths){string[]hexLengths=newstring[lengths.Length];for(inti=0;i<lengths.Length;i++){intcurLen=lengths[i];if(PlatformDetection.IsWindows){hexLengths[i]=$"84{curLen:X8}";}elseif(curLen<0x80){hexLengths[i]=curLen.ToString("X2");}elseif(curLen<=byte.MaxValue){hexLengths[i]=$"81{curLen:X2}";}elseif(curLen<=ushort.MaxValue){hexLengths[i]=$"82{curLen:X4}";}elseif(curLen<=0xFFFFFF){hexLengths[i]=$"83{curLen:X6}";}else{hexLengths[i]=$"84{curLen:X8}";}}stringhex=string.Format(hexFormat,hexLengths);// This presupposes something like the crypto tests have:returnhex.HexToByteArray();}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this would be valuable, but if that's ok with you I would rather do that in a subsequent PR so to not add more complexity to this PR for now. Once I have tests running on Mono and OSX and passing, then I think it would be a great time to address this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferring the test helper is totally fine.

Comment threadsrc/libraries/Common/src/Interop/Interop.Ldap.cs Outdated
@joperezr
joperezr merged commit 55d3260 into dotnet:masterMay 7, 2020
@joperezr
joperezr deleted the SDSP branch May 7, 2020 20:37
@joperezr

Copy link
Copy Markdown
MemberAuthor

Thanks a lot to all for the reviews! 😄

@danmoseley

Copy link
Copy Markdown
Contributor

Nice! 🔥

@JunTaoLuo

Copy link
Copy Markdown
Contributor

🎆

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support System.DirectoryServices.Protocols on Linux/Mac

9 participants

@joperezr@tarekgh@danmoseley@JunTaoLuo@abatishchev@stephentoub@gfoidl@bartonjs@Dotnet-GitSync-Bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Adding Linux support to System.DirectoryServices.Protocols - #35380

Merged
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP
May 7, 2020
Merged

Adding Linux support to System.DirectoryServices.Protocols#35380
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP

Conversation

@joperezr

Copy link
Copy Markdown
Member

Fixes#23944

Creating this as a draft PR first as I expect a lot of feedback plus some things will still need to happen in order to get tests working in Linux, but I want to start getting feedback so I can address it while I finish setting up running tests in Linux. This adds support to Linux for the System.DirectoryServices.Protocols namespace, by simply creating a PAL layer that abstracts the calls between wldap32 and openldap. Given both native libraries are just implementing ldap protocol, most of their APIs are the same with very similar or same structures which makes it so that managed code didn't actually have to change that much, we just change the native calls we do underneath. There are some open questions, like for example, there are some settings that are specific to wldap32 and not supported on any other library, the question is, should they be No-Ops for compatibility with previous apps working in Windows or should they instead throw PNSE?

One thing that is missing from this PR is the support for Default Credentials. That means that if you have your code running on a Kestrel Server which is domain-joined to an AD and have a valid session token already authenticated in the machine, then you wouldn't need to pass in credentials again when creating an LDAPConnection since you would just reuse that token. That works today in Windows, and I'm working on fixing the native calls on Bind to make sure that it also works on Linux. That support will come either as part of this PR later, or it will be added as a subsequent PR.

cc: @ericstj@tarekgh@bartonjs@stephentoub@davidsh

FYI: @JunTaoLuo@Tratcher

@tarekgh

tarekgh commented Apr 23, 2020

Copy link
Copy Markdown
Member

CC @tquerec

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CharSet = CharSet.Unicode [](start = 41, length = 25)

why this needed here?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure, this was the way this struct already existed, I just moved them to a new file.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disposeHandle here seems to be redundant with the one from the base class... which you're always giving the literal true.

If it means "false to never call ReleaseHandle", that's exactly what the base bool ownsHandle is for.

fPerformRelease = ((oldState & (SH_State_RefCount | SH_State_Closed)) == SH_RefCountOne) && m_ownsHandle;
if (fPerformRelease)
{
GCPROTECT_BEGIN(sh);
CLR_BOOLfIsInvalid = FALSE;
DECLARE_ARGHOLDER_ARRAY(args, 1);
args[ARGNUM_0] = OBJECTREF_TO_ARGHOLDER(sh);
PREPARE_SIMPLE_VIRTUAL_CALLSITE_USING_SLOT(s_IsInvalidHandleMethodSlot, sh);
CRITICAL_CALLSITE;
CALL_MANAGED_METHOD(fIsInvalid, CLR_BOOL, args);
if (fIsInvalid)
{
fPerformRelease = false;
}
GCPROTECT_END();
}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is your suggestion to basically instead of having a field for _needDispose and checking this in ReleaseHandle, I should just call base passing in disposeHandle value and not need to check in ReleaseHandle as it would not get called in the case it was set to false right? Just making sure I understood this comment..

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually we do have one problem with doing this. The one in the base I believe can't be overwritten after creation, and we have some (internal) methods in LdapConnection that can flip the _needsDispose, so I guess the reason why the original design was like this, was so that ReleaseHandle would get called always, and we would only actually dispose if we need to.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value, (maybe I'm wrong and it just finds a ctor that looks like one it knows how to call? or maybe a default is required and I'm having a very bad recollection day). I'm not sure this code actually prevents invalid handles.

A more canonical representation of this would be to declare the ldap_init call to return a SafeConnectionHandle (instead of IntPtr) and the caller should do

SafeConnectionHandlehandle=LdapPal.Initialize(null,389);

// In LdapPal.Windows.cs

internalstaticSafeConnectionHandleInitialize(stringhost,intport){SafeConnectionHandlehandle=Interop.wldap32.ldap_init(host,port);if(handle.IsInvalid){handle.Dispose();// this error handling code}returnhandle;}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value

Nope :)

e.g. try this:

usingSystem;usingSystem.Runtime.InteropServices;classProgram{staticvoidMain()=>GetStdHandle(-10);[DllImport("kernel32.dll",SetLastError=true)]privatestaticexternMySafeHandleGetStdHandle(intnStdHandle);}classMySafeHandle:SafeHandle{publicMySafeHandle(stringhello,stringworld):base(IntPtr.Zero,ownsHandle:false){}publicoverrideboolIsInvalid=>true;protectedoverrideboolReleaseHandle()=>true;}

You'll get an exception like this:

Unhandled Exception: System.MissingMethodException: .ctor
at Program.GetStdHandle(Int32 nStdHandle)
at Program.Main()


namespace System.DirectoryServices.Protocols
{
internal sealed class ConnectionHandle : SafeHandleZeroOrMinusOneIsInvalid

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is used by the P/Invoke declarations in Interop, I'd put this file in that same directory (similar to the Windows one). e.g. https://github.com/dotnet/runtime/blob/master/src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.OCSP.cs#L103-L134

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Following the OCSP example, you could just nestle the appropriate SafeHandle types in the bottom of the Interop.Ber.cs and Interop.Ldap.cs files, so that if some other library needs them it's just one include)

Comment threadsrc/libraries/System.DirectoryServices.Protocols/tests/BerConverterTests.cs Outdated
public static IEnumerable<object[]> Ctor_BeforeCount_AfterCount_ByteArrayTarget_Data()
{
yield return new object[] { 0, 0, null, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 18, 2, 1, 0, 2, 1, 0, 160, 132, 0, 0, 0, 6, 2, 1, 0, 2, 1, 0 } : new byte[] { 48, 14, 2, 1, 0, 2, 1, 0, 160, 6, 2, 1, 0, 2, 1, 0 } };
yield return new object[] { 10, 10, new byte[] { 1, 2, 3 }, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } : new byte[] { 48, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } };

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For long-term test-building maintenance, consider a helper. E.g.

yieldreturnnewobject[]{0,0,null,FormatLengths("30{0}020100020100A0{1}020100020100",18,6)};
...private static byte[]FormatLengths(stringhexFormat,paramsint[]lengths){string[]hexLengths=newstring[lengths.Length];for(inti=0;i<lengths.Length;i++){intcurLen=lengths[i];if(PlatformDetection.IsWindows){hexLengths[i]=$"84{curLen:X8}";}elseif(curLen<0x80){hexLengths[i]=curLen.ToString("X2");}elseif(curLen<=byte.MaxValue){hexLengths[i]=$"81{curLen:X2}";}elseif(curLen<=ushort.MaxValue){hexLengths[i]=$"82{curLen:X4}";}elseif(curLen<=0xFFFFFF){hexLengths[i]=$"83{curLen:X6}";}else{hexLengths[i]=$"84{curLen:X8}";}}stringhex=string.Format(hexFormat,hexLengths);// This presupposes something like the crypto tests have:returnhex.HexToByteArray();}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this would be valuable, but if that's ok with you I would rather do that in a subsequent PR so to not add more complexity to this PR for now. Once I have tests running on Mono and OSX and passing, then I think it would be a great time to address this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferring the test helper is totally fine.

Comment threadsrc/libraries/Common/src/Interop/Interop.Ldap.cs Outdated
@joperezr
joperezr merged commit 55d3260 into dotnet:masterMay 7, 2020
@joperezr
joperezr deleted the SDSP branch May 7, 2020 20:37
@joperezr

Copy link
Copy Markdown
MemberAuthor

Thanks a lot to all for the reviews! 😄

@danmoseley

Copy link
Copy Markdown
Contributor

Nice! 🔥

@JunTaoLuo

Copy link
Copy Markdown
Contributor

🎆

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support System.DirectoryServices.Protocols on Linux/Mac

9 participants

@joperezr@tarekgh@danmoseley@JunTaoLuo@abatishchev@stephentoub@gfoidl@bartonjs@Dotnet-GitSync-Bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Adding Linux support to System.DirectoryServices.Protocols - #35380

Merged
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP
May 7, 2020
Merged

Adding Linux support to System.DirectoryServices.Protocols#35380
joperezr merged 10 commits into
dotnet:masterfrom
joperezr:SDSP

Conversation

@joperezr

Copy link
Copy Markdown
Member

Fixes#23944

Creating this as a draft PR first as I expect a lot of feedback plus some things will still need to happen in order to get tests working in Linux, but I want to start getting feedback so I can address it while I finish setting up running tests in Linux. This adds support to Linux for the System.DirectoryServices.Protocols namespace, by simply creating a PAL layer that abstracts the calls between wldap32 and openldap. Given both native libraries are just implementing ldap protocol, most of their APIs are the same with very similar or same structures which makes it so that managed code didn't actually have to change that much, we just change the native calls we do underneath. There are some open questions, like for example, there are some settings that are specific to wldap32 and not supported on any other library, the question is, should they be No-Ops for compatibility with previous apps working in Windows or should they instead throw PNSE?

One thing that is missing from this PR is the support for Default Credentials. That means that if you have your code running on a Kestrel Server which is domain-joined to an AD and have a valid session token already authenticated in the machine, then you wouldn't need to pass in credentials again when creating an LDAPConnection since you would just reuse that token. That works today in Windows, and I'm working on fixing the native calls on Bind to make sure that it also works on Linux. That support will come either as part of this PR later, or it will be added as a subsequent PR.

cc: @ericstj@tarekgh@bartonjs@stephentoub@davidsh

FYI: @JunTaoLuo@Tratcher

@tarekgh

tarekgh commented Apr 23, 2020

Copy link
Copy Markdown
Member

CC @tquerec

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CharSet = CharSet.Unicode [](start = 41, length = 25)

why this needed here?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure, this was the way this struct already existed, I just moved them to a new file.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disposeHandle here seems to be redundant with the one from the base class... which you're always giving the literal true.

If it means "false to never call ReleaseHandle", that's exactly what the base bool ownsHandle is for.

fPerformRelease = ((oldState & (SH_State_RefCount | SH_State_Closed)) == SH_RefCountOne) && m_ownsHandle;
if (fPerformRelease)
{
GCPROTECT_BEGIN(sh);
CLR_BOOLfIsInvalid = FALSE;
DECLARE_ARGHOLDER_ARRAY(args, 1);
args[ARGNUM_0] = OBJECTREF_TO_ARGHOLDER(sh);
PREPARE_SIMPLE_VIRTUAL_CALLSITE_USING_SLOT(s_IsInvalidHandleMethodSlot, sh);
CRITICAL_CALLSITE;
CALL_MANAGED_METHOD(fIsInvalid, CLR_BOOL, args);
if (fIsInvalid)
{
fPerformRelease = false;
}
GCPROTECT_END();
}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is your suggestion to basically instead of having a field for _needDispose and checking this in ReleaseHandle, I should just call base passing in disposeHandle value and not need to check in ReleaseHandle as it would not get called in the case it was set to false right? Just making sure I understood this comment..

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually we do have one problem with doing this. The one in the base I believe can't be overwritten after creation, and we have some (internal) methods in LdapConnection that can flip the _needsDispose, so I guess the reason why the original design was like this, was so that ReleaseHandle would get called always, and we would only actually dispose if we need to.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value, (maybe I'm wrong and it just finds a ctor that looks like one it knows how to call? or maybe a default is required and I'm having a very bad recollection day). I'm not sure this code actually prevents invalid handles.

A more canonical representation of this would be to declare the ldap_init call to return a SafeConnectionHandle (instead of IntPtr) and the caller should do

SafeConnectionHandlehandle=LdapPal.Initialize(null,389);

// In LdapPal.Windows.cs

internalstaticSafeConnectionHandleInitialize(stringhost,intport){SafeConnectionHandlehandle=Interop.wldap32.ldap_init(host,port);if(handle.IsInvalid){handle.Dispose();// this error handling code}returnhandle;}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the marshaller bypasses ctors when it creates SafeHandles as a return value

Nope :)

e.g. try this:

usingSystem;usingSystem.Runtime.InteropServices;classProgram{staticvoidMain()=>GetStdHandle(-10);[DllImport("kernel32.dll",SetLastError=true)]privatestaticexternMySafeHandleGetStdHandle(intnStdHandle);}classMySafeHandle:SafeHandle{publicMySafeHandle(stringhello,stringworld):base(IntPtr.Zero,ownsHandle:false){}publicoverrideboolIsInvalid=>true;protectedoverrideboolReleaseHandle()=>true;}

You'll get an exception like this:

Unhandled Exception: System.MissingMethodException: .ctor
at Program.GetStdHandle(Int32 nStdHandle)
at Program.Main()


namespace System.DirectoryServices.Protocols
{
internal sealed class ConnectionHandle : SafeHandleZeroOrMinusOneIsInvalid

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is used by the P/Invoke declarations in Interop, I'd put this file in that same directory (similar to the Windows one). e.g. https://github.com/dotnet/runtime/blob/master/src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.OCSP.cs#L103-L134

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Following the OCSP example, you could just nestle the appropriate SafeHandle types in the bottom of the Interop.Ber.cs and Interop.Ldap.cs files, so that if some other library needs them it's just one include)

Comment threadsrc/libraries/System.DirectoryServices.Protocols/tests/BerConverterTests.cs Outdated
public static IEnumerable<object[]> Ctor_BeforeCount_AfterCount_ByteArrayTarget_Data()
{
yield return new object[] { 0, 0, null, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 18, 2, 1, 0, 2, 1, 0, 160, 132, 0, 0, 0, 6, 2, 1, 0, 2, 1, 0 } : new byte[] { 48, 14, 2, 1, 0, 2, 1, 0, 160, 6, 2, 1, 0, 2, 1, 0 } };
yield return new object[] { 10, 10, new byte[] { 1, 2, 3 }, (PlatformDetection.IsWindows) ? new byte[] { 48, 132, 0, 0, 0, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } : new byte[] { 48, 11, 2, 1, 10, 2, 1, 10, 129, 3, 1, 2, 3 } };

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For long-term test-building maintenance, consider a helper. E.g.

yieldreturnnewobject[]{0,0,null,FormatLengths("30{0}020100020100A0{1}020100020100",18,6)};
...private static byte[]FormatLengths(stringhexFormat,paramsint[]lengths){string[]hexLengths=newstring[lengths.Length];for(inti=0;i<lengths.Length;i++){intcurLen=lengths[i];if(PlatformDetection.IsWindows){hexLengths[i]=$"84{curLen:X8}";}elseif(curLen<0x80){hexLengths[i]=curLen.ToString("X2");}elseif(curLen<=byte.MaxValue){hexLengths[i]=$"81{curLen:X2}";}elseif(curLen<=ushort.MaxValue){hexLengths[i]=$"82{curLen:X4}";}elseif(curLen<=0xFFFFFF){hexLengths[i]=$"83{curLen:X6}";}else{hexLengths[i]=$"84{curLen:X8}";}}stringhex=string.Format(hexFormat,hexLengths);// This presupposes something like the crypto tests have:returnhex.HexToByteArray();}

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this would be valuable, but if that's ok with you I would rather do that in a subsequent PR so to not add more complexity to this PR for now. Once I have tests running on Mono and OSX and passing, then I think it would be a great time to address this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deferring the test helper is totally fine.

Comment threadsrc/libraries/Common/src/Interop/Interop.Ldap.cs Outdated
@joperezr
joperezr merged commit 55d3260 into dotnet:masterMay 7, 2020
@joperezr
joperezr deleted the SDSP branch May 7, 2020 20:37
@joperezr

Copy link
Copy Markdown
MemberAuthor

Thanks a lot to all for the reviews! 😄

@danmoseley

Copy link
Copy Markdown
Contributor

Nice! 🔥

@JunTaoLuo

Copy link
Copy Markdown
Contributor

🎆

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support System.DirectoryServices.Protocols on Linux/Mac

9 participants

@joperezr@tarekgh@danmoseley@JunTaoLuo@abatishchev@stephentoub@gfoidl@bartonjs@Dotnet-GitSync-Bot