Enable building NativeAOT for Apple mobile platforms - #81780

Merged
akoeplinger merged 16 commits into
dotnet:mainfrom
akoeplinger:nativeaot-mobile
Feb 16, 2023
Merged

Enable building NativeAOT for Apple mobile platforms#81780
akoeplinger merged 16 commits into
dotnet:mainfrom
akoeplinger:nativeaot-mobile

Conversation

@akoeplinger

@akoeplingerakoeplinger commented Feb 7, 2023

Copy link
Copy Markdown
Member

These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.

Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.

/cc @dotnet/ilc-contrib @ivanpovazan

@akoeplinger
Enable building NativeAOT for Apple mobile platforms
268e281
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to dotnet#81024
@ghostghost assigned akoeplingerFeb 7, 2023
@ghost

ghost commented Feb 7, 2023

Copy link
Copy Markdown

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.

Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.

/cc @dotnet/ilc-contrib @ivanpovazan

Author:akoeplinger
Assignees:akoeplinger
Labels:

area-NativeAOT-coreclr

Milestone:-

akoeplinger
akoeplinger commented Feb 7, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/thread/context.cpp Outdated
@akoeplingerakoeplinger mentioned this pull request Feb 7, 2023
Closed
4 tasks
@akoeplingerakoeplinger linked an issue Feb 7, 2023 that may be closed by this pull request
Enable building NativeAOT for Apple mobile platforms #81024
Closed
4 tasks
@akoeplingerakoeplinger removed a link to an issue Feb 7, 2023
Enable building NativeAOT for Apple mobile platforms #81024
Closed
4 tasks
MichalStrehovsky
MichalStrehovsky reviewed Feb 8, 2023
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/System/Globalization/GlobalizationMode.LoadICU.iOS.cs Outdated
Comment threadsrc/coreclr/pal/src/thread/context.cpp Outdated
Comment threadeng/native/build-commons.sh Outdated
Comment threadsrc/coreclr/pal/src/thread/process.cpp Outdated
marek-safar
marek-safar reviewed Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
ivanpovazan
ivanpovazan reviewed Feb 8, 2023
View reviewed changes
Comment threadsrc/mono/Directory.Build.props Outdated
ivanpovazan
ivanpovazan reviewed Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp Outdated
ivanpovazan
ivanpovazan reviewed Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp Outdated
ivanpovazan
ivanpovazan reviewed Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/map/virtual.cpp Outdated
ivanpovazan
ivanpovazan reviewed Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
@filipnavara

filipnavara commented Feb 8, 2023•
edited
Loading

Copy link
Copy Markdown
Member

W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.

@ivanpovazan

ivanpovazan commented Feb 8, 2023

Copy link
Copy Markdown
Member

W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.

Good point! My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates. Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the top-level tracking issue for NativeAOT on iOS work (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.

@filipnavara

filipnavara commented Feb 8, 2023

Copy link
Copy Markdown
Member

My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates.

Yes. It's used for delegates that have closure over some variables and need an associated context to reference the closure object, and the delegate is passed to native code. There's some test for it in src/tests/nativeaot/SmokeTests/PInvoke IIRC.

Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the #80905 (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.

I think it may be worth creating an issue so that the knowledge about FEATURE_FIXED_POOL_THUNKS is not lost in unrelated PRs/issues. I mainly mentioned it because this PR touched code paths in FEATURE_RX_THUNKS that would not really be usable long term...

@filipnavara

filipnavara commented Feb 9, 2023•
edited
Loading

Copy link
Copy Markdown
Member

Turns out there are actually three different code paths for the thunks:

  • FEATURE_RX_THUNKS where thunk pages are generated at runtime (cannot work on iOS)
  • FEATURE_FIXED_POOL_THUNKS where the thunks are pre-generated but limited to certain number at compile time (used by some unofficial console ports)
  • If neither of the feature flags is set, then it runs in the mode with a pre-generated thunk page that is mapped multiple times

Currently there's no Unix code for anything but the FEATURE_RX_THUNKS mode. I wrote a small experimental app that mmaped its own executable page and run a code from it. It works on macOS/arm64 without any of the MAP_JIT flags, so it's likely work on iOS too. That makes the last mode look like the most viable one.

UPD: I updated my prototype to generate the static thunks, on macOS it passes the PInvoke test, so that's a good start.

janvorli
janvorli reviewed Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/coreclr/pal/src/thread/process.cpp Outdated
Comment threadsrc/coreclr/inc/executableallocator.h Outdated
@ivanpovazanivanpovazan mentioned this pull request Feb 14, 2023
Closed
akoeplinger
akoeplinger commented Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/dlls/mscoree/coreclr/CMakeLists.txt
endif(FEATURE_MERGE_JIT_AND_ENGINE)

if (CLR_CMAKE_TARGET_OSX OR CLR_CMAKE_TARGET_MACCATALYST)
if (CLR_CMAKE_TARGET_OSX)

@akoeplingerakoeplingerFeb 14, 2023•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was erroneously added in 45887d1, MacCatalyst isn't building this piece.

janvorli
janvorli reviewed Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/gc.cpp
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/native/corehost/apphost/standalone/CMakeLists.txt Outdated
@filipnavarafilipnavara mentioned this pull request Feb 14, 2023
Merged
kotlarmilos
kotlarmilos reviewed Feb 15, 2023
View reviewed changes
Comment threadeng/native/configurecompiler.cmake
jkotas
jkotas reviewed Feb 15, 2023
View reviewed changes
Comment threadsrc/native/corehost/apphost/static/CMakeLists.txt
janvorli
janvorli reviewed Feb 15, 2023
View reviewed changes
Comment threadsrc/coreclr/CMakeLists.txt Outdated
janvorli
janvorli approved these changes Feb 15, 2023
View reviewed changes

@janvorlijanvorli left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!

@build-analysisbuild-analysisBot mentioned this pull request Feb 15, 2023
Closed
@akoeplinger
akoeplinger merged commit e56db76 into dotnet:mainFeb 16, 2023
@akoeplinger
akoeplinger deleted the nativeaot-mobile branch February 16, 2023 01:00
@ghostghost locked as resolved and limited conversation to collaborators Mar 18, 2023
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@marek-safarmarek-safarmarek-safar left review comments
@jkotasjkotasjkotas left review comments
@kotlarmiloskotlarmiloskotlarmilos left review comments
@ivanpovazanivanpovazanivanpovazan left review comments
@janvorlijanvorlijanvorli approved these changes
@MichalStrehovskyMichalStrehovskyAwaiting requested review from MichalStrehovskyMichalStrehovsky is a code owner
+1 more reviewer
@filipnavarafilipnavarafilipnavara left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@akoeplingerakoeplinger

Labels

area-NativeAOT-coreclr

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@akoeplinger@filipnavara@ivanpovazan@marek-safar@jkotas@janvorli@kotlarmilos@MichalStrehovsky
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Enable building NativeAOT for Apple mobile platforms - #81780
Merged
akoeplinger merged 16 commits into
dotnet:mainfrom 
akoeplinger:nativeaot-mobile
Feb 16, 2023
Merged
Enable building NativeAOT for Apple mobile platforms#81780
akoeplinger merged 16 commits into
dotnet:mainfrom 
akoeplinger:nativeaot-mobile
Conversation
@akoeplinger
@akoeplingerakoeplinger
commented
Feb 7, 2023

Copy link


Copy Markdown

Member
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.
/cc @dotnet/ilc-contrib @ivanpovazan
@akoeplinger
Enable building NativeAOT for Apple mobile platforms
268e281
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to dotnet#81024
@ghostghost
assigned akoeplingerFeb 7, 2023
@ghost
ghost
commented
Feb 7, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas

See info in area-owners.md if you want to be subscribed.
Issue Details
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.
/cc @dotnet/ilc-contrib @ivanpovazan
Author:akoeplinger
Assignees:akoeplinger
Labels:
area-NativeAOT-coreclr
Milestone:-
akoeplinger
akoeplinger
commented
Feb 7, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/thread/context.cpp
Outdated

@akoeplingerakoeplinger
mentioned this pull request
Feb 7, 2023

Closed

4 tasks
@akoeplingerakoeplinger
linked an issue
Feb 7, 2023
that may be
closed
by this pull request


Enable building NativeAOT for Apple mobile platforms
#81024

Closed

4 tasks
@akoeplingerakoeplinger
removed a link to an issue
Feb 7, 2023

Enable building NativeAOT for Apple mobile platforms
#81024

Closed

4 tasks
MichalStrehovsky
MichalStrehovsky
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/System/Globalization/GlobalizationMode.LoadICU.iOS.cs
Outdated

Comment threadsrc/coreclr/pal/src/thread/context.cpp
Outdated

Comment threadeng/native/build-commons.sh
Outdated

Comment threadsrc/coreclr/pal/src/thread/process.cpp
Outdated

marek-safar
marek-safar
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/mono/Directory.Build.props
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/map/virtual.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
@filipnavara
filipnavara
commented
Feb 8, 2023•

edited

Loading

Copy link


Copy Markdown

Member
W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.
@ivanpovazan
ivanpovazan
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.
Good point! My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates. Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the top-level tracking issue for NativeAOT on iOS work (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.
@filipnavara
filipnavara
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates.
Yes. It's used for delegates that have closure over some variables and need an associated context to reference the closure object, and the delegate is passed to native code. There's some test for it in src/tests/nativeaot/SmokeTests/PInvoke IIRC.
Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the #80905 (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.
I think it may be worth creating an issue so that the knowledge about FEATURE_FIXED_POOL_THUNKS is not lost in unrelated PRs/issues. I mainly mentioned it because this PR touched code paths in FEATURE_RX_THUNKS that would not really be usable long term...
@filipnavara
filipnavara
commented
Feb 9, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Turns out there are actually three different code paths for the thunks:
  • FEATURE_RX_THUNKS where thunk pages are generated at runtime (cannot work on iOS)
  • FEATURE_FIXED_POOL_THUNKS where the thunks are pre-generated but limited to certain number at compile time (used by some unofficial console ports)
  • If neither of the feature flags is set, then it runs in the mode with a pre-generated thunk page that is mapped multiple times
Currently there's no Unix code for anything but the FEATURE_RX_THUNKS mode. I wrote a small experimental app that mmaped its own executable page and run a code from it. It works on macOS/arm64 without any of the MAP_JIT flags, so it's likely work on iOS too. That makes the last mode look like the most viable one.
UPD: I updated my prototype to generate the static thunks, on macOS it passes the PInvoke test, so that's a good start.
janvorli
janvorli
reviewed
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/coreclr/pal/src/thread/process.cpp
Outdated

Comment threadsrc/coreclr/inc/executableallocator.h
Outdated

@ivanpovazanivanpovazan
mentioned this pull request
Feb 14, 2023

Closed

akoeplinger
akoeplinger
commented
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/dlls/mscoree/coreclr/CMakeLists.txt
endif(FEATURE_MERGE_JIT_AND_ENGINE)

if (CLR_CMAKE_TARGET_OSX OR CLR_CMAKE_TARGET_MACCATALYST)
if (CLR_CMAKE_TARGET_OSX)
@akoeplingerakoeplingerFeb 14, 2023•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was erroneously added in 45887d1, MacCatalyst isn't building this piece.
janvorli
janvorli
reviewed
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/gc.cpp
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/native/corehost/apphost/standalone/CMakeLists.txt
Outdated

@filipnavarafilipnavara
mentioned this pull request
Feb 14, 2023
 Merged

kotlarmilos
kotlarmilos
reviewed
Feb 15, 2023
View reviewed changes
Comment threadeng/native/configurecompiler.cmake
jkotas
jkotas
reviewed
Feb 15, 2023
View reviewed changes
Comment threadsrc/native/corehost/apphost/static/CMakeLists.txt
janvorli
janvorli
reviewed
Feb 15, 2023
View reviewed changes
Comment threadsrc/coreclr/CMakeLists.txt
Outdated

janvorli
janvorli
approved these changes
Feb 15, 2023
View reviewed changes
@janvorlijanvorli
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!
@build-analysisbuild-analysisBot
mentioned this pull request
Feb 15, 2023

Closed

@akoeplinger
akoeplinger
merged commit e56db76
into
dotnet:mainFeb 16, 2023
@akoeplinger
akoeplinger
deleted the

nativeaot-mobile

branch
February 16, 2023 01:00
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 18, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@marek-safarmarek-safarmarek-safar left review comments
@jkotasjkotasjkotas left review comments
@kotlarmiloskotlarmiloskotlarmilos left review comments
@ivanpovazanivanpovazanivanpovazan left review comments
@janvorlijanvorlijanvorli approved these changes
@MichalStrehovskyMichalStrehovskyAwaiting requested review from MichalStrehovskyMichalStrehovsky is a code owner

+1 more reviewer

@filipnavarafilipnavarafilipnavara left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@akoeplingerakoeplinger

Labels

area-NativeAOT-coreclr

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

8 participants

@akoeplinger@filipnavara@ivanpovazan@marek-safar@jkotas@janvorli@kotlarmilos@MichalStrehovsky

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Enable building NativeAOT for Apple mobile platforms - #81780
Merged
akoeplinger merged 16 commits into
dotnet:mainfrom 
akoeplinger:nativeaot-mobile
Feb 16, 2023
Merged
Enable building NativeAOT for Apple mobile platforms#81780
akoeplinger merged 16 commits into
dotnet:mainfrom 
akoeplinger:nativeaot-mobile
Conversation
@akoeplinger
@akoeplingerakoeplinger
commented
Feb 7, 2023

Copy link


Copy Markdown

Member
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.
/cc @dotnet/ilc-contrib @ivanpovazan
@akoeplinger
Enable building NativeAOT for Apple mobile platforms
268e281
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to dotnet#81024
@ghostghost
assigned akoeplingerFeb 7, 2023
@ghost
ghost
commented
Feb 7, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas

See info in area-owners.md if you want to be subscribed.
Issue Details
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.
/cc @dotnet/ilc-contrib @ivanpovazan
Author:akoeplinger
Assignees:akoeplinger
Labels:
area-NativeAOT-coreclr
Milestone:-
akoeplinger
akoeplinger
commented
Feb 7, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/thread/context.cpp
Outdated

@akoeplingerakoeplinger
mentioned this pull request
Feb 7, 2023

Closed

4 tasks
@akoeplingerakoeplinger
linked an issue
Feb 7, 2023
that may be
closed
by this pull request


Enable building NativeAOT for Apple mobile platforms
#81024

Closed

4 tasks
@akoeplingerakoeplinger
removed a link to an issue
Feb 7, 2023

Enable building NativeAOT for Apple mobile platforms
#81024

Closed

4 tasks
MichalStrehovsky
MichalStrehovsky
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/System/Globalization/GlobalizationMode.LoadICU.iOS.cs
Outdated

Comment threadsrc/coreclr/pal/src/thread/context.cpp
Outdated

Comment threadeng/native/build-commons.sh
Outdated

Comment threadsrc/coreclr/pal/src/thread/process.cpp
Outdated

marek-safar
marek-safar
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/mono/Directory.Build.props
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/map/virtual.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
@filipnavara
filipnavara
commented
Feb 8, 2023•

edited

Loading

Copy link


Copy Markdown

Member
W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.
@ivanpovazan
ivanpovazan
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.
Good point! My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates. Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the top-level tracking issue for NativeAOT on iOS work (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.
@filipnavara
filipnavara
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates.
Yes. It's used for delegates that have closure over some variables and need an associated context to reference the closure object, and the delegate is passed to native code. There's some test for it in src/tests/nativeaot/SmokeTests/PInvoke IIRC.
Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the #80905 (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.
I think it may be worth creating an issue so that the knowledge about FEATURE_FIXED_POOL_THUNKS is not lost in unrelated PRs/issues. I mainly mentioned it because this PR touched code paths in FEATURE_RX_THUNKS that would not really be usable long term...
@filipnavara
filipnavara
commented
Feb 9, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Turns out there are actually three different code paths for the thunks:
  • FEATURE_RX_THUNKS where thunk pages are generated at runtime (cannot work on iOS)
  • FEATURE_FIXED_POOL_THUNKS where the thunks are pre-generated but limited to certain number at compile time (used by some unofficial console ports)
  • If neither of the feature flags is set, then it runs in the mode with a pre-generated thunk page that is mapped multiple times
Currently there's no Unix code for anything but the FEATURE_RX_THUNKS mode. I wrote a small experimental app that mmaped its own executable page and run a code from it. It works on macOS/arm64 without any of the MAP_JIT flags, so it's likely work on iOS too. That makes the last mode look like the most viable one.
UPD: I updated my prototype to generate the static thunks, on macOS it passes the PInvoke test, so that's a good start.
janvorli
janvorli
reviewed
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/coreclr/pal/src/thread/process.cpp
Outdated

Comment threadsrc/coreclr/inc/executableallocator.h
Outdated

@ivanpovazanivanpovazan
mentioned this pull request
Feb 14, 2023

Closed

akoeplinger
akoeplinger
commented
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/dlls/mscoree/coreclr/CMakeLists.txt
endif(FEATURE_MERGE_JIT_AND_ENGINE)

if (CLR_CMAKE_TARGET_OSX OR CLR_CMAKE_TARGET_MACCATALYST)
if (CLR_CMAKE_TARGET_OSX)
@akoeplingerakoeplingerFeb 14, 2023•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was erroneously added in 45887d1, MacCatalyst isn't building this piece.
janvorli
janvorli
reviewed
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/gc.cpp
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/native/corehost/apphost/standalone/CMakeLists.txt
Outdated

@filipnavarafilipnavara
mentioned this pull request
Feb 14, 2023
 Merged

kotlarmilos
kotlarmilos
reviewed
Feb 15, 2023
View reviewed changes
Comment threadeng/native/configurecompiler.cmake
jkotas
jkotas
reviewed
Feb 15, 2023
View reviewed changes
Comment threadsrc/native/corehost/apphost/static/CMakeLists.txt
janvorli
janvorli
reviewed
Feb 15, 2023
View reviewed changes
Comment threadsrc/coreclr/CMakeLists.txt
Outdated

janvorli
janvorli
approved these changes
Feb 15, 2023
View reviewed changes
@janvorlijanvorli
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!
@build-analysisbuild-analysisBot
mentioned this pull request
Feb 15, 2023

Closed

@akoeplinger
akoeplinger
merged commit e56db76
into
dotnet:mainFeb 16, 2023
@akoeplinger
akoeplinger
deleted the

nativeaot-mobile

branch
February 16, 2023 01:00
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 18, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@marek-safarmarek-safarmarek-safar left review comments
@jkotasjkotasjkotas left review comments
@kotlarmiloskotlarmiloskotlarmilos left review comments
@ivanpovazanivanpovazanivanpovazan left review comments
@janvorlijanvorlijanvorli approved these changes
@MichalStrehovskyMichalStrehovskyAwaiting requested review from MichalStrehovskyMichalStrehovsky is a code owner

+1 more reviewer

@filipnavarafilipnavarafilipnavara left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@akoeplingerakoeplinger

Labels

area-NativeAOT-coreclr

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

8 participants

@akoeplinger@filipnavara@ivanpovazan@marek-safar@jkotas@janvorli@kotlarmilos@MichalStrehovsky

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Enable building NativeAOT for Apple mobile platforms - #81780
Merged
akoeplinger merged 16 commits into
dotnet:mainfrom 
akoeplinger:nativeaot-mobile
Feb 16, 2023
Merged
Enable building NativeAOT for Apple mobile platforms#81780
akoeplinger merged 16 commits into
dotnet:mainfrom 
akoeplinger:nativeaot-mobile
Conversation
@akoeplinger
@akoeplingerakoeplinger
commented
Feb 7, 2023

Copy link


Copy Markdown

Member
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.
/cc @dotnet/ilc-contrib @ivanpovazan
@akoeplinger
Enable building NativeAOT for Apple mobile platforms
268e281
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to dotnet#81024
@ghostghost
assigned akoeplingerFeb 7, 2023
@ghost
ghost
commented
Feb 7, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas

See info in area-owners.md if you want to be subscribed.
Issue Details
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.
/cc @dotnet/ilc-contrib @ivanpovazan
Author:akoeplinger
Assignees:akoeplinger
Labels:
area-NativeAOT-coreclr
Milestone:-
akoeplinger
akoeplinger
commented
Feb 7, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/thread/context.cpp
Outdated

@akoeplingerakoeplinger
mentioned this pull request
Feb 7, 2023

Closed

4 tasks
@akoeplingerakoeplinger
linked an issue
Feb 7, 2023
that may be
closed
by this pull request


Enable building NativeAOT for Apple mobile platforms
#81024

Closed

4 tasks
@akoeplingerakoeplinger
removed a link to an issue
Feb 7, 2023

Enable building NativeAOT for Apple mobile platforms
#81024

Closed

4 tasks
MichalStrehovsky
MichalStrehovsky
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/System/Globalization/GlobalizationMode.LoadICU.iOS.cs
Outdated

Comment threadsrc/coreclr/pal/src/thread/context.cpp
Outdated

Comment threadeng/native/build-commons.sh
Outdated

Comment threadsrc/coreclr/pal/src/thread/process.cpp
Outdated

marek-safar
marek-safar
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/mono/Directory.Build.props
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/map/virtual.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
@filipnavara
filipnavara
commented
Feb 8, 2023•

edited

Loading

Copy link


Copy Markdown

Member
W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.
@ivanpovazan
ivanpovazan
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.
Good point! My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates. Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the top-level tracking issue for NativeAOT on iOS work (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.
@filipnavara
filipnavara
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates.
Yes. It's used for delegates that have closure over some variables and need an associated context to reference the closure object, and the delegate is passed to native code. There's some test for it in src/tests/nativeaot/SmokeTests/PInvoke IIRC.
Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the #80905 (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.
I think it may be worth creating an issue so that the knowledge about FEATURE_FIXED_POOL_THUNKS is not lost in unrelated PRs/issues. I mainly mentioned it because this PR touched code paths in FEATURE_RX_THUNKS that would not really be usable long term...
@filipnavara
filipnavara
commented
Feb 9, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Turns out there are actually three different code paths for the thunks:
  • FEATURE_RX_THUNKS where thunk pages are generated at runtime (cannot work on iOS)
  • FEATURE_FIXED_POOL_THUNKS where the thunks are pre-generated but limited to certain number at compile time (used by some unofficial console ports)
  • If neither of the feature flags is set, then it runs in the mode with a pre-generated thunk page that is mapped multiple times
Currently there's no Unix code for anything but the FEATURE_RX_THUNKS mode. I wrote a small experimental app that mmaped its own executable page and run a code from it. It works on macOS/arm64 without any of the MAP_JIT flags, so it's likely work on iOS too. That makes the last mode look like the most viable one.
UPD: I updated my prototype to generate the static thunks, on macOS it passes the PInvoke test, so that's a good start.
janvorli
janvorli
reviewed
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/coreclr/pal/src/thread/process.cpp
Outdated

Comment threadsrc/coreclr/inc/executableallocator.h
Outdated

@ivanpovazanivanpovazan
mentioned this pull request
Feb 14, 2023

Closed

akoeplinger
akoeplinger
commented
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/dlls/mscoree/coreclr/CMakeLists.txt
endif(FEATURE_MERGE_JIT_AND_ENGINE)

if (CLR_CMAKE_TARGET_OSX OR CLR_CMAKE_TARGET_MACCATALYST)
if (CLR_CMAKE_TARGET_OSX)
@akoeplingerakoeplingerFeb 14, 2023•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was erroneously added in 45887d1, MacCatalyst isn't building this piece.
janvorli
janvorli
reviewed
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/gc.cpp
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/native/corehost/apphost/standalone/CMakeLists.txt
Outdated

@filipnavarafilipnavara
mentioned this pull request
Feb 14, 2023
 Merged

kotlarmilos
kotlarmilos
reviewed
Feb 15, 2023
View reviewed changes
Comment threadeng/native/configurecompiler.cmake
jkotas
jkotas
reviewed
Feb 15, 2023
View reviewed changes
Comment threadsrc/native/corehost/apphost/static/CMakeLists.txt
janvorli
janvorli
reviewed
Feb 15, 2023
View reviewed changes
Comment threadsrc/coreclr/CMakeLists.txt
Outdated

janvorli
janvorli
approved these changes
Feb 15, 2023
View reviewed changes
@janvorlijanvorli
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!
@build-analysisbuild-analysisBot
mentioned this pull request
Feb 15, 2023

Closed

@akoeplinger
akoeplinger
merged commit e56db76
into
dotnet:mainFeb 16, 2023
@akoeplinger
akoeplinger
deleted the

nativeaot-mobile

branch
February 16, 2023 01:00
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 18, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@marek-safarmarek-safarmarek-safar left review comments
@jkotasjkotasjkotas left review comments
@kotlarmiloskotlarmiloskotlarmilos left review comments
@ivanpovazanivanpovazanivanpovazan left review comments
@janvorlijanvorlijanvorli approved these changes
@MichalStrehovskyMichalStrehovskyAwaiting requested review from MichalStrehovskyMichalStrehovsky is a code owner

+1 more reviewer

@filipnavarafilipnavarafilipnavara left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@akoeplingerakoeplinger

Labels

area-NativeAOT-coreclr

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

8 participants

@akoeplinger@filipnavara@ivanpovazan@marek-safar@jkotas@janvorli@kotlarmilos@MichalStrehovsky

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Enable building NativeAOT for Apple mobile platforms - #81780
Merged
akoeplinger merged 16 commits into
dotnet:mainfrom 
akoeplinger:nativeaot-mobile
Feb 16, 2023
Merged
Enable building NativeAOT for Apple mobile platforms#81780
akoeplinger merged 16 commits into
dotnet:mainfrom 
akoeplinger:nativeaot-mobile
Conversation
@akoeplinger
@akoeplingerakoeplinger
commented
Feb 7, 2023

Copy link


Copy Markdown

Member
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.
/cc @dotnet/ilc-contrib @ivanpovazan
@akoeplinger
Enable building NativeAOT for Apple mobile platforms
268e281
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to dotnet#81024
@ghostghost
assigned akoeplingerFeb 7, 2023
@ghost
ghost
commented
Feb 7, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas

See info in area-owners.md if you want to be subscribed.
Issue Details
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.
/cc @dotnet/ilc-contrib @ivanpovazan
Author:akoeplinger
Assignees:akoeplinger
Labels:
area-NativeAOT-coreclr
Milestone:-
akoeplinger
akoeplinger
commented
Feb 7, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/thread/context.cpp
Outdated

@akoeplingerakoeplinger
mentioned this pull request
Feb 7, 2023

Closed

4 tasks
@akoeplingerakoeplinger
linked an issue
Feb 7, 2023
that may be
closed
by this pull request


Enable building NativeAOT for Apple mobile platforms
#81024

Closed

4 tasks
@akoeplingerakoeplinger
removed a link to an issue
Feb 7, 2023

Enable building NativeAOT for Apple mobile platforms
#81024

Closed

4 tasks
MichalStrehovsky
MichalStrehovsky
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/System/Globalization/GlobalizationMode.LoadICU.iOS.cs
Outdated

Comment threadsrc/coreclr/pal/src/thread/context.cpp
Outdated

Comment threadeng/native/build-commons.sh
Outdated

Comment threadsrc/coreclr/pal/src/thread/process.cpp
Outdated

marek-safar
marek-safar
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/mono/Directory.Build.props
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/map/virtual.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
@filipnavara
filipnavara
commented
Feb 8, 2023•

edited

Loading

Copy link


Copy Markdown

Member
W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.
@ivanpovazan
ivanpovazan
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.
Good point! My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates. Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the top-level tracking issue for NativeAOT on iOS work (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.
@filipnavara
filipnavara
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates.
Yes. It's used for delegates that have closure over some variables and need an associated context to reference the closure object, and the delegate is passed to native code. There's some test for it in src/tests/nativeaot/SmokeTests/PInvoke IIRC.
Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the #80905 (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.
I think it may be worth creating an issue so that the knowledge about FEATURE_FIXED_POOL_THUNKS is not lost in unrelated PRs/issues. I mainly mentioned it because this PR touched code paths in FEATURE_RX_THUNKS that would not really be usable long term...
@filipnavara
filipnavara
commented
Feb 9, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Turns out there are actually three different code paths for the thunks:
  • FEATURE_RX_THUNKS where thunk pages are generated at runtime (cannot work on iOS)
  • FEATURE_FIXED_POOL_THUNKS where the thunks are pre-generated but limited to certain number at compile time (used by some unofficial console ports)
  • If neither of the feature flags is set, then it runs in the mode with a pre-generated thunk page that is mapped multiple times
Currently there's no Unix code for anything but the FEATURE_RX_THUNKS mode. I wrote a small experimental app that mmaped its own executable page and run a code from it. It works on macOS/arm64 without any of the MAP_JIT flags, so it's likely work on iOS too. That makes the last mode look like the most viable one.
UPD: I updated my prototype to generate the static thunks, on macOS it passes the PInvoke test, so that's a good start.
janvorli
janvorli
reviewed
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/coreclr/pal/src/thread/process.cpp
Outdated

Comment threadsrc/coreclr/inc/executableallocator.h
Outdated

@ivanpovazanivanpovazan
mentioned this pull request
Feb 14, 2023

Closed

akoeplinger
akoeplinger
commented
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/dlls/mscoree/coreclr/CMakeLists.txt
endif(FEATURE_MERGE_JIT_AND_ENGINE)

if (CLR_CMAKE_TARGET_OSX OR CLR_CMAKE_TARGET_MACCATALYST)
if (CLR_CMAKE_TARGET_OSX)
@akoeplingerakoeplingerFeb 14, 2023•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was erroneously added in 45887d1, MacCatalyst isn't building this piece.
janvorli
janvorli
reviewed
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/gc.cpp
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/native/corehost/apphost/standalone/CMakeLists.txt
Outdated

@filipnavarafilipnavara
mentioned this pull request
Feb 14, 2023
 Merged

kotlarmilos
kotlarmilos
reviewed
Feb 15, 2023
View reviewed changes
Comment threadeng/native/configurecompiler.cmake
jkotas
jkotas
reviewed
Feb 15, 2023
View reviewed changes
Comment threadsrc/native/corehost/apphost/static/CMakeLists.txt
janvorli
janvorli
reviewed
Feb 15, 2023
View reviewed changes
Comment threadsrc/coreclr/CMakeLists.txt
Outdated

janvorli
janvorli
approved these changes
Feb 15, 2023
View reviewed changes
@janvorlijanvorli
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!
@build-analysisbuild-analysisBot
mentioned this pull request
Feb 15, 2023

Closed

@akoeplinger
akoeplinger
merged commit e56db76
into
dotnet:mainFeb 16, 2023
@akoeplinger
akoeplinger
deleted the

nativeaot-mobile

branch
February 16, 2023 01:00
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 18, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@marek-safarmarek-safarmarek-safar left review comments
@jkotasjkotasjkotas left review comments
@kotlarmiloskotlarmiloskotlarmilos left review comments
@ivanpovazanivanpovazanivanpovazan left review comments
@janvorlijanvorlijanvorli approved these changes
@MichalStrehovskyMichalStrehovskyAwaiting requested review from MichalStrehovskyMichalStrehovsky is a code owner

+1 more reviewer

@filipnavarafilipnavarafilipnavara left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@akoeplingerakoeplinger

Labels

area-NativeAOT-coreclr

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

8 participants

@akoeplinger@filipnavara@ivanpovazan@marek-safar@jkotas@janvorli@kotlarmilos@MichalStrehovsky

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Enable building NativeAOT for Apple mobile platforms - #81780
Merged
akoeplinger merged 16 commits into
dotnet:mainfrom 
akoeplinger:nativeaot-mobile
Feb 16, 2023
Merged
Enable building NativeAOT for Apple mobile platforms#81780
akoeplinger merged 16 commits into
dotnet:mainfrom 
akoeplinger:nativeaot-mobile
Conversation
@akoeplinger
@akoeplingerakoeplinger
commented
Feb 7, 2023

Copy link


Copy Markdown

Member
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.
/cc @dotnet/ilc-contrib @ivanpovazan
@akoeplinger
Enable building NativeAOT for Apple mobile platforms
268e281
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to dotnet#81024
@ghostghost
assigned akoeplingerFeb 7, 2023
@ghost
ghost
commented
Feb 7, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas

See info in area-owners.md if you want to be subscribed.
Issue Details
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.
/cc @dotnet/ilc-contrib @ivanpovazan
Author:akoeplinger
Assignees:akoeplinger
Labels:
area-NativeAOT-coreclr
Milestone:-
akoeplinger
akoeplinger
commented
Feb 7, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/thread/context.cpp
Outdated

@akoeplingerakoeplinger
mentioned this pull request
Feb 7, 2023

Closed

4 tasks
@akoeplingerakoeplinger
linked an issue
Feb 7, 2023
that may be
closed
by this pull request


Enable building NativeAOT for Apple mobile platforms
#81024

Closed

4 tasks
@akoeplingerakoeplinger
removed a link to an issue
Feb 7, 2023

Enable building NativeAOT for Apple mobile platforms
#81024

Closed

4 tasks
MichalStrehovsky
MichalStrehovsky
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/System/Globalization/GlobalizationMode.LoadICU.iOS.cs
Outdated

Comment threadsrc/coreclr/pal/src/thread/context.cpp
Outdated

Comment threadeng/native/build-commons.sh
Outdated

Comment threadsrc/coreclr/pal/src/thread/process.cpp
Outdated

marek-safar
marek-safar
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/mono/Directory.Build.props
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/map/virtual.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
@filipnavara
filipnavara
commented
Feb 8, 2023•

edited

Loading

Copy link


Copy Markdown

Member
W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.
@ivanpovazan
ivanpovazan
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.
Good point! My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates. Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the top-level tracking issue for NativeAOT on iOS work (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.
@filipnavara
filipnavara
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates.
Yes. It's used for delegates that have closure over some variables and need an associated context to reference the closure object, and the delegate is passed to native code. There's some test for it in src/tests/nativeaot/SmokeTests/PInvoke IIRC.
Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the #80905 (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.
I think it may be worth creating an issue so that the knowledge about FEATURE_FIXED_POOL_THUNKS is not lost in unrelated PRs/issues. I mainly mentioned it because this PR touched code paths in FEATURE_RX_THUNKS that would not really be usable long term...
@filipnavara
filipnavara
commented
Feb 9, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Turns out there are actually three different code paths for the thunks:
  • FEATURE_RX_THUNKS where thunk pages are generated at runtime (cannot work on iOS)
  • FEATURE_FIXED_POOL_THUNKS where the thunks are pre-generated but limited to certain number at compile time (used by some unofficial console ports)
  • If neither of the feature flags is set, then it runs in the mode with a pre-generated thunk page that is mapped multiple times
Currently there's no Unix code for anything but the FEATURE_RX_THUNKS mode. I wrote a small experimental app that mmaped its own executable page and run a code from it. It works on macOS/arm64 without any of the MAP_JIT flags, so it's likely work on iOS too. That makes the last mode look like the most viable one.
UPD: I updated my prototype to generate the static thunks, on macOS it passes the PInvoke test, so that's a good start.
janvorli
janvorli
reviewed
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/coreclr/pal/src/thread/process.cpp
Outdated

Comment threadsrc/coreclr/inc/executableallocator.h
Outdated

@ivanpovazanivanpovazan
mentioned this pull request
Feb 14, 2023

Closed

akoeplinger
akoeplinger
commented
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/dlls/mscoree/coreclr/CMakeLists.txt
endif(FEATURE_MERGE_JIT_AND_ENGINE)

if (CLR_CMAKE_TARGET_OSX OR CLR_CMAKE_TARGET_MACCATALYST)
if (CLR_CMAKE_TARGET_OSX)
@akoeplingerakoeplingerFeb 14, 2023•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was erroneously added in 45887d1, MacCatalyst isn't building this piece.
janvorli
janvorli
reviewed
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/gc.cpp
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/native/corehost/apphost/standalone/CMakeLists.txt
Outdated

@filipnavarafilipnavara
mentioned this pull request
Feb 14, 2023
 Merged

kotlarmilos
kotlarmilos
reviewed
Feb 15, 2023
View reviewed changes
Comment threadeng/native/configurecompiler.cmake
jkotas
jkotas
reviewed
Feb 15, 2023
View reviewed changes
Comment threadsrc/native/corehost/apphost/static/CMakeLists.txt
janvorli
janvorli
reviewed
Feb 15, 2023
View reviewed changes
Comment threadsrc/coreclr/CMakeLists.txt
Outdated

janvorli
janvorli
approved these changes
Feb 15, 2023
View reviewed changes
@janvorlijanvorli
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!
@build-analysisbuild-analysisBot
mentioned this pull request
Feb 15, 2023

Closed

@akoeplinger
akoeplinger
merged commit e56db76
into
dotnet:mainFeb 16, 2023
@akoeplinger
akoeplinger
deleted the

nativeaot-mobile

branch
February 16, 2023 01:00
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 18, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@marek-safarmarek-safarmarek-safar left review comments
@jkotasjkotasjkotas left review comments
@kotlarmiloskotlarmiloskotlarmilos left review comments
@ivanpovazanivanpovazanivanpovazan left review comments
@janvorlijanvorlijanvorli approved these changes
@MichalStrehovskyMichalStrehovskyAwaiting requested review from MichalStrehovskyMichalStrehovsky is a code owner

+1 more reviewer

@filipnavarafilipnavarafilipnavara left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@akoeplingerakoeplinger

Labels

area-NativeAOT-coreclr

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

8 participants

@akoeplinger@filipnavara@ivanpovazan@marek-safar@jkotas@janvorli@kotlarmilos@MichalStrehovsky

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Enable building NativeAOT for Apple mobile platforms - #81780
Merged
akoeplinger merged 16 commits into
dotnet:mainfrom 
akoeplinger:nativeaot-mobile
Feb 16, 2023
Merged
Enable building NativeAOT for Apple mobile platforms#81780
akoeplinger merged 16 commits into
dotnet:mainfrom 
akoeplinger:nativeaot-mobile
Conversation
@akoeplinger
@akoeplingerakoeplinger
commented
Feb 7, 2023

Copy link


Copy Markdown

Member
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.
/cc @dotnet/ilc-contrib @ivanpovazan
@akoeplinger
Enable building NativeAOT for Apple mobile platforms
268e281
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to dotnet#81024
@ghostghost
assigned akoeplingerFeb 7, 2023
@ghost
ghost
commented
Feb 7, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas

See info in area-owners.md if you want to be subscribed.
Issue Details
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.
/cc @dotnet/ilc-contrib @ivanpovazan
Author:akoeplinger
Assignees:akoeplinger
Labels:
area-NativeAOT-coreclr
Milestone:-
akoeplinger
akoeplinger
commented
Feb 7, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/thread/context.cpp
Outdated

@akoeplingerakoeplinger
mentioned this pull request
Feb 7, 2023

Closed

4 tasks
@akoeplingerakoeplinger
linked an issue
Feb 7, 2023
that may be
closed
by this pull request


Enable building NativeAOT for Apple mobile platforms
#81024

Closed

4 tasks
@akoeplingerakoeplinger
removed a link to an issue
Feb 7, 2023

Enable building NativeAOT for Apple mobile platforms
#81024

Closed

4 tasks
MichalStrehovsky
MichalStrehovsky
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/System/Globalization/GlobalizationMode.LoadICU.iOS.cs
Outdated

Comment threadsrc/coreclr/pal/src/thread/context.cpp
Outdated

Comment threadeng/native/build-commons.sh
Outdated

Comment threadsrc/coreclr/pal/src/thread/process.cpp
Outdated

marek-safar
marek-safar
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/mono/Directory.Build.props
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/map/virtual.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
@filipnavara
filipnavara
commented
Feb 8, 2023•

edited

Loading

Copy link


Copy Markdown

Member
W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.
@ivanpovazan
ivanpovazan
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.
Good point! My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates. Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the top-level tracking issue for NativeAOT on iOS work (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.
@filipnavara
filipnavara
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates.
Yes. It's used for delegates that have closure over some variables and need an associated context to reference the closure object, and the delegate is passed to native code. There's some test for it in src/tests/nativeaot/SmokeTests/PInvoke IIRC.
Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the #80905 (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.
I think it may be worth creating an issue so that the knowledge about FEATURE_FIXED_POOL_THUNKS is not lost in unrelated PRs/issues. I mainly mentioned it because this PR touched code paths in FEATURE_RX_THUNKS that would not really be usable long term...
@filipnavara
filipnavara
commented
Feb 9, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Turns out there are actually three different code paths for the thunks:
  • FEATURE_RX_THUNKS where thunk pages are generated at runtime (cannot work on iOS)
  • FEATURE_FIXED_POOL_THUNKS where the thunks are pre-generated but limited to certain number at compile time (used by some unofficial console ports)
  • If neither of the feature flags is set, then it runs in the mode with a pre-generated thunk page that is mapped multiple times
Currently there's no Unix code for anything but the FEATURE_RX_THUNKS mode. I wrote a small experimental app that mmaped its own executable page and run a code from it. It works on macOS/arm64 without any of the MAP_JIT flags, so it's likely work on iOS too. That makes the last mode look like the most viable one.
UPD: I updated my prototype to generate the static thunks, on macOS it passes the PInvoke test, so that's a good start.
janvorli
janvorli
reviewed
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/coreclr/pal/src/thread/process.cpp
Outdated

Comment threadsrc/coreclr/inc/executableallocator.h
Outdated

@ivanpovazanivanpovazan
mentioned this pull request
Feb 14, 2023

Closed

akoeplinger
akoeplinger
commented
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/dlls/mscoree/coreclr/CMakeLists.txt
endif(FEATURE_MERGE_JIT_AND_ENGINE)

if (CLR_CMAKE_TARGET_OSX OR CLR_CMAKE_TARGET_MACCATALYST)
if (CLR_CMAKE_TARGET_OSX)
@akoeplingerakoeplingerFeb 14, 2023•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was erroneously added in 45887d1, MacCatalyst isn't building this piece.
janvorli
janvorli
reviewed
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/gc.cpp
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/native/corehost/apphost/standalone/CMakeLists.txt
Outdated

@filipnavarafilipnavara
mentioned this pull request
Feb 14, 2023
 Merged

kotlarmilos
kotlarmilos
reviewed
Feb 15, 2023
View reviewed changes
Comment threadeng/native/configurecompiler.cmake
jkotas
jkotas
reviewed
Feb 15, 2023
View reviewed changes
Comment threadsrc/native/corehost/apphost/static/CMakeLists.txt
janvorli
janvorli
reviewed
Feb 15, 2023
View reviewed changes
Comment threadsrc/coreclr/CMakeLists.txt
Outdated

janvorli
janvorli
approved these changes
Feb 15, 2023
View reviewed changes
@janvorlijanvorli
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!
@build-analysisbuild-analysisBot
mentioned this pull request
Feb 15, 2023

Closed

@akoeplinger
akoeplinger
merged commit e56db76
into
dotnet:mainFeb 16, 2023
@akoeplinger
akoeplinger
deleted the

nativeaot-mobile

branch
February 16, 2023 01:00
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 18, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@marek-safarmarek-safarmarek-safar left review comments
@jkotasjkotasjkotas left review comments
@kotlarmiloskotlarmiloskotlarmilos left review comments
@ivanpovazanivanpovazanivanpovazan left review comments
@janvorlijanvorlijanvorli approved these changes
@MichalStrehovskyMichalStrehovskyAwaiting requested review from MichalStrehovskyMichalStrehovsky is a code owner

+1 more reviewer

@filipnavarafilipnavarafilipnavara left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@akoeplingerakoeplinger

Labels

area-NativeAOT-coreclr

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

8 participants

@akoeplinger@filipnavara@ivanpovazan@marek-safar@jkotas@janvorli@kotlarmilos@MichalStrehovsky

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Enable building NativeAOT for Apple mobile platforms - #81780
Merged
akoeplinger merged 16 commits into
dotnet:mainfrom 
akoeplinger:nativeaot-mobile
Feb 16, 2023
Merged
Enable building NativeAOT for Apple mobile platforms#81780
akoeplinger merged 16 commits into
dotnet:mainfrom 
akoeplinger:nativeaot-mobile
Conversation
@akoeplinger
@akoeplingerakoeplinger
commented
Feb 7, 2023

Copy link


Copy Markdown

Member
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.
/cc @dotnet/ilc-contrib @ivanpovazan
@akoeplinger
Enable building NativeAOT for Apple mobile platforms
268e281
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to dotnet#81024
@ghostghost
assigned akoeplingerFeb 7, 2023
@ghost
ghost
commented
Feb 7, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas

See info in area-owners.md if you want to be subscribed.
Issue Details
These changes allow the NativeAOT runtime to compile for and run on iOS, tvOS and MacCatalyst.
Contributes to #81024, to actually compile an app we still have a few more changes that need to be upstreamed.
/cc @dotnet/ilc-contrib @ivanpovazan
Author:akoeplinger
Assignees:akoeplinger
Labels:
area-NativeAOT-coreclr
Milestone:-
akoeplinger
akoeplinger
commented
Feb 7, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/thread/context.cpp
Outdated

@akoeplingerakoeplinger
mentioned this pull request
Feb 7, 2023

Closed

4 tasks
@akoeplingerakoeplinger
linked an issue
Feb 7, 2023
that may be
closed
by this pull request


Enable building NativeAOT for Apple mobile platforms
#81024

Closed

4 tasks
@akoeplingerakoeplinger
removed a link to an issue
Feb 7, 2023

Enable building NativeAOT for Apple mobile platforms
#81024

Closed

4 tasks
MichalStrehovsky
MichalStrehovsky
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/System/Globalization/GlobalizationMode.LoadICU.iOS.cs
Outdated

Comment threadsrc/coreclr/pal/src/thread/context.cpp
Outdated

Comment threadeng/native/build-commons.sh
Outdated

Comment threadsrc/coreclr/pal/src/thread/process.cpp
Outdated

marek-safar
marek-safar
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/mono/Directory.Build.props
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/pal/src/map/virtual.cpp
Outdated

ivanpovazan
ivanpovazan
reviewed
Feb 8, 2023
View reviewed changes
Comment threadsrc/coreclr/nativeaot/Runtime/ThunksMapping.cpp
@filipnavara
filipnavara
commented
Feb 8, 2023•

edited

Loading

Copy link


Copy Markdown

Member
W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.
@ivanpovazan
ivanpovazan
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
W.r.t. thunk mapping: I believe NativeAOT implements two strategies for how to generate these thunks. The current default (FEATURE_RX_THUNKS) generates code at runtime and would not work on iOS. The alternative code path (FEATURE_FIXED_POOL_THUNKS) works by precompiling the thunk code and then mapping it multiple times (IIRC; @jkotas, @MichalStrehovsky feel free to correct me). I don't think the alternative code path is enabled anywhere (including this PR) and/or tested. We should probably open a separate issue to track that.
Good point! My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates. Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the top-level tracking issue for NativeAOT on iOS work (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.
@filipnavara
filipnavara
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
My understanding is that FEATURE_FIXED_POOL_THUNKS is mainly used for supporting marshalled delegates.
Yes. It's used for delegates that have closure over some variables and need an associated context to reference the closure object, and the delegate is passed to native code. There's some test for it in src/tests/nativeaot/SmokeTests/PInvoke IIRC.
Therefore, there is a "dedicated task": Marshalled delegates vs function pointers support with NativeAOT in the #80905 (Priority1 section). However, there is still no separate-dedicated issue for it, as we wanted to get some initial bits first. Nevertheless, if you think this should be escalated, we can open a tracking issue for it asap.
I think it may be worth creating an issue so that the knowledge about FEATURE_FIXED_POOL_THUNKS is not lost in unrelated PRs/issues. I mainly mentioned it because this PR touched code paths in FEATURE_RX_THUNKS that would not really be usable long term...
@filipnavara
filipnavara
commented
Feb 9, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Turns out there are actually three different code paths for the thunks:
  • FEATURE_RX_THUNKS where thunk pages are generated at runtime (cannot work on iOS)
  • FEATURE_FIXED_POOL_THUNKS where the thunks are pre-generated but limited to certain number at compile time (used by some unofficial console ports)
  • If neither of the feature flags is set, then it runs in the mode with a pre-generated thunk page that is mapped multiple times
Currently there's no Unix code for anything but the FEATURE_RX_THUNKS mode. I wrote a small experimental app that mmaped its own executable page and run a code from it. It works on macOS/arm64 without any of the MAP_JIT flags, so it's likely work on iOS too. That makes the last mode look like the most viable one.
UPD: I updated my prototype to generate the static thunks, on macOS it passes the PInvoke test, so that's a good start.
janvorli
janvorli
reviewed
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/coreclr/pal/src/thread/process.cpp
Outdated

Comment threadsrc/coreclr/inc/executableallocator.h
Outdated

@ivanpovazanivanpovazan
mentioned this pull request
Feb 14, 2023

Closed

akoeplinger
akoeplinger
commented
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/dlls/mscoree/coreclr/CMakeLists.txt
endif(FEATURE_MERGE_JIT_AND_ENGINE)

if (CLR_CMAKE_TARGET_OSX OR CLR_CMAKE_TARGET_MACCATALYST)
if (CLR_CMAKE_TARGET_OSX)
@akoeplingerakoeplingerFeb 14, 2023•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was erroneously added in 45887d1, MacCatalyst isn't building this piece.
janvorli
janvorli
reviewed
Feb 14, 2023
View reviewed changes
Comment threadsrc/coreclr/gc/gc.cpp
Comment threadsrc/coreclr/gc/unix/gcenv.unix.cpp
Comment threadsrc/native/corehost/apphost/standalone/CMakeLists.txt
Outdated

@filipnavarafilipnavara
mentioned this pull request
Feb 14, 2023
 Merged

kotlarmilos
kotlarmilos
reviewed
Feb 15, 2023
View reviewed changes
Comment threadeng/native/configurecompiler.cmake
jkotas
jkotas
reviewed
Feb 15, 2023
View reviewed changes
Comment threadsrc/native/corehost/apphost/static/CMakeLists.txt
janvorli
janvorli
reviewed
Feb 15, 2023
View reviewed changes
Comment threadsrc/coreclr/CMakeLists.txt
Outdated

janvorli
janvorli
approved these changes
Feb 15, 2023
View reviewed changes
@janvorlijanvorli
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!
@build-analysisbuild-analysisBot
mentioned this pull request
Feb 15, 2023

Closed

@akoeplinger
akoeplinger
merged commit e56db76
into
dotnet:mainFeb 16, 2023
@akoeplinger
akoeplinger
deleted the

nativeaot-mobile

branch
February 16, 2023 01:00
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 18, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@marek-safarmarek-safarmarek-safar left review comments
@jkotasjkotasjkotas left review comments
@kotlarmiloskotlarmiloskotlarmilos left review comments
@ivanpovazanivanpovazanivanpovazan left review comments
@janvorlijanvorlijanvorli approved these changes
@MichalStrehovskyMichalStrehovskyAwaiting requested review from MichalStrehovskyMichalStrehovsky is a code owner

+1 more reviewer

@filipnavarafilipnavarafilipnavara left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@akoeplingerakoeplinger

Labels

area-NativeAOT-coreclr

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

8 participants

@akoeplinger@filipnavara@ivanpovazan@marek-safar@jkotas@janvorli@kotlarmilos@MichalStrehovsky