Skip to content

Syscall rewriter improvements - #812

Merged
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format
May 13, 2026
Merged

Syscall rewriter improvements#812
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format

Conversation

@wdcui

@wdcuiWeidong Cui (wdcui) commented Apr 25, 2026

Copy link
Copy Markdown
Member

This PR changes how the return address is saved in rcx when syscalls are patched so that it can used to calculate the restart address. It also adds an empty trampoline to ELF files that don't have syscalls so the runtime knows it's already patched.

@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from 75ea7ec to 0bcb6ffCompareApril 25, 2026 04:06
@wdcui
Weidong Cui (wdcui) marked this pull request as ready for review April 25, 2026 18:50
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from a897889 to fd4fcb7CompareApril 25, 2026 21:31
@wdcui

Copy link
Copy Markdown
MemberAuthor

This PR is ready for review. Ignore the rtld changes since it will be removed after PR #810 is merged.

@jaybosamiya-msJay Bosamiya (Microsoft) (jaybosamiya-ms) added the expmt:shadow-kiln Tag to quickly find the different PRs as part of the "shadow kiln" experiment. label Apr 28, 2026
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch 2 times, most recently from d083228 to 097c515CompareMay 7, 2026 03:25
@wdcui

Copy link
Copy Markdown
MemberAuthor

Weiteng Chen (@CvvT) and Sangho Lee (@sangho2) this PR is ready for your review. Thanks!

@CvvT

Weiteng Chen (CvvT) commented May 8, 2026

Copy link
Copy Markdown
Contributor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

@CvvT

Copy link
Copy Markdown
Contributor

I don't see why we need the red zone.

@wdcui

Copy link
Copy Markdown
MemberAuthor

I don't see why we need the red zone.

Good point. It's removed now.

@wdcui

Copy link
Copy Markdown
MemberAuthor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

ctx.rip = ctx.rcx - 6 since we don't need to rerun the lea of rcx (and lea rsp is removed now).

@wdcuiWeidong Cui (wdcui) changed the title Add trampoline preamble with red zone reservation and R11 restart addressSyscall rewriter improvementsMay 12, 2026

@CvvTWeiteng Chen (CvvT) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Weidong Cui (wdcui)and others added 8 commits May 12, 2026 20:10
…ress
The syscall rewriter now emits a 12-byte preamble before each trampoline
stub: LEA RSP,[RSP-0x80] to reserve the SysV 128-byte red zone, and
LEA R11,[RIP+disp32] to load the call-site restart address into R11 for
future SA_RESTART support.
Platform callbacks are renamed to syscall_callback_redzone to reflect
the new calling convention. The callback recovers the architectural RSP
with LEA R11,[RSP+128] and saves the restart address to TLS
(saved_restart_addr) before clobbering R11.
The rewriter also emits a size=0 header sentinel for binaries with no
syscall instructions, allowing the loader to distinguish 'checked, no
syscalls' from 'never processed.' The is_already_hooked check treats
size=0 as already-hooked. The rtld_audit library is updated to handle
both the new calling convention (red zone + R11) and size=0 binaries
(via mincore probe before reading trampoline memory).
The pre-built binary had old-format trampolines that jumped to
syscall_callback_redzone without reserving the red zone, causing
an access violation (0xc0000005) on Windows.
…oline anchor
Per CvvT's review: the R11 restart-address load and the SysV red zone
preamble exist to support each other (the Windows callback's R11 shuttle
was the only thing that wrote to the guest stack pre-stack-switch, and
the red zone reservation existed to protect against that shuttle).
Replace the scheme so every stub satisfies:
pt_regs.rcx - 6 == address of JMP [syscall_entry_slot]
RCX uniformly points at the in-trampoline `post_jmp`, which is now a
5-byte `JMP rel32 -> guest_text` tail in every stub. The callback's
`jmp rcx` return path lands at post_jmp and falls through to guest text.
The SA_RESTART handler can rewind ctx.rip with a single subtract, with
no per-variant layout knowledge.
Drops, both platforms:
- saved_restart_addr TLS slot
- Windows TEB-slot shuttle (mov gs:[0x28], r11; etc.)
- lea r11, [rsp+128] architectural-RSP recovery
- LEA RSP, [RSP-0x80] red zone preamble
- LEA R11, [RIP+disp32] restart-address load
The post_jmp tail adds 5 bytes per stub. Net: -7 bytes per stub vs.
the prior approach, with the SA_RESTART invariant baked in for free.
Snapshot regenerated to reflect the new 18-byte-per-stub layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🤖 SemverChecks 🤖 No breaking API changes detected

Note: this does not mean API is unchanged, or even that there are no breaking changes; simply, none of the detections triggered.

@wdcui
Weidong Cui (wdcui) added this pull request to the merge queueMay 13, 2026
Merged via the queue into main with commit e728793May 13, 2026
13 checks passed
@wdcui
Weidong Cui (wdcui) deleted the wdcui/pr1-trampoline-format branch May 13, 2026 03:27
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

expmt:shadow-kilnTag to quickly find the different PRs as part of the "shadow kiln" experiment.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@wdcui@CvvT@jaybosamiya-ms
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Syscall rewriter improvements by wdcui · Pull Request #812 · microsoft/litebox · GitHub
Skip to content

Syscall rewriter improvements - #812

Merged
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format
May 13, 2026
Merged

Syscall rewriter improvements#812
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format

Conversation

@wdcui

@wdcuiWeidong Cui (wdcui) commented Apr 25, 2026

Copy link
Copy Markdown
Member

This PR changes how the return address is saved in rcx when syscalls are patched so that it can used to calculate the restart address. It also adds an empty trampoline to ELF files that don't have syscalls so the runtime knows it's already patched.

@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from 75ea7ec to 0bcb6ffCompareApril 25, 2026 04:06
@wdcui
Weidong Cui (wdcui) marked this pull request as ready for review April 25, 2026 18:50
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from a897889 to fd4fcb7CompareApril 25, 2026 21:31
@wdcui

Copy link
Copy Markdown
MemberAuthor

This PR is ready for review. Ignore the rtld changes since it will be removed after PR #810 is merged.

@jaybosamiya-msJay Bosamiya (Microsoft) (jaybosamiya-ms) added the expmt:shadow-kiln Tag to quickly find the different PRs as part of the "shadow kiln" experiment. label Apr 28, 2026
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch 2 times, most recently from d083228 to 097c515CompareMay 7, 2026 03:25
@wdcui

Copy link
Copy Markdown
MemberAuthor

Weiteng Chen (@CvvT) and Sangho Lee (@sangho2) this PR is ready for your review. Thanks!

@CvvT

Weiteng Chen (CvvT) commented May 8, 2026

Copy link
Copy Markdown
Contributor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

@CvvT

Copy link
Copy Markdown
Contributor

I don't see why we need the red zone.

@wdcui

Copy link
Copy Markdown
MemberAuthor

I don't see why we need the red zone.

Good point. It's removed now.

@wdcui

Copy link
Copy Markdown
MemberAuthor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

ctx.rip = ctx.rcx - 6 since we don't need to rerun the lea of rcx (and lea rsp is removed now).

@wdcuiWeidong Cui (wdcui) changed the title Add trampoline preamble with red zone reservation and R11 restart addressSyscall rewriter improvementsMay 12, 2026

@CvvTWeiteng Chen (CvvT) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Weidong Cui (wdcui)and others added 8 commits May 12, 2026 20:10
…ress
The syscall rewriter now emits a 12-byte preamble before each trampoline
stub: LEA RSP,[RSP-0x80] to reserve the SysV 128-byte red zone, and
LEA R11,[RIP+disp32] to load the call-site restart address into R11 for
future SA_RESTART support.
Platform callbacks are renamed to syscall_callback_redzone to reflect
the new calling convention. The callback recovers the architectural RSP
with LEA R11,[RSP+128] and saves the restart address to TLS
(saved_restart_addr) before clobbering R11.
The rewriter also emits a size=0 header sentinel for binaries with no
syscall instructions, allowing the loader to distinguish 'checked, no
syscalls' from 'never processed.' The is_already_hooked check treats
size=0 as already-hooked. The rtld_audit library is updated to handle
both the new calling convention (red zone + R11) and size=0 binaries
(via mincore probe before reading trampoline memory).
The pre-built binary had old-format trampolines that jumped to
syscall_callback_redzone without reserving the red zone, causing
an access violation (0xc0000005) on Windows.
…oline anchor
Per CvvT's review: the R11 restart-address load and the SysV red zone
preamble exist to support each other (the Windows callback's R11 shuttle
was the only thing that wrote to the guest stack pre-stack-switch, and
the red zone reservation existed to protect against that shuttle).
Replace the scheme so every stub satisfies:
pt_regs.rcx - 6 == address of JMP [syscall_entry_slot]
RCX uniformly points at the in-trampoline `post_jmp`, which is now a
5-byte `JMP rel32 -> guest_text` tail in every stub. The callback's
`jmp rcx` return path lands at post_jmp and falls through to guest text.
The SA_RESTART handler can rewind ctx.rip with a single subtract, with
no per-variant layout knowledge.
Drops, both platforms:
- saved_restart_addr TLS slot
- Windows TEB-slot shuttle (mov gs:[0x28], r11; etc.)
- lea r11, [rsp+128] architectural-RSP recovery
- LEA RSP, [RSP-0x80] red zone preamble
- LEA R11, [RIP+disp32] restart-address load
The post_jmp tail adds 5 bytes per stub. Net: -7 bytes per stub vs.
the prior approach, with the SA_RESTART invariant baked in for free.
Snapshot regenerated to reflect the new 18-byte-per-stub layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🤖 SemverChecks 🤖 No breaking API changes detected

Note: this does not mean API is unchanged, or even that there are no breaking changes; simply, none of the detections triggered.

@wdcui
Weidong Cui (wdcui) added this pull request to the merge queueMay 13, 2026
Merged via the queue into main with commit e728793May 13, 2026
13 checks passed
@wdcui
Weidong Cui (wdcui) deleted the wdcui/pr1-trampoline-format branch May 13, 2026 03:27
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

expmt:shadow-kilnTag to quickly find the different PRs as part of the "shadow kiln" experiment.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@wdcui@CvvT@jaybosamiya-ms
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Syscall rewriter improvements by wdcui · Pull Request #812 · microsoft/litebox · GitHub
Skip to content

Syscall rewriter improvements - #812

Merged
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format
May 13, 2026
Merged

Syscall rewriter improvements#812
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format

Conversation

@wdcui

@wdcuiWeidong Cui (wdcui) commented Apr 25, 2026

Copy link
Copy Markdown
Member

This PR changes how the return address is saved in rcx when syscalls are patched so that it can used to calculate the restart address. It also adds an empty trampoline to ELF files that don't have syscalls so the runtime knows it's already patched.

@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from 75ea7ec to 0bcb6ffCompareApril 25, 2026 04:06
@wdcui
Weidong Cui (wdcui) marked this pull request as ready for review April 25, 2026 18:50
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from a897889 to fd4fcb7CompareApril 25, 2026 21:31
@wdcui

Copy link
Copy Markdown
MemberAuthor

This PR is ready for review. Ignore the rtld changes since it will be removed after PR #810 is merged.

@jaybosamiya-msJay Bosamiya (Microsoft) (jaybosamiya-ms) added the expmt:shadow-kiln Tag to quickly find the different PRs as part of the "shadow kiln" experiment. label Apr 28, 2026
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch 2 times, most recently from d083228 to 097c515CompareMay 7, 2026 03:25
@wdcui

Copy link
Copy Markdown
MemberAuthor

Weiteng Chen (@CvvT) and Sangho Lee (@sangho2) this PR is ready for your review. Thanks!

@CvvT

Weiteng Chen (CvvT) commented May 8, 2026

Copy link
Copy Markdown
Contributor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

@CvvT

Copy link
Copy Markdown
Contributor

I don't see why we need the red zone.

@wdcui

Copy link
Copy Markdown
MemberAuthor

I don't see why we need the red zone.

Good point. It's removed now.

@wdcui

Copy link
Copy Markdown
MemberAuthor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

ctx.rip = ctx.rcx - 6 since we don't need to rerun the lea of rcx (and lea rsp is removed now).

@wdcuiWeidong Cui (wdcui) changed the title Add trampoline preamble with red zone reservation and R11 restart addressSyscall rewriter improvementsMay 12, 2026

@CvvTWeiteng Chen (CvvT) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Weidong Cui (wdcui)and others added 8 commits May 12, 2026 20:10
…ress
The syscall rewriter now emits a 12-byte preamble before each trampoline
stub: LEA RSP,[RSP-0x80] to reserve the SysV 128-byte red zone, and
LEA R11,[RIP+disp32] to load the call-site restart address into R11 for
future SA_RESTART support.
Platform callbacks are renamed to syscall_callback_redzone to reflect
the new calling convention. The callback recovers the architectural RSP
with LEA R11,[RSP+128] and saves the restart address to TLS
(saved_restart_addr) before clobbering R11.
The rewriter also emits a size=0 header sentinel for binaries with no
syscall instructions, allowing the loader to distinguish 'checked, no
syscalls' from 'never processed.' The is_already_hooked check treats
size=0 as already-hooked. The rtld_audit library is updated to handle
both the new calling convention (red zone + R11) and size=0 binaries
(via mincore probe before reading trampoline memory).
The pre-built binary had old-format trampolines that jumped to
syscall_callback_redzone without reserving the red zone, causing
an access violation (0xc0000005) on Windows.
…oline anchor
Per CvvT's review: the R11 restart-address load and the SysV red zone
preamble exist to support each other (the Windows callback's R11 shuttle
was the only thing that wrote to the guest stack pre-stack-switch, and
the red zone reservation existed to protect against that shuttle).
Replace the scheme so every stub satisfies:
pt_regs.rcx - 6 == address of JMP [syscall_entry_slot]
RCX uniformly points at the in-trampoline `post_jmp`, which is now a
5-byte `JMP rel32 -> guest_text` tail in every stub. The callback's
`jmp rcx` return path lands at post_jmp and falls through to guest text.
The SA_RESTART handler can rewind ctx.rip with a single subtract, with
no per-variant layout knowledge.
Drops, both platforms:
- saved_restart_addr TLS slot
- Windows TEB-slot shuttle (mov gs:[0x28], r11; etc.)
- lea r11, [rsp+128] architectural-RSP recovery
- LEA RSP, [RSP-0x80] red zone preamble
- LEA R11, [RIP+disp32] restart-address load
The post_jmp tail adds 5 bytes per stub. Net: -7 bytes per stub vs.
the prior approach, with the SA_RESTART invariant baked in for free.
Snapshot regenerated to reflect the new 18-byte-per-stub layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🤖 SemverChecks 🤖 No breaking API changes detected

Note: this does not mean API is unchanged, or even that there are no breaking changes; simply, none of the detections triggered.

@wdcui
Weidong Cui (wdcui) added this pull request to the merge queueMay 13, 2026
Merged via the queue into main with commit e728793May 13, 2026
13 checks passed
@wdcui
Weidong Cui (wdcui) deleted the wdcui/pr1-trampoline-format branch May 13, 2026 03:27
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

expmt:shadow-kilnTag to quickly find the different PRs as part of the "shadow kiln" experiment.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@wdcui@CvvT@jaybosamiya-ms
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Syscall rewriter improvements by wdcui · Pull Request #812 · microsoft/litebox · GitHub
Skip to content

Syscall rewriter improvements - #812

Merged
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format
May 13, 2026
Merged

Syscall rewriter improvements#812
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format

Conversation

@wdcui

@wdcuiWeidong Cui (wdcui) commented Apr 25, 2026

Copy link
Copy Markdown
Member

This PR changes how the return address is saved in rcx when syscalls are patched so that it can used to calculate the restart address. It also adds an empty trampoline to ELF files that don't have syscalls so the runtime knows it's already patched.

@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from 75ea7ec to 0bcb6ffCompareApril 25, 2026 04:06
@wdcui
Weidong Cui (wdcui) marked this pull request as ready for review April 25, 2026 18:50
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from a897889 to fd4fcb7CompareApril 25, 2026 21:31
@wdcui

Copy link
Copy Markdown
MemberAuthor

This PR is ready for review. Ignore the rtld changes since it will be removed after PR #810 is merged.

@jaybosamiya-msJay Bosamiya (Microsoft) (jaybosamiya-ms) added the expmt:shadow-kiln Tag to quickly find the different PRs as part of the "shadow kiln" experiment. label Apr 28, 2026
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch 2 times, most recently from d083228 to 097c515CompareMay 7, 2026 03:25
@wdcui

Copy link
Copy Markdown
MemberAuthor

Weiteng Chen (@CvvT) and Sangho Lee (@sangho2) this PR is ready for your review. Thanks!

@CvvT

Weiteng Chen (CvvT) commented May 8, 2026

Copy link
Copy Markdown
Contributor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

@CvvT

Copy link
Copy Markdown
Contributor

I don't see why we need the red zone.

@wdcui

Copy link
Copy Markdown
MemberAuthor

I don't see why we need the red zone.

Good point. It's removed now.

@wdcui

Copy link
Copy Markdown
MemberAuthor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

ctx.rip = ctx.rcx - 6 since we don't need to rerun the lea of rcx (and lea rsp is removed now).

@wdcuiWeidong Cui (wdcui) changed the title Add trampoline preamble with red zone reservation and R11 restart addressSyscall rewriter improvementsMay 12, 2026

@CvvTWeiteng Chen (CvvT) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Weidong Cui (wdcui)and others added 8 commits May 12, 2026 20:10
…ress
The syscall rewriter now emits a 12-byte preamble before each trampoline
stub: LEA RSP,[RSP-0x80] to reserve the SysV 128-byte red zone, and
LEA R11,[RIP+disp32] to load the call-site restart address into R11 for
future SA_RESTART support.
Platform callbacks are renamed to syscall_callback_redzone to reflect
the new calling convention. The callback recovers the architectural RSP
with LEA R11,[RSP+128] and saves the restart address to TLS
(saved_restart_addr) before clobbering R11.
The rewriter also emits a size=0 header sentinel for binaries with no
syscall instructions, allowing the loader to distinguish 'checked, no
syscalls' from 'never processed.' The is_already_hooked check treats
size=0 as already-hooked. The rtld_audit library is updated to handle
both the new calling convention (red zone + R11) and size=0 binaries
(via mincore probe before reading trampoline memory).
The pre-built binary had old-format trampolines that jumped to
syscall_callback_redzone without reserving the red zone, causing
an access violation (0xc0000005) on Windows.
…oline anchor
Per CvvT's review: the R11 restart-address load and the SysV red zone
preamble exist to support each other (the Windows callback's R11 shuttle
was the only thing that wrote to the guest stack pre-stack-switch, and
the red zone reservation existed to protect against that shuttle).
Replace the scheme so every stub satisfies:
pt_regs.rcx - 6 == address of JMP [syscall_entry_slot]
RCX uniformly points at the in-trampoline `post_jmp`, which is now a
5-byte `JMP rel32 -> guest_text` tail in every stub. The callback's
`jmp rcx` return path lands at post_jmp and falls through to guest text.
The SA_RESTART handler can rewind ctx.rip with a single subtract, with
no per-variant layout knowledge.
Drops, both platforms:
- saved_restart_addr TLS slot
- Windows TEB-slot shuttle (mov gs:[0x28], r11; etc.)
- lea r11, [rsp+128] architectural-RSP recovery
- LEA RSP, [RSP-0x80] red zone preamble
- LEA R11, [RIP+disp32] restart-address load
The post_jmp tail adds 5 bytes per stub. Net: -7 bytes per stub vs.
the prior approach, with the SA_RESTART invariant baked in for free.
Snapshot regenerated to reflect the new 18-byte-per-stub layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🤖 SemverChecks 🤖 No breaking API changes detected

Note: this does not mean API is unchanged, or even that there are no breaking changes; simply, none of the detections triggered.

@wdcui
Weidong Cui (wdcui) added this pull request to the merge queueMay 13, 2026
Merged via the queue into main with commit e728793May 13, 2026
13 checks passed
@wdcui
Weidong Cui (wdcui) deleted the wdcui/pr1-trampoline-format branch May 13, 2026 03:27
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

expmt:shadow-kilnTag to quickly find the different PRs as part of the "shadow kiln" experiment.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@wdcui@CvvT@jaybosamiya-ms
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' Syscall rewriter improvements by wdcui · Pull Request #812 · microsoft/litebox · GitHub
Skip to content

Syscall rewriter improvements - #812

Merged
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format
May 13, 2026
Merged

Syscall rewriter improvements#812
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format

Conversation

@wdcui

@wdcuiWeidong Cui (wdcui) commented Apr 25, 2026

Copy link
Copy Markdown
Member

This PR changes how the return address is saved in rcx when syscalls are patched so that it can used to calculate the restart address. It also adds an empty trampoline to ELF files that don't have syscalls so the runtime knows it's already patched.

@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from 75ea7ec to 0bcb6ffCompareApril 25, 2026 04:06
@wdcui
Weidong Cui (wdcui) marked this pull request as ready for review April 25, 2026 18:50
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from a897889 to fd4fcb7CompareApril 25, 2026 21:31
@wdcui

Copy link
Copy Markdown
MemberAuthor

This PR is ready for review. Ignore the rtld changes since it will be removed after PR #810 is merged.

@jaybosamiya-msJay Bosamiya (Microsoft) (jaybosamiya-ms) added the expmt:shadow-kiln Tag to quickly find the different PRs as part of the "shadow kiln" experiment. label Apr 28, 2026
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch 2 times, most recently from d083228 to 097c515CompareMay 7, 2026 03:25
@wdcui

Copy link
Copy Markdown
MemberAuthor

Weiteng Chen (@CvvT) and Sangho Lee (@sangho2) this PR is ready for your review. Thanks!

@CvvT

Weiteng Chen (CvvT) commented May 8, 2026

Copy link
Copy Markdown
Contributor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

@CvvT

Copy link
Copy Markdown
Contributor

I don't see why we need the red zone.

@wdcui

Copy link
Copy Markdown
MemberAuthor

I don't see why we need the red zone.

Good point. It's removed now.

@wdcui

Copy link
Copy Markdown
MemberAuthor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

ctx.rip = ctx.rcx - 6 since we don't need to rerun the lea of rcx (and lea rsp is removed now).

@wdcuiWeidong Cui (wdcui) changed the title Add trampoline preamble with red zone reservation and R11 restart addressSyscall rewriter improvementsMay 12, 2026

@CvvTWeiteng Chen (CvvT) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Weidong Cui (wdcui)and others added 8 commits May 12, 2026 20:10
…ress
The syscall rewriter now emits a 12-byte preamble before each trampoline
stub: LEA RSP,[RSP-0x80] to reserve the SysV 128-byte red zone, and
LEA R11,[RIP+disp32] to load the call-site restart address into R11 for
future SA_RESTART support.
Platform callbacks are renamed to syscall_callback_redzone to reflect
the new calling convention. The callback recovers the architectural RSP
with LEA R11,[RSP+128] and saves the restart address to TLS
(saved_restart_addr) before clobbering R11.
The rewriter also emits a size=0 header sentinel for binaries with no
syscall instructions, allowing the loader to distinguish 'checked, no
syscalls' from 'never processed.' The is_already_hooked check treats
size=0 as already-hooked. The rtld_audit library is updated to handle
both the new calling convention (red zone + R11) and size=0 binaries
(via mincore probe before reading trampoline memory).
The pre-built binary had old-format trampolines that jumped to
syscall_callback_redzone without reserving the red zone, causing
an access violation (0xc0000005) on Windows.
…oline anchor
Per CvvT's review: the R11 restart-address load and the SysV red zone
preamble exist to support each other (the Windows callback's R11 shuttle
was the only thing that wrote to the guest stack pre-stack-switch, and
the red zone reservation existed to protect against that shuttle).
Replace the scheme so every stub satisfies:
pt_regs.rcx - 6 == address of JMP [syscall_entry_slot]
RCX uniformly points at the in-trampoline `post_jmp`, which is now a
5-byte `JMP rel32 -> guest_text` tail in every stub. The callback's
`jmp rcx` return path lands at post_jmp and falls through to guest text.
The SA_RESTART handler can rewind ctx.rip with a single subtract, with
no per-variant layout knowledge.
Drops, both platforms:
- saved_restart_addr TLS slot
- Windows TEB-slot shuttle (mov gs:[0x28], r11; etc.)
- lea r11, [rsp+128] architectural-RSP recovery
- LEA RSP, [RSP-0x80] red zone preamble
- LEA R11, [RIP+disp32] restart-address load
The post_jmp tail adds 5 bytes per stub. Net: -7 bytes per stub vs.
the prior approach, with the SA_RESTART invariant baked in for free.
Snapshot regenerated to reflect the new 18-byte-per-stub layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🤖 SemverChecks 🤖 No breaking API changes detected

Note: this does not mean API is unchanged, or even that there are no breaking changes; simply, none of the detections triggered.

@wdcui
Weidong Cui (wdcui) added this pull request to the merge queueMay 13, 2026
Merged via the queue into main with commit e728793May 13, 2026
13 checks passed
@wdcui
Weidong Cui (wdcui) deleted the wdcui/pr1-trampoline-format branch May 13, 2026 03:27
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

expmt:shadow-kilnTag to quickly find the different PRs as part of the "shadow kiln" experiment.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@wdcui@CvvT@jaybosamiya-ms
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Syscall rewriter improvements by wdcui · Pull Request #812 · microsoft/litebox · GitHub
Skip to content

Syscall rewriter improvements - #812

Merged
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format
May 13, 2026
Merged

Syscall rewriter improvements#812
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format

Conversation

@wdcui

@wdcuiWeidong Cui (wdcui) commented Apr 25, 2026

Copy link
Copy Markdown
Member

This PR changes how the return address is saved in rcx when syscalls are patched so that it can used to calculate the restart address. It also adds an empty trampoline to ELF files that don't have syscalls so the runtime knows it's already patched.

@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from 75ea7ec to 0bcb6ffCompareApril 25, 2026 04:06
@wdcui
Weidong Cui (wdcui) marked this pull request as ready for review April 25, 2026 18:50
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from a897889 to fd4fcb7CompareApril 25, 2026 21:31
@wdcui

Copy link
Copy Markdown
MemberAuthor

This PR is ready for review. Ignore the rtld changes since it will be removed after PR #810 is merged.

@jaybosamiya-msJay Bosamiya (Microsoft) (jaybosamiya-ms) added the expmt:shadow-kiln Tag to quickly find the different PRs as part of the "shadow kiln" experiment. label Apr 28, 2026
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch 2 times, most recently from d083228 to 097c515CompareMay 7, 2026 03:25
@wdcui

Copy link
Copy Markdown
MemberAuthor

Weiteng Chen (@CvvT) and Sangho Lee (@sangho2) this PR is ready for your review. Thanks!

@CvvT

Weiteng Chen (CvvT) commented May 8, 2026

Copy link
Copy Markdown
Contributor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

@CvvT

Copy link
Copy Markdown
Contributor

I don't see why we need the red zone.

@wdcui

Copy link
Copy Markdown
MemberAuthor

I don't see why we need the red zone.

Good point. It's removed now.

@wdcui

Copy link
Copy Markdown
MemberAuthor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

ctx.rip = ctx.rcx - 6 since we don't need to rerun the lea of rcx (and lea rsp is removed now).

@wdcuiWeidong Cui (wdcui) changed the title Add trampoline preamble with red zone reservation and R11 restart addressSyscall rewriter improvementsMay 12, 2026

@CvvTWeiteng Chen (CvvT) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Weidong Cui (wdcui)and others added 8 commits May 12, 2026 20:10
…ress
The syscall rewriter now emits a 12-byte preamble before each trampoline
stub: LEA RSP,[RSP-0x80] to reserve the SysV 128-byte red zone, and
LEA R11,[RIP+disp32] to load the call-site restart address into R11 for
future SA_RESTART support.
Platform callbacks are renamed to syscall_callback_redzone to reflect
the new calling convention. The callback recovers the architectural RSP
with LEA R11,[RSP+128] and saves the restart address to TLS
(saved_restart_addr) before clobbering R11.
The rewriter also emits a size=0 header sentinel for binaries with no
syscall instructions, allowing the loader to distinguish 'checked, no
syscalls' from 'never processed.' The is_already_hooked check treats
size=0 as already-hooked. The rtld_audit library is updated to handle
both the new calling convention (red zone + R11) and size=0 binaries
(via mincore probe before reading trampoline memory).
The pre-built binary had old-format trampolines that jumped to
syscall_callback_redzone without reserving the red zone, causing
an access violation (0xc0000005) on Windows.
…oline anchor
Per CvvT's review: the R11 restart-address load and the SysV red zone
preamble exist to support each other (the Windows callback's R11 shuttle
was the only thing that wrote to the guest stack pre-stack-switch, and
the red zone reservation existed to protect against that shuttle).
Replace the scheme so every stub satisfies:
pt_regs.rcx - 6 == address of JMP [syscall_entry_slot]
RCX uniformly points at the in-trampoline `post_jmp`, which is now a
5-byte `JMP rel32 -> guest_text` tail in every stub. The callback's
`jmp rcx` return path lands at post_jmp and falls through to guest text.
The SA_RESTART handler can rewind ctx.rip with a single subtract, with
no per-variant layout knowledge.
Drops, both platforms:
- saved_restart_addr TLS slot
- Windows TEB-slot shuttle (mov gs:[0x28], r11; etc.)
- lea r11, [rsp+128] architectural-RSP recovery
- LEA RSP, [RSP-0x80] red zone preamble
- LEA R11, [RIP+disp32] restart-address load
The post_jmp tail adds 5 bytes per stub. Net: -7 bytes per stub vs.
the prior approach, with the SA_RESTART invariant baked in for free.
Snapshot regenerated to reflect the new 18-byte-per-stub layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🤖 SemverChecks 🤖 No breaking API changes detected

Note: this does not mean API is unchanged, or even that there are no breaking changes; simply, none of the detections triggered.

@wdcui
Weidong Cui (wdcui) added this pull request to the merge queueMay 13, 2026
Merged via the queue into main with commit e728793May 13, 2026
13 checks passed
@wdcui
Weidong Cui (wdcui) deleted the wdcui/pr1-trampoline-format branch May 13, 2026 03:27
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

expmt:shadow-kilnTag to quickly find the different PRs as part of the "shadow kiln" experiment.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@wdcui@CvvT@jaybosamiya-ms
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Syscall rewriter improvements by wdcui · Pull Request #812 · microsoft/litebox · GitHub
Skip to content

Syscall rewriter improvements - #812

Merged
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format
May 13, 2026
Merged

Syscall rewriter improvements#812
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format

Conversation

@wdcui

@wdcuiWeidong Cui (wdcui) commented Apr 25, 2026

Copy link
Copy Markdown
Member

This PR changes how the return address is saved in rcx when syscalls are patched so that it can used to calculate the restart address. It also adds an empty trampoline to ELF files that don't have syscalls so the runtime knows it's already patched.

@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from 75ea7ec to 0bcb6ffCompareApril 25, 2026 04:06
@wdcui
Weidong Cui (wdcui) marked this pull request as ready for review April 25, 2026 18:50
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from a897889 to fd4fcb7CompareApril 25, 2026 21:31
@wdcui

Copy link
Copy Markdown
MemberAuthor

This PR is ready for review. Ignore the rtld changes since it will be removed after PR #810 is merged.

@jaybosamiya-msJay Bosamiya (Microsoft) (jaybosamiya-ms) added the expmt:shadow-kiln Tag to quickly find the different PRs as part of the "shadow kiln" experiment. label Apr 28, 2026
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch 2 times, most recently from d083228 to 097c515CompareMay 7, 2026 03:25
@wdcui

Copy link
Copy Markdown
MemberAuthor

Weiteng Chen (@CvvT) and Sangho Lee (@sangho2) this PR is ready for your review. Thanks!

@CvvT

Weiteng Chen (CvvT) commented May 8, 2026

Copy link
Copy Markdown
Contributor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

@CvvT

Copy link
Copy Markdown
Contributor

I don't see why we need the red zone.

@wdcui

Copy link
Copy Markdown
MemberAuthor

I don't see why we need the red zone.

Good point. It's removed now.

@wdcui

Copy link
Copy Markdown
MemberAuthor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

ctx.rip = ctx.rcx - 6 since we don't need to rerun the lea of rcx (and lea rsp is removed now).

@wdcuiWeidong Cui (wdcui) changed the title Add trampoline preamble with red zone reservation and R11 restart addressSyscall rewriter improvementsMay 12, 2026

@CvvTWeiteng Chen (CvvT) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Weidong Cui (wdcui)and others added 8 commits May 12, 2026 20:10
…ress
The syscall rewriter now emits a 12-byte preamble before each trampoline
stub: LEA RSP,[RSP-0x80] to reserve the SysV 128-byte red zone, and
LEA R11,[RIP+disp32] to load the call-site restart address into R11 for
future SA_RESTART support.
Platform callbacks are renamed to syscall_callback_redzone to reflect
the new calling convention. The callback recovers the architectural RSP
with LEA R11,[RSP+128] and saves the restart address to TLS
(saved_restart_addr) before clobbering R11.
The rewriter also emits a size=0 header sentinel for binaries with no
syscall instructions, allowing the loader to distinguish 'checked, no
syscalls' from 'never processed.' The is_already_hooked check treats
size=0 as already-hooked. The rtld_audit library is updated to handle
both the new calling convention (red zone + R11) and size=0 binaries
(via mincore probe before reading trampoline memory).
The pre-built binary had old-format trampolines that jumped to
syscall_callback_redzone without reserving the red zone, causing
an access violation (0xc0000005) on Windows.
…oline anchor
Per CvvT's review: the R11 restart-address load and the SysV red zone
preamble exist to support each other (the Windows callback's R11 shuttle
was the only thing that wrote to the guest stack pre-stack-switch, and
the red zone reservation existed to protect against that shuttle).
Replace the scheme so every stub satisfies:
pt_regs.rcx - 6 == address of JMP [syscall_entry_slot]
RCX uniformly points at the in-trampoline `post_jmp`, which is now a
5-byte `JMP rel32 -> guest_text` tail in every stub. The callback's
`jmp rcx` return path lands at post_jmp and falls through to guest text.
The SA_RESTART handler can rewind ctx.rip with a single subtract, with
no per-variant layout knowledge.
Drops, both platforms:
- saved_restart_addr TLS slot
- Windows TEB-slot shuttle (mov gs:[0x28], r11; etc.)
- lea r11, [rsp+128] architectural-RSP recovery
- LEA RSP, [RSP-0x80] red zone preamble
- LEA R11, [RIP+disp32] restart-address load
The post_jmp tail adds 5 bytes per stub. Net: -7 bytes per stub vs.
the prior approach, with the SA_RESTART invariant baked in for free.
Snapshot regenerated to reflect the new 18-byte-per-stub layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🤖 SemverChecks 🤖 No breaking API changes detected

Note: this does not mean API is unchanged, or even that there are no breaking changes; simply, none of the detections triggered.

@wdcui
Weidong Cui (wdcui) added this pull request to the merge queueMay 13, 2026
Merged via the queue into main with commit e728793May 13, 2026
13 checks passed
@wdcui
Weidong Cui (wdcui) deleted the wdcui/pr1-trampoline-format branch May 13, 2026 03:27
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

expmt:shadow-kilnTag to quickly find the different PRs as part of the "shadow kiln" experiment.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@wdcui@CvvT@jaybosamiya-ms
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); Syscall rewriter improvements by wdcui · Pull Request #812 · microsoft/litebox · GitHub
Skip to content

Syscall rewriter improvements - #812

Merged
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format
May 13, 2026
Merged

Syscall rewriter improvements#812
Weidong Cui (wdcui) merged 8 commits into
mainfrom
wdcui/pr1-trampoline-format

Conversation

@wdcui

@wdcuiWeidong Cui (wdcui) commented Apr 25, 2026

Copy link
Copy Markdown
Member

This PR changes how the return address is saved in rcx when syscalls are patched so that it can used to calculate the restart address. It also adds an empty trampoline to ELF files that don't have syscalls so the runtime knows it's already patched.

@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from 75ea7ec to 0bcb6ffCompareApril 25, 2026 04:06
@wdcui
Weidong Cui (wdcui) marked this pull request as ready for review April 25, 2026 18:50
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch from a897889 to fd4fcb7CompareApril 25, 2026 21:31
@wdcui

Copy link
Copy Markdown
MemberAuthor

This PR is ready for review. Ignore the rtld changes since it will be removed after PR #810 is merged.

@jaybosamiya-msJay Bosamiya (Microsoft) (jaybosamiya-ms) added the expmt:shadow-kiln Tag to quickly find the different PRs as part of the "shadow kiln" experiment. label Apr 28, 2026
@wdcui
Weidong Cui (wdcui)force-pushed the wdcui/pr1-trampoline-format branch 2 times, most recently from d083228 to 097c515CompareMay 7, 2026 03:25
@wdcui

Copy link
Copy Markdown
MemberAuthor

Weiteng Chen (@CvvT) and Sangho Lee (@sangho2) this PR is ready for your review. Thanks!

@CvvT

Weiteng Chen (CvvT) commented May 8, 2026

Copy link
Copy Markdown
Contributor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

@CvvT

Copy link
Copy Markdown
Contributor

I don't see why we need the red zone.

@wdcui

Copy link
Copy Markdown
MemberAuthor

I don't see why we need the red zone.

Good point. It's removed now.

@wdcui

Copy link
Copy Markdown
MemberAuthor

Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from.

Trampoline stub (LEA R11 dropped):
stub_addr: ; ← restart target
LEA RSP, [RSP - 0x80] ; 5 bytes
LEA RCX, [post_jmp] ; 7 bytes — points inside trampoline
JMP [syscall_entry_slot] ; 6 bytes
post_jmp: ; offset 18 from stub_addr
JMP [guest_next_insn_slot] Restart:
ctx.rip = ctx.rcx - 18; // 18 = sizeof(LEA RSP) + sizeof(LEA RCX) + sizeof(JMP)

For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well.

ctx.rip = ctx.rcx - 6 since we don't need to rerun the lea of rcx (and lea rsp is removed now).

@wdcuiWeidong Cui (wdcui) changed the title Add trampoline preamble with red zone reservation and R11 restart addressSyscall rewriter improvementsMay 12, 2026

@CvvTWeiteng Chen (CvvT) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Weidong Cui (wdcui)and others added 8 commits May 12, 2026 20:10
…ress
The syscall rewriter now emits a 12-byte preamble before each trampoline
stub: LEA RSP,[RSP-0x80] to reserve the SysV 128-byte red zone, and
LEA R11,[RIP+disp32] to load the call-site restart address into R11 for
future SA_RESTART support.
Platform callbacks are renamed to syscall_callback_redzone to reflect
the new calling convention. The callback recovers the architectural RSP
with LEA R11,[RSP+128] and saves the restart address to TLS
(saved_restart_addr) before clobbering R11.
The rewriter also emits a size=0 header sentinel for binaries with no
syscall instructions, allowing the loader to distinguish 'checked, no
syscalls' from 'never processed.' The is_already_hooked check treats
size=0 as already-hooked. The rtld_audit library is updated to handle
both the new calling convention (red zone + R11) and size=0 binaries
(via mincore probe before reading trampoline memory).
The pre-built binary had old-format trampolines that jumped to
syscall_callback_redzone without reserving the red zone, causing
an access violation (0xc0000005) on Windows.
…oline anchor
Per CvvT's review: the R11 restart-address load and the SysV red zone
preamble exist to support each other (the Windows callback's R11 shuttle
was the only thing that wrote to the guest stack pre-stack-switch, and
the red zone reservation existed to protect against that shuttle).
Replace the scheme so every stub satisfies:
pt_regs.rcx - 6 == address of JMP [syscall_entry_slot]
RCX uniformly points at the in-trampoline `post_jmp`, which is now a
5-byte `JMP rel32 -> guest_text` tail in every stub. The callback's
`jmp rcx` return path lands at post_jmp and falls through to guest text.
The SA_RESTART handler can rewind ctx.rip with a single subtract, with
no per-variant layout knowledge.
Drops, both platforms:
- saved_restart_addr TLS slot
- Windows TEB-slot shuttle (mov gs:[0x28], r11; etc.)
- lea r11, [rsp+128] architectural-RSP recovery
- LEA RSP, [RSP-0x80] red zone preamble
- LEA R11, [RIP+disp32] restart-address load
The post_jmp tail adds 5 bytes per stub. Net: -7 bytes per stub vs.
the prior approach, with the SA_RESTART invariant baked in for free.
Snapshot regenerated to reflect the new 18-byte-per-stub layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🤖 SemverChecks 🤖 No breaking API changes detected

Note: this does not mean API is unchanged, or even that there are no breaking changes; simply, none of the detections triggered.

@wdcui
Weidong Cui (wdcui) added this pull request to the merge queueMay 13, 2026
Merged via the queue into main with commit e728793May 13, 2026
13 checks passed
@wdcui
Weidong Cui (wdcui) deleted the wdcui/pr1-trampoline-format branch May 13, 2026 03:27
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

expmt:shadow-kilnTag to quickly find the different PRs as part of the "shadow kiln" experiment.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@wdcui@CvvT@jaybosamiya-ms