Document syntax movement tables in the grimoire#833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
jackfirth wants to merge 2 commits into
base:master
Choose a base branch
from
Draft

Document syntax movement tables in the grimoire #833

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
25bab47
Move syntax-movement module into resyntax/grimoire
jackfirth Jul 17, 2026
dcbffc0
Draft API reference for syntax movement tables
jackfirth Jul 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions grimoire.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,5 +17,6 @@ programmatically on anything found here.
@include-section[(lib "resyntax/grimoire/syntax-path.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-property-bundle.scrbl")]
@include-section[(lib "resyntax/grimoire/expansion-analyzers.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-movement.scrbl")]
@include-section[(lib "resyntax/grimoire/string-replacement.scrbl")]
@include-section[(lib "resyntax/grimoire/linemap.scrbl")]
0 private/syntax-movement.rkt → grimoire/syntax-movement.rkt
View file
Open in desktop
File renamed without changes.
87 changes: 87 additions & 0 deletions grimoire/syntax-movement.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
#lang scribble/manual


@(require (for-label racket/base
racket/contract/base
rebellion/collection/sorted-map
rebellion/collection/sorted-set
resyntax/grimoire/source
resyntax/grimoire/syntax-movement
resyntax/grimoire/syntax-path)
scribble/example
(submod resyntax/private/scribble-evaluator-factory doc))


@(define make-evaluator
(make-module-sharing-evaluator-factory
#:public (list 'rebellion/collection/sorted-map
'resyntax/grimoire/source
'resyntax/grimoire/syntax-movement
'resyntax/grimoire/syntax-path)
#:private (list 'racket/base)))


@title[#:tag "syntax-movement"]{Syntax Movement Tables}
@defmodule[resyntax/grimoire/syntax-movement]

A @deftech{syntax movement table} records where each piece of a program's original syntax ended up
after the program was transformed, as a mapping between @tech{syntax paths}. The table's keys are
@tech{original syntax paths} --- the paths that subforms of the untransformed program were located
at when it was first read, as recorded by the @racket['original-syntax-path] property that
@racket[source-read-syntax] attaches. The table's values are sets of paths identifying every
location in the @emph{transformed} program where a subform claiming that original path can be
found. Resyntax primarily builds movement tables for fully expanded programs, in which case the
table describes where the macro expander moved each piece of the original program.

@; TODO(@notjack.space): broader overview goes here. Why movement tables exist: the role they play
@; in the analysis pipeline (translating expansion analyzer findings and expansion-time lexical
@; context back onto the original unexpanded program) and the role they're expected to play in the
@; pluggable whole-module analyzer system.

Each original path maps to a @emph{set} of paths in the transformed program, rather than to a
single path, because a single piece of original syntax can end up in several places at once ---
macros are free to copy their subforms. A @racket[struct] definition, for example, copies the
struct name into the several identifiers it defines, so the original path of the name maps to
every expanded location those copies ended up at. The reverse situation also occurs: original
syntax that the transformation discarded entirely appears nowhere in the transformed program, and
its path is simply absent from the table's keys.

@; TODO(@notjack.space): this one-to-many ambiguity is the crux of a lot of behavior worth
@; explaining in your own words: when several expanded forms claim the same original path,
@; Resyntax's analysis refuses to pick a winner, discarding expansion analyzer properties and
@; expansion-time lexical context for that path (the "multiple expanded forms claim to originate
@; from that path" log messages). Worth mentioning how that policy blocks things like issue #688
@; and what a more principled disambiguation might look like.

@; TODO(@notjack.space): possibly also worth hinting here: movement tables currently describe what
@; the macro expander did, but the same input-paths-to-output-paths shape describes what a
@; refactoring rule's transformation did. If tables are ever built for rule outputs, they'd be the
@; basis for automatically matching up input and output shape tags in UTS syntax deltas.


@defproc[(syntax-movement-table [result-stx syntax?]) immutable-sorted-map?]{
Builds a @tech{syntax movement table} for @racket[result-stx] by traversing it and collecting
every subform that carries an @tech{original syntax path}, including subforms nested within other
collected subforms. The returned table is an immutable sorted map whose keys are the original
syntax paths that were found, ordered by @racket[syntax-path<=>], and whose values are immutable
sorted sets of the paths within @racket[result-stx] at which the claiming subforms are located.

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded (source-expand (string-source "#lang racket/base\n(void)\n"))))
(syntax-movement-table expanded))

When the transformation copies a piece of original syntax into several places, every copy's
location appears in that original path's set:

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded
(source-expand (string-source "#lang racket/base\n(struct point (x y))\n")))
(define table (syntax-movement-table expanded)))
(code:comment "The original path of the identifier `point` within `(struct point (x y))`:")
(define point-path (syntax-path (list 3 1 1)))
(code:comment "Expansion copied that identifier into four different definitions.")
(sorted-map-get table point-path))}
2 changes: 1 addition & 1 deletion private/analysis.rkt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,7 +46,7 @@
resyntax/private/logger
resyntax/grimoire/source
resyntax/private/string-indent
resyntax/private/syntax-movement
resyntax/grimoire/syntax-movement
resyntax/private/syntax-neighbors
resyntax/grimoire/syntax-path
resyntax/grimoire/syntax-property-bundle
Expand Down
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all \u003cpre\u003e\u003ccode\u003e blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks"); } } catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); } })(); (function(){ try { var __m = "github.com"; var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Document syntax movement tables in the grimoire#833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
jackfirth wants to merge 2 commits into
base:master
Choose a base branch
from
Draft

Document syntax movement tables in the grimoire #833

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
25bab47
Move syntax-movement module into resyntax/grimoire
jackfirth Jul 17, 2026
dcbffc0
Draft API reference for syntax movement tables
jackfirth Jul 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions grimoire.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,5 +17,6 @@ programmatically on anything found here.
@include-section[(lib "resyntax/grimoire/syntax-path.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-property-bundle.scrbl")]
@include-section[(lib "resyntax/grimoire/expansion-analyzers.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-movement.scrbl")]
@include-section[(lib "resyntax/grimoire/string-replacement.scrbl")]
@include-section[(lib "resyntax/grimoire/linemap.scrbl")]
0 private/syntax-movement.rkt → grimoire/syntax-movement.rkt
View file
Open in desktop
File renamed without changes.
87 changes: 87 additions & 0 deletions grimoire/syntax-movement.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
#lang scribble/manual


@(require (for-label racket/base
racket/contract/base
rebellion/collection/sorted-map
rebellion/collection/sorted-set
resyntax/grimoire/source
resyntax/grimoire/syntax-movement
resyntax/grimoire/syntax-path)
scribble/example
(submod resyntax/private/scribble-evaluator-factory doc))


@(define make-evaluator
(make-module-sharing-evaluator-factory
#:public (list 'rebellion/collection/sorted-map
'resyntax/grimoire/source
'resyntax/grimoire/syntax-movement
'resyntax/grimoire/syntax-path)
#:private (list 'racket/base)))


@title[#:tag "syntax-movement"]{Syntax Movement Tables}
@defmodule[resyntax/grimoire/syntax-movement]

A @deftech{syntax movement table} records where each piece of a program's original syntax ended up
after the program was transformed, as a mapping between @tech{syntax paths}. The table's keys are
@tech{original syntax paths} --- the paths that subforms of the untransformed program were located
at when it was first read, as recorded by the @racket['original-syntax-path] property that
@racket[source-read-syntax] attaches. The table's values are sets of paths identifying every
location in the @emph{transformed} program where a subform claiming that original path can be
found. Resyntax primarily builds movement tables for fully expanded programs, in which case the
table describes where the macro expander moved each piece of the original program.

@; TODO(@notjack.space): broader overview goes here. Why movement tables exist: the role they play
@; in the analysis pipeline (translating expansion analyzer findings and expansion-time lexical
@; context back onto the original unexpanded program) and the role they're expected to play in the
@; pluggable whole-module analyzer system.

Each original path maps to a @emph{set} of paths in the transformed program, rather than to a
single path, because a single piece of original syntax can end up in several places at once ---
macros are free to copy their subforms. A @racket[struct] definition, for example, copies the
struct name into the several identifiers it defines, so the original path of the name maps to
every expanded location those copies ended up at. The reverse situation also occurs: original
syntax that the transformation discarded entirely appears nowhere in the transformed program, and
its path is simply absent from the table's keys.

@; TODO(@notjack.space): this one-to-many ambiguity is the crux of a lot of behavior worth
@; explaining in your own words: when several expanded forms claim the same original path,
@; Resyntax's analysis refuses to pick a winner, discarding expansion analyzer properties and
@; expansion-time lexical context for that path (the "multiple expanded forms claim to originate
@; from that path" log messages). Worth mentioning how that policy blocks things like issue #688
@; and what a more principled disambiguation might look like.

@; TODO(@notjack.space): possibly also worth hinting here: movement tables currently describe what
@; the macro expander did, but the same input-paths-to-output-paths shape describes what a
@; refactoring rule's transformation did. If tables are ever built for rule outputs, they'd be the
@; basis for automatically matching up input and output shape tags in UTS syntax deltas.


@defproc[(syntax-movement-table [result-stx syntax?]) immutable-sorted-map?]{
Builds a @tech{syntax movement table} for @racket[result-stx] by traversing it and collecting
every subform that carries an @tech{original syntax path}, including subforms nested within other
collected subforms. The returned table is an immutable sorted map whose keys are the original
syntax paths that were found, ordered by @racket[syntax-path<=>], and whose values are immutable
sorted sets of the paths within @racket[result-stx] at which the claiming subforms are located.

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded (source-expand (string-source "#lang racket/base\n(void)\n"))))
(syntax-movement-table expanded))

When the transformation copies a piece of original syntax into several places, every copy's
location appears in that original path's set:

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded
(source-expand (string-source "#lang racket/base\n(struct point (x y))\n")))
(define table (syntax-movement-table expanded)))
(code:comment "The original path of the identifier `point` within `(struct point (x y))`:")
(define point-path (syntax-path (list 3 1 1)))
(code:comment "Expansion copied that identifier into four different definitions.")
(sorted-map-get table point-path))}
2 changes: 1 addition & 1 deletion private/analysis.rkt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,7 +46,7 @@
resyntax/private/logger
resyntax/grimoire/source
resyntax/private/string-indent
resyntax/private/syntax-movement
resyntax/grimoire/syntax-movement
resyntax/private/syntax-neighbors
resyntax/grimoire/syntax-path
resyntax/grimoire/syntax-property-bundle
Expand Down
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Document syntax movement tables in the grimoire#833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
jackfirth wants to merge 2 commits into
base:master
Choose a base branch
from
Draft

Document syntax movement tables in the grimoire #833

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
25bab47
Move syntax-movement module into resyntax/grimoire
jackfirth Jul 17, 2026
dcbffc0
Draft API reference for syntax movement tables
jackfirth Jul 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions grimoire.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,5 +17,6 @@ programmatically on anything found here.
@include-section[(lib "resyntax/grimoire/syntax-path.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-property-bundle.scrbl")]
@include-section[(lib "resyntax/grimoire/expansion-analyzers.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-movement.scrbl")]
@include-section[(lib "resyntax/grimoire/string-replacement.scrbl")]
@include-section[(lib "resyntax/grimoire/linemap.scrbl")]
0 private/syntax-movement.rkt → grimoire/syntax-movement.rkt
View file
Open in desktop
File renamed without changes.
87 changes: 87 additions & 0 deletions grimoire/syntax-movement.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
#lang scribble/manual


@(require (for-label racket/base
racket/contract/base
rebellion/collection/sorted-map
rebellion/collection/sorted-set
resyntax/grimoire/source
resyntax/grimoire/syntax-movement
resyntax/grimoire/syntax-path)
scribble/example
(submod resyntax/private/scribble-evaluator-factory doc))


@(define make-evaluator
(make-module-sharing-evaluator-factory
#:public (list 'rebellion/collection/sorted-map
'resyntax/grimoire/source
'resyntax/grimoire/syntax-movement
'resyntax/grimoire/syntax-path)
#:private (list 'racket/base)))


@title[#:tag "syntax-movement"]{Syntax Movement Tables}
@defmodule[resyntax/grimoire/syntax-movement]

A @deftech{syntax movement table} records where each piece of a program's original syntax ended up
after the program was transformed, as a mapping between @tech{syntax paths}. The table's keys are
@tech{original syntax paths} --- the paths that subforms of the untransformed program were located
at when it was first read, as recorded by the @racket['original-syntax-path] property that
@racket[source-read-syntax] attaches. The table's values are sets of paths identifying every
location in the @emph{transformed} program where a subform claiming that original path can be
found. Resyntax primarily builds movement tables for fully expanded programs, in which case the
table describes where the macro expander moved each piece of the original program.

@; TODO(@notjack.space): broader overview goes here. Why movement tables exist: the role they play
@; in the analysis pipeline (translating expansion analyzer findings and expansion-time lexical
@; context back onto the original unexpanded program) and the role they're expected to play in the
@; pluggable whole-module analyzer system.

Each original path maps to a @emph{set} of paths in the transformed program, rather than to a
single path, because a single piece of original syntax can end up in several places at once ---
macros are free to copy their subforms. A @racket[struct] definition, for example, copies the
struct name into the several identifiers it defines, so the original path of the name maps to
every expanded location those copies ended up at. The reverse situation also occurs: original
syntax that the transformation discarded entirely appears nowhere in the transformed program, and
its path is simply absent from the table's keys.

@; TODO(@notjack.space): this one-to-many ambiguity is the crux of a lot of behavior worth
@; explaining in your own words: when several expanded forms claim the same original path,
@; Resyntax's analysis refuses to pick a winner, discarding expansion analyzer properties and
@; expansion-time lexical context for that path (the "multiple expanded forms claim to originate
@; from that path" log messages). Worth mentioning how that policy blocks things like issue #688
@; and what a more principled disambiguation might look like.

@; TODO(@notjack.space): possibly also worth hinting here: movement tables currently describe what
@; the macro expander did, but the same input-paths-to-output-paths shape describes what a
@; refactoring rule's transformation did. If tables are ever built for rule outputs, they'd be the
@; basis for automatically matching up input and output shape tags in UTS syntax deltas.


@defproc[(syntax-movement-table [result-stx syntax?]) immutable-sorted-map?]{
Builds a @tech{syntax movement table} for @racket[result-stx] by traversing it and collecting
every subform that carries an @tech{original syntax path}, including subforms nested within other
collected subforms. The returned table is an immutable sorted map whose keys are the original
syntax paths that were found, ordered by @racket[syntax-path<=>], and whose values are immutable
sorted sets of the paths within @racket[result-stx] at which the claiming subforms are located.

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded (source-expand (string-source "#lang racket/base\n(void)\n"))))
(syntax-movement-table expanded))

When the transformation copies a piece of original syntax into several places, every copy's
location appears in that original path's set:

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded
(source-expand (string-source "#lang racket/base\n(struct point (x y))\n")))
(define table (syntax-movement-table expanded)))
(code:comment "The original path of the identifier `point` within `(struct point (x y))`:")
(define point-path (syntax-path (list 3 1 1)))
(code:comment "Expansion copied that identifier into four different definitions.")
(sorted-map-get table point-path))}
2 changes: 1 addition & 1 deletion private/analysis.rkt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,7 +46,7 @@
resyntax/private/logger
resyntax/grimoire/source
resyntax/private/string-indent
resyntax/private/syntax-movement
resyntax/grimoire/syntax-movement
resyntax/private/syntax-neighbors
resyntax/grimoire/syntax-path
resyntax/grimoire/syntax-property-bundle
Expand Down
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length \u003e 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Document syntax movement tables in the grimoire#833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
jackfirth wants to merge 2 commits into
base:master
Choose a base branch
from
Draft

Document syntax movement tables in the grimoire #833

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
25bab47
Move syntax-movement module into resyntax/grimoire
jackfirth Jul 17, 2026
dcbffc0
Draft API reference for syntax movement tables
jackfirth Jul 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions grimoire.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,5 +17,6 @@ programmatically on anything found here.
@include-section[(lib "resyntax/grimoire/syntax-path.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-property-bundle.scrbl")]
@include-section[(lib "resyntax/grimoire/expansion-analyzers.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-movement.scrbl")]
@include-section[(lib "resyntax/grimoire/string-replacement.scrbl")]
@include-section[(lib "resyntax/grimoire/linemap.scrbl")]
0 private/syntax-movement.rkt → grimoire/syntax-movement.rkt
View file
Open in desktop
File renamed without changes.
87 changes: 87 additions & 0 deletions grimoire/syntax-movement.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
#lang scribble/manual


@(require (for-label racket/base
racket/contract/base
rebellion/collection/sorted-map
rebellion/collection/sorted-set
resyntax/grimoire/source
resyntax/grimoire/syntax-movement
resyntax/grimoire/syntax-path)
scribble/example
(submod resyntax/private/scribble-evaluator-factory doc))


@(define make-evaluator
(make-module-sharing-evaluator-factory
#:public (list 'rebellion/collection/sorted-map
'resyntax/grimoire/source
'resyntax/grimoire/syntax-movement
'resyntax/grimoire/syntax-path)
#:private (list 'racket/base)))


@title[#:tag "syntax-movement"]{Syntax Movement Tables}
@defmodule[resyntax/grimoire/syntax-movement]

A @deftech{syntax movement table} records where each piece of a program's original syntax ended up
after the program was transformed, as a mapping between @tech{syntax paths}. The table's keys are
@tech{original syntax paths} --- the paths that subforms of the untransformed program were located
at when it was first read, as recorded by the @racket['original-syntax-path] property that
@racket[source-read-syntax] attaches. The table's values are sets of paths identifying every
location in the @emph{transformed} program where a subform claiming that original path can be
found. Resyntax primarily builds movement tables for fully expanded programs, in which case the
table describes where the macro expander moved each piece of the original program.

@; TODO(@notjack.space): broader overview goes here. Why movement tables exist: the role they play
@; in the analysis pipeline (translating expansion analyzer findings and expansion-time lexical
@; context back onto the original unexpanded program) and the role they're expected to play in the
@; pluggable whole-module analyzer system.

Each original path maps to a @emph{set} of paths in the transformed program, rather than to a
single path, because a single piece of original syntax can end up in several places at once ---
macros are free to copy their subforms. A @racket[struct] definition, for example, copies the
struct name into the several identifiers it defines, so the original path of the name maps to
every expanded location those copies ended up at. The reverse situation also occurs: original
syntax that the transformation discarded entirely appears nowhere in the transformed program, and
its path is simply absent from the table's keys.

@; TODO(@notjack.space): this one-to-many ambiguity is the crux of a lot of behavior worth
@; explaining in your own words: when several expanded forms claim the same original path,
@; Resyntax's analysis refuses to pick a winner, discarding expansion analyzer properties and
@; expansion-time lexical context for that path (the "multiple expanded forms claim to originate
@; from that path" log messages). Worth mentioning how that policy blocks things like issue #688
@; and what a more principled disambiguation might look like.

@; TODO(@notjack.space): possibly also worth hinting here: movement tables currently describe what
@; the macro expander did, but the same input-paths-to-output-paths shape describes what a
@; refactoring rule's transformation did. If tables are ever built for rule outputs, they'd be the
@; basis for automatically matching up input and output shape tags in UTS syntax deltas.


@defproc[(syntax-movement-table [result-stx syntax?]) immutable-sorted-map?]{
Builds a @tech{syntax movement table} for @racket[result-stx] by traversing it and collecting
every subform that carries an @tech{original syntax path}, including subforms nested within other
collected subforms. The returned table is an immutable sorted map whose keys are the original
syntax paths that were found, ordered by @racket[syntax-path<=>], and whose values are immutable
sorted sets of the paths within @racket[result-stx] at which the claiming subforms are located.

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded (source-expand (string-source "#lang racket/base\n(void)\n"))))
(syntax-movement-table expanded))

When the transformation copies a piece of original syntax into several places, every copy's
location appears in that original path's set:

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded
(source-expand (string-source "#lang racket/base\n(struct point (x y))\n")))
(define table (syntax-movement-table expanded)))
(code:comment "The original path of the identifier `point` within `(struct point (x y))`:")
(define point-path (syntax-path (list 3 1 1)))
(code:comment "Expansion copied that identifier into four different definitions.")
(sorted-map-get table point-path))}
2 changes: 1 addition & 1 deletion private/analysis.rkt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,7 +46,7 @@
resyntax/private/logger
resyntax/grimoire/source
resyntax/private/string-indent
resyntax/private/syntax-movement
resyntax/grimoire/syntax-movement
resyntax/private/syntax-neighbors
resyntax/grimoire/syntax-path
resyntax/grimoire/syntax-property-bundle
Expand Down
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Document syntax movement tables in the grimoire#833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
jackfirth wants to merge 2 commits into
base:master
Choose a base branch
from
Draft

Document syntax movement tables in the grimoire #833

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
25bab47
Move syntax-movement module into resyntax/grimoire
jackfirth Jul 17, 2026
dcbffc0
Draft API reference for syntax movement tables
jackfirth Jul 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions grimoire.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,5 +17,6 @@ programmatically on anything found here.
@include-section[(lib "resyntax/grimoire/syntax-path.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-property-bundle.scrbl")]
@include-section[(lib "resyntax/grimoire/expansion-analyzers.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-movement.scrbl")]
@include-section[(lib "resyntax/grimoire/string-replacement.scrbl")]
@include-section[(lib "resyntax/grimoire/linemap.scrbl")]
0 private/syntax-movement.rkt → grimoire/syntax-movement.rkt
View file
Open in desktop
File renamed without changes.
87 changes: 87 additions & 0 deletions grimoire/syntax-movement.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
#lang scribble/manual


@(require (for-label racket/base
racket/contract/base
rebellion/collection/sorted-map
rebellion/collection/sorted-set
resyntax/grimoire/source
resyntax/grimoire/syntax-movement
resyntax/grimoire/syntax-path)
scribble/example
(submod resyntax/private/scribble-evaluator-factory doc))


@(define make-evaluator
(make-module-sharing-evaluator-factory
#:public (list 'rebellion/collection/sorted-map
'resyntax/grimoire/source
'resyntax/grimoire/syntax-movement
'resyntax/grimoire/syntax-path)
#:private (list 'racket/base)))


@title[#:tag "syntax-movement"]{Syntax Movement Tables}
@defmodule[resyntax/grimoire/syntax-movement]

A @deftech{syntax movement table} records where each piece of a program's original syntax ended up
after the program was transformed, as a mapping between @tech{syntax paths}. The table's keys are
@tech{original syntax paths} --- the paths that subforms of the untransformed program were located
at when it was first read, as recorded by the @racket['original-syntax-path] property that
@racket[source-read-syntax] attaches. The table's values are sets of paths identifying every
location in the @emph{transformed} program where a subform claiming that original path can be
found. Resyntax primarily builds movement tables for fully expanded programs, in which case the
table describes where the macro expander moved each piece of the original program.

@; TODO(@notjack.space): broader overview goes here. Why movement tables exist: the role they play
@; in the analysis pipeline (translating expansion analyzer findings and expansion-time lexical
@; context back onto the original unexpanded program) and the role they're expected to play in the
@; pluggable whole-module analyzer system.

Each original path maps to a @emph{set} of paths in the transformed program, rather than to a
single path, because a single piece of original syntax can end up in several places at once ---
macros are free to copy their subforms. A @racket[struct] definition, for example, copies the
struct name into the several identifiers it defines, so the original path of the name maps to
every expanded location those copies ended up at. The reverse situation also occurs: original
syntax that the transformation discarded entirely appears nowhere in the transformed program, and
its path is simply absent from the table's keys.

@; TODO(@notjack.space): this one-to-many ambiguity is the crux of a lot of behavior worth
@; explaining in your own words: when several expanded forms claim the same original path,
@; Resyntax's analysis refuses to pick a winner, discarding expansion analyzer properties and
@; expansion-time lexical context for that path (the "multiple expanded forms claim to originate
@; from that path" log messages). Worth mentioning how that policy blocks things like issue #688
@; and what a more principled disambiguation might look like.

@; TODO(@notjack.space): possibly also worth hinting here: movement tables currently describe what
@; the macro expander did, but the same input-paths-to-output-paths shape describes what a
@; refactoring rule's transformation did. If tables are ever built for rule outputs, they'd be the
@; basis for automatically matching up input and output shape tags in UTS syntax deltas.


@defproc[(syntax-movement-table [result-stx syntax?]) immutable-sorted-map?]{
Builds a @tech{syntax movement table} for @racket[result-stx] by traversing it and collecting
every subform that carries an @tech{original syntax path}, including subforms nested within other
collected subforms. The returned table is an immutable sorted map whose keys are the original
syntax paths that were found, ordered by @racket[syntax-path<=>], and whose values are immutable
sorted sets of the paths within @racket[result-stx] at which the claiming subforms are located.

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded (source-expand (string-source "#lang racket/base\n(void)\n"))))
(syntax-movement-table expanded))

When the transformation copies a piece of original syntax into several places, every copy's
location appears in that original path's set:

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded
(source-expand (string-source "#lang racket/base\n(struct point (x y))\n")))
(define table (syntax-movement-table expanded)))
(code:comment "The original path of the identifier `point` within `(struct point (x y))`:")
(define point-path (syntax-path (list 3 1 1)))
(code:comment "Expansion copied that identifier into four different definitions.")
(sorted-map-get table point-path))}
2 changes: 1 addition & 1 deletion private/analysis.rkt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,7 +46,7 @@
resyntax/private/logger
resyntax/grimoire/source
resyntax/private/string-indent
resyntax/private/syntax-movement
resyntax/grimoire/syntax-movement
resyntax/private/syntax-neighbors
resyntax/grimoire/syntax-path
resyntax/grimoire/syntax-property-bundle
Expand Down
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Document syntax movement tables in the grimoire#833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
jackfirth wants to merge 2 commits into
base:master
Choose a base branch
from
Draft

Document syntax movement tables in the grimoire #833

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
25bab47
Move syntax-movement module into resyntax/grimoire
jackfirth Jul 17, 2026
dcbffc0
Draft API reference for syntax movement tables
jackfirth Jul 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions grimoire.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,5 +17,6 @@ programmatically on anything found here.
@include-section[(lib "resyntax/grimoire/syntax-path.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-property-bundle.scrbl")]
@include-section[(lib "resyntax/grimoire/expansion-analyzers.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-movement.scrbl")]
@include-section[(lib "resyntax/grimoire/string-replacement.scrbl")]
@include-section[(lib "resyntax/grimoire/linemap.scrbl")]
0 private/syntax-movement.rkt → grimoire/syntax-movement.rkt
View file
Open in desktop
File renamed without changes.
87 changes: 87 additions & 0 deletions grimoire/syntax-movement.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
#lang scribble/manual


@(require (for-label racket/base
racket/contract/base
rebellion/collection/sorted-map
rebellion/collection/sorted-set
resyntax/grimoire/source
resyntax/grimoire/syntax-movement
resyntax/grimoire/syntax-path)
scribble/example
(submod resyntax/private/scribble-evaluator-factory doc))


@(define make-evaluator
(make-module-sharing-evaluator-factory
#:public (list 'rebellion/collection/sorted-map
'resyntax/grimoire/source
'resyntax/grimoire/syntax-movement
'resyntax/grimoire/syntax-path)
#:private (list 'racket/base)))


@title[#:tag "syntax-movement"]{Syntax Movement Tables}
@defmodule[resyntax/grimoire/syntax-movement]

A @deftech{syntax movement table} records where each piece of a program's original syntax ended up
after the program was transformed, as a mapping between @tech{syntax paths}. The table's keys are
@tech{original syntax paths} --- the paths that subforms of the untransformed program were located
at when it was first read, as recorded by the @racket['original-syntax-path] property that
@racket[source-read-syntax] attaches. The table's values are sets of paths identifying every
location in the @emph{transformed} program where a subform claiming that original path can be
found. Resyntax primarily builds movement tables for fully expanded programs, in which case the
table describes where the macro expander moved each piece of the original program.

@; TODO(@notjack.space): broader overview goes here. Why movement tables exist: the role they play
@; in the analysis pipeline (translating expansion analyzer findings and expansion-time lexical
@; context back onto the original unexpanded program) and the role they're expected to play in the
@; pluggable whole-module analyzer system.

Each original path maps to a @emph{set} of paths in the transformed program, rather than to a
single path, because a single piece of original syntax can end up in several places at once ---
macros are free to copy their subforms. A @racket[struct] definition, for example, copies the
struct name into the several identifiers it defines, so the original path of the name maps to
every expanded location those copies ended up at. The reverse situation also occurs: original
syntax that the transformation discarded entirely appears nowhere in the transformed program, and
its path is simply absent from the table's keys.

@; TODO(@notjack.space): this one-to-many ambiguity is the crux of a lot of behavior worth
@; explaining in your own words: when several expanded forms claim the same original path,
@; Resyntax's analysis refuses to pick a winner, discarding expansion analyzer properties and
@; expansion-time lexical context for that path (the "multiple expanded forms claim to originate
@; from that path" log messages). Worth mentioning how that policy blocks things like issue #688
@; and what a more principled disambiguation might look like.

@; TODO(@notjack.space): possibly also worth hinting here: movement tables currently describe what
@; the macro expander did, but the same input-paths-to-output-paths shape describes what a
@; refactoring rule's transformation did. If tables are ever built for rule outputs, they'd be the
@; basis for automatically matching up input and output shape tags in UTS syntax deltas.


@defproc[(syntax-movement-table [result-stx syntax?]) immutable-sorted-map?]{
Builds a @tech{syntax movement table} for @racket[result-stx] by traversing it and collecting
every subform that carries an @tech{original syntax path}, including subforms nested within other
collected subforms. The returned table is an immutable sorted map whose keys are the original
syntax paths that were found, ordered by @racket[syntax-path<=>], and whose values are immutable
sorted sets of the paths within @racket[result-stx] at which the claiming subforms are located.

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded (source-expand (string-source "#lang racket/base\n(void)\n"))))
(syntax-movement-table expanded))

When the transformation copies a piece of original syntax into several places, every copy's
location appears in that original path's set:

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded
(source-expand (string-source "#lang racket/base\n(struct point (x y))\n")))
(define table (syntax-movement-table expanded)))
(code:comment "The original path of the identifier `point` within `(struct point (x y))`:")
(define point-path (syntax-path (list 3 1 1)))
(code:comment "Expansion copied that identifier into four different definitions.")
(sorted-map-get table point-path))}
2 changes: 1 addition & 1 deletion private/analysis.rkt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,7 +46,7 @@
resyntax/private/logger
resyntax/grimoire/source
resyntax/private/string-indent
resyntax/private/syntax-movement
resyntax/grimoire/syntax-movement
resyntax/private/syntax-neighbors
resyntax/grimoire/syntax-path
resyntax/grimoire/syntax-property-bundle
Expand Down
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Document syntax movement tables in the grimoire#833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
jackfirth wants to merge 2 commits into
base:master
Choose a base branch
from
Draft

Document syntax movement tables in the grimoire #833

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
25bab47
Move syntax-movement module into resyntax/grimoire
jackfirth Jul 17, 2026
dcbffc0
Draft API reference for syntax movement tables
jackfirth Jul 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions grimoire.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,5 +17,6 @@ programmatically on anything found here.
@include-section[(lib "resyntax/grimoire/syntax-path.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-property-bundle.scrbl")]
@include-section[(lib "resyntax/grimoire/expansion-analyzers.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-movement.scrbl")]
@include-section[(lib "resyntax/grimoire/string-replacement.scrbl")]
@include-section[(lib "resyntax/grimoire/linemap.scrbl")]
0 private/syntax-movement.rkt → grimoire/syntax-movement.rkt
View file
Open in desktop
File renamed without changes.
87 changes: 87 additions & 0 deletions grimoire/syntax-movement.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
#lang scribble/manual


@(require (for-label racket/base
racket/contract/base
rebellion/collection/sorted-map
rebellion/collection/sorted-set
resyntax/grimoire/source
resyntax/grimoire/syntax-movement
resyntax/grimoire/syntax-path)
scribble/example
(submod resyntax/private/scribble-evaluator-factory doc))


@(define make-evaluator
(make-module-sharing-evaluator-factory
#:public (list 'rebellion/collection/sorted-map
'resyntax/grimoire/source
'resyntax/grimoire/syntax-movement
'resyntax/grimoire/syntax-path)
#:private (list 'racket/base)))


@title[#:tag "syntax-movement"]{Syntax Movement Tables}
@defmodule[resyntax/grimoire/syntax-movement]

A @deftech{syntax movement table} records where each piece of a program's original syntax ended up
after the program was transformed, as a mapping between @tech{syntax paths}. The table's keys are
@tech{original syntax paths} --- the paths that subforms of the untransformed program were located
at when it was first read, as recorded by the @racket['original-syntax-path] property that
@racket[source-read-syntax] attaches. The table's values are sets of paths identifying every
location in the @emph{transformed} program where a subform claiming that original path can be
found. Resyntax primarily builds movement tables for fully expanded programs, in which case the
table describes where the macro expander moved each piece of the original program.

@; TODO(@notjack.space): broader overview goes here. Why movement tables exist: the role they play
@; in the analysis pipeline (translating expansion analyzer findings and expansion-time lexical
@; context back onto the original unexpanded program) and the role they're expected to play in the
@; pluggable whole-module analyzer system.

Each original path maps to a @emph{set} of paths in the transformed program, rather than to a
single path, because a single piece of original syntax can end up in several places at once ---
macros are free to copy their subforms. A @racket[struct] definition, for example, copies the
struct name into the several identifiers it defines, so the original path of the name maps to
every expanded location those copies ended up at. The reverse situation also occurs: original
syntax that the transformation discarded entirely appears nowhere in the transformed program, and
its path is simply absent from the table's keys.

@; TODO(@notjack.space): this one-to-many ambiguity is the crux of a lot of behavior worth
@; explaining in your own words: when several expanded forms claim the same original path,
@; Resyntax's analysis refuses to pick a winner, discarding expansion analyzer properties and
@; expansion-time lexical context for that path (the "multiple expanded forms claim to originate
@; from that path" log messages). Worth mentioning how that policy blocks things like issue #688
@; and what a more principled disambiguation might look like.

@; TODO(@notjack.space): possibly also worth hinting here: movement tables currently describe what
@; the macro expander did, but the same input-paths-to-output-paths shape describes what a
@; refactoring rule's transformation did. If tables are ever built for rule outputs, they'd be the
@; basis for automatically matching up input and output shape tags in UTS syntax deltas.


@defproc[(syntax-movement-table [result-stx syntax?]) immutable-sorted-map?]{
Builds a @tech{syntax movement table} for @racket[result-stx] by traversing it and collecting
every subform that carries an @tech{original syntax path}, including subforms nested within other
collected subforms. The returned table is an immutable sorted map whose keys are the original
syntax paths that were found, ordered by @racket[syntax-path<=>], and whose values are immutable
sorted sets of the paths within @racket[result-stx] at which the claiming subforms are located.

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded (source-expand (string-source "#lang racket/base\n(void)\n"))))
(syntax-movement-table expanded))

When the transformation copies a piece of original syntax into several places, every copy's
location appears in that original path's set:

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded
(source-expand (string-source "#lang racket/base\n(struct point (x y))\n")))
(define table (syntax-movement-table expanded)))
(code:comment "The original path of the identifier `point` within `(struct point (x y))`:")
(define point-path (syntax-path (list 3 1 1)))
(code:comment "Expansion copied that identifier into four different definitions.")
(sorted-map-get table point-path))}
2 changes: 1 addition & 1 deletion private/analysis.rkt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,7 +46,7 @@
resyntax/private/logger
resyntax/grimoire/source
resyntax/private/string-indent
resyntax/private/syntax-movement
resyntax/grimoire/syntax-movement
resyntax/private/syntax-neighbors
resyntax/grimoire/syntax-path
resyntax/grimoire/syntax-property-bundle
Expand Down
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Document syntax movement tables in the grimoire#833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
jackfirth wants to merge 2 commits into
base:master
Choose a base branch
from
Draft

Document syntax movement tables in the grimoire #833

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
25bab47
Move syntax-movement module into resyntax/grimoire
jackfirth Jul 17, 2026
dcbffc0
Draft API reference for syntax movement tables
jackfirth Jul 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions grimoire.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,5 +17,6 @@ programmatically on anything found here.
@include-section[(lib "resyntax/grimoire/syntax-path.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-property-bundle.scrbl")]
@include-section[(lib "resyntax/grimoire/expansion-analyzers.scrbl")]
@include-section[(lib "resyntax/grimoire/syntax-movement.scrbl")]
@include-section[(lib "resyntax/grimoire/string-replacement.scrbl")]
@include-section[(lib "resyntax/grimoire/linemap.scrbl")]
0 private/syntax-movement.rkt → grimoire/syntax-movement.rkt
View file
Open in desktop
File renamed without changes.
87 changes: 87 additions & 0 deletions grimoire/syntax-movement.scrbl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
#lang scribble/manual


@(require (for-label racket/base
racket/contract/base
rebellion/collection/sorted-map
rebellion/collection/sorted-set
resyntax/grimoire/source
resyntax/grimoire/syntax-movement
resyntax/grimoire/syntax-path)
scribble/example
(submod resyntax/private/scribble-evaluator-factory doc))


@(define make-evaluator
(make-module-sharing-evaluator-factory
#:public (list 'rebellion/collection/sorted-map
'resyntax/grimoire/source
'resyntax/grimoire/syntax-movement
'resyntax/grimoire/syntax-path)
#:private (list 'racket/base)))


@title[#:tag "syntax-movement"]{Syntax Movement Tables}
@defmodule[resyntax/grimoire/syntax-movement]

A @deftech{syntax movement table} records where each piece of a program's original syntax ended up
after the program was transformed, as a mapping between @tech{syntax paths}. The table's keys are
@tech{original syntax paths} --- the paths that subforms of the untransformed program were located
at when it was first read, as recorded by the @racket['original-syntax-path] property that
@racket[source-read-syntax] attaches. The table's values are sets of paths identifying every
location in the @emph{transformed} program where a subform claiming that original path can be
found. Resyntax primarily builds movement tables for fully expanded programs, in which case the
table describes where the macro expander moved each piece of the original program.

@; TODO(@notjack.space): broader overview goes here. Why movement tables exist: the role they play
@; in the analysis pipeline (translating expansion analyzer findings and expansion-time lexical
@; context back onto the original unexpanded program) and the role they're expected to play in the
@; pluggable whole-module analyzer system.

Each original path maps to a @emph{set} of paths in the transformed program, rather than to a
single path, because a single piece of original syntax can end up in several places at once ---
macros are free to copy their subforms. A @racket[struct] definition, for example, copies the
struct name into the several identifiers it defines, so the original path of the name maps to
every expanded location those copies ended up at. The reverse situation also occurs: original
syntax that the transformation discarded entirely appears nowhere in the transformed program, and
its path is simply absent from the table's keys.

@; TODO(@notjack.space): this one-to-many ambiguity is the crux of a lot of behavior worth
@; explaining in your own words: when several expanded forms claim the same original path,
@; Resyntax's analysis refuses to pick a winner, discarding expansion analyzer properties and
@; expansion-time lexical context for that path (the "multiple expanded forms claim to originate
@; from that path" log messages). Worth mentioning how that policy blocks things like issue #688
@; and what a more principled disambiguation might look like.

@; TODO(@notjack.space): possibly also worth hinting here: movement tables currently describe what
@; the macro expander did, but the same input-paths-to-output-paths shape describes what a
@; refactoring rule's transformation did. If tables are ever built for rule outputs, they'd be the
@; basis for automatically matching up input and output shape tags in UTS syntax deltas.


@defproc[(syntax-movement-table [result-stx syntax?]) immutable-sorted-map?]{
Builds a @tech{syntax movement table} for @racket[result-stx] by traversing it and collecting
every subform that carries an @tech{original syntax path}, including subforms nested within other
collected subforms. The returned table is an immutable sorted map whose keys are the original
syntax paths that were found, ordered by @racket[syntax-path<=>], and whose values are immutable
sorted sets of the paths within @racket[result-stx] at which the claiming subforms are located.

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded (source-expand (string-source "#lang racket/base\n(void)\n"))))
(syntax-movement-table expanded))

When the transformation copies a piece of original syntax into several places, every copy's
location appears in that original path's set:

@(examples
#:eval (make-evaluator) #:once
(eval:no-prompt
(define expanded
(source-expand (string-source "#lang racket/base\n(struct point (x y))\n")))
(define table (syntax-movement-table expanded)))
(code:comment "The original path of the identifier `point` within `(struct point (x y))`:")
(define point-path (syntax-path (list 3 1 1)))
(code:comment "Expansion copied that identifier into four different definitions.")
(sorted-map-get table point-path))}
2 changes: 1 addition & 1 deletion private/analysis.rkt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,7 +46,7 @@
resyntax/private/logger
resyntax/grimoire/source
resyntax/private/string-indent
resyntax/private/syntax-movement
resyntax/grimoire/syntax-movement
resyntax/private/syntax-neighbors
resyntax/grimoire/syntax-path
resyntax/grimoire/syntax-property-bundle
Expand Down
Loading