Skip to content

[5.x] Augmentation performance improvements - #9636

Merged
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation
May 2, 2024
Merged

[5.x] Augmentation performance improvements#9636
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation

Conversation

@JohnathonKoster

@JohnathonKosterJohnathonKoster commented Mar 2, 2024

Copy link
Copy Markdown
Contributor

This PR is an extension of #9574, and reworks some more internal augmentation behaviors. The changes in this PR come together to allow us to reuse fields arrays and augmentation keys as much as possible in a new BulkAugmentor. This new class will work to reduce the overhead of resolving this information as much as possible. For example, if you want to augment 1,000 entries and they can all safely reuse the same fields/keys, the BulkAugmentor will only do those calls once. This PR only focuses on pages/entries for now.

  • Delay execution of "normal" Value objects
  • If a user calls get directly, they shouldn't need to care about materialization
  • Introduce Transient values - these will be passed around until something actually needs a value. This makes InvokableValue and DeferredValue even more lazy 🥳
  • Rework some shared logic across deferred values
  • Rework augmentation selection to support passing in a known list of fields
  • Add support for "bulk augmentation". Augmentor 50000 ™️
    • Augmentable items need a way to tell us "what" they are in general terms. i.e., Page/entry for collection.posts.blueprint_a, or page/entry for collection.posts.blueprint_b
  • Rework nav tag tree builder to bulk augment
  • Rework collection tag to use transient values will just handle this Antlers-side to cause less chaos
  • Rework internal Antlers variable logic to use transient values
  • Rework dump tag and modifier to resolve transient/deferred values

@JohnathonKoster
JohnathonKoster marked this pull request as draft March 2, 2024 16:02
@JohnathonKosterJohnathonKoster changed the title [5.x] Deferred all augmentation[5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustmentsMar 2, 2024
* Allow for implementors to return `null`, which disables the bulk process for that item
* Entry now takes data keys into consideration to prevent "locking" an entry into something if it has custom data
@JohnathonKosterJohnathonKoster mentioned this pull request Mar 2, 2024
37 tasks
@JohnathonKoster
JohnathonKoster marked this pull request as ready for review March 2, 2024 21:25
jasonvarga
jasonvarga previously requested changes Mar 7, 2024

@jasonvargajasonvarga left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please target master.

@JohnathonKoster
JohnathonKoster changed the base branch from 4.x to masterMarch 8, 2024 01:07
@jasonvarga
jasonvarga dismissed their stale reviewMarch 11, 2024 16:55

Requested changes were made.

jasonvargaand others added 6 commits March 12, 2024 12:37
…some just because ...
- Dedicated AbstractAugmented 'wrap' methods, to avoid passing around hard to follow booleans.
- Avoid a hard to follow boolean in InvokableValue
- Make InvokableValue easier to follow both paths in general.
- TransientValue and DeferredValue now pass their references when calling shallow(), which were missing, and was the reason for needing to check for nulls.
- Typehints and visibility
… ...
If you pass a closure as the raw value, it'll lazily evaluate them when resolving.
materialize() just called resolve() and returned a new Value instance.
Now that the subclasses are gone, there's no need to new up a separate Value instance. It can just return itself. With that gone, we might as well just call resolve since materialize would just be an alias.
@jasonvargajasonvarga changed the title [5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustments[5.x] Augmentation performance improvementsMay 1, 2024
jasonvarga added 15 commits May 1, 2024 16:44
- Add static factory methods to BulkAugmentor.
- Rename BulkAugmentor augmented() to toArray(). Seemed more inline with what it did. Looks like it makes sense when compared to the toDeferredAugmentedArray in the "else" where its used in RuntimeValues.
- Reduce visibility where possible
Since dd(), dump(), etc can have variable arguments, avoid making our own Dumper methods. Just use the class to resolve the values and pass them into the original functions.
Rename resolveValues to resolve.
Simplify map.
Remove unnecessary guard.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@JohnathonKoster@jasonvarga
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
[5.x] Augmentation performance improvements by JohnathonKoster · Pull Request #9636 · statamic/cms · GitHub
Skip to content

[5.x] Augmentation performance improvements - #9636

Merged
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation
May 2, 2024
Merged

[5.x] Augmentation performance improvements#9636
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation

Conversation

@JohnathonKoster

@JohnathonKosterJohnathonKoster commented Mar 2, 2024

Copy link
Copy Markdown
Contributor

This PR is an extension of #9574, and reworks some more internal augmentation behaviors. The changes in this PR come together to allow us to reuse fields arrays and augmentation keys as much as possible in a new BulkAugmentor. This new class will work to reduce the overhead of resolving this information as much as possible. For example, if you want to augment 1,000 entries and they can all safely reuse the same fields/keys, the BulkAugmentor will only do those calls once. This PR only focuses on pages/entries for now.

  • Delay execution of "normal" Value objects
  • If a user calls get directly, they shouldn't need to care about materialization
  • Introduce Transient values - these will be passed around until something actually needs a value. This makes InvokableValue and DeferredValue even more lazy 🥳
  • Rework some shared logic across deferred values
  • Rework augmentation selection to support passing in a known list of fields
  • Add support for "bulk augmentation". Augmentor 50000 ™️
    • Augmentable items need a way to tell us "what" they are in general terms. i.e., Page/entry for collection.posts.blueprint_a, or page/entry for collection.posts.blueprint_b
  • Rework nav tag tree builder to bulk augment
  • Rework collection tag to use transient values will just handle this Antlers-side to cause less chaos
  • Rework internal Antlers variable logic to use transient values
  • Rework dump tag and modifier to resolve transient/deferred values

@JohnathonKoster
JohnathonKoster marked this pull request as draft March 2, 2024 16:02
@JohnathonKosterJohnathonKoster changed the title [5.x] Deferred all augmentation[5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustmentsMar 2, 2024
* Allow for implementors to return `null`, which disables the bulk process for that item
* Entry now takes data keys into consideration to prevent "locking" an entry into something if it has custom data
@JohnathonKosterJohnathonKoster mentioned this pull request Mar 2, 2024
37 tasks
@JohnathonKoster
JohnathonKoster marked this pull request as ready for review March 2, 2024 21:25
jasonvarga
jasonvarga previously requested changes Mar 7, 2024

@jasonvargajasonvarga left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please target master.

@JohnathonKoster
JohnathonKoster changed the base branch from 4.x to masterMarch 8, 2024 01:07
@jasonvarga
jasonvarga dismissed their stale reviewMarch 11, 2024 16:55

Requested changes were made.

jasonvargaand others added 6 commits March 12, 2024 12:37
…some just because ...
- Dedicated AbstractAugmented 'wrap' methods, to avoid passing around hard to follow booleans.
- Avoid a hard to follow boolean in InvokableValue
- Make InvokableValue easier to follow both paths in general.
- TransientValue and DeferredValue now pass their references when calling shallow(), which were missing, and was the reason for needing to check for nulls.
- Typehints and visibility
… ...
If you pass a closure as the raw value, it'll lazily evaluate them when resolving.
materialize() just called resolve() and returned a new Value instance.
Now that the subclasses are gone, there's no need to new up a separate Value instance. It can just return itself. With that gone, we might as well just call resolve since materialize would just be an alias.
@jasonvargajasonvarga changed the title [5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustments[5.x] Augmentation performance improvementsMay 1, 2024
jasonvarga added 15 commits May 1, 2024 16:44
- Add static factory methods to BulkAugmentor.
- Rename BulkAugmentor augmented() to toArray(). Seemed more inline with what it did. Looks like it makes sense when compared to the toDeferredAugmentedArray in the "else" where its used in RuntimeValues.
- Reduce visibility where possible
Since dd(), dump(), etc can have variable arguments, avoid making our own Dumper methods. Just use the class to resolve the values and pass them into the original functions.
Rename resolveValues to resolve.
Simplify map.
Remove unnecessary guard.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@JohnathonKoster@jasonvarga
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [5.x] Augmentation performance improvements by JohnathonKoster · Pull Request #9636 · statamic/cms · GitHub
Skip to content

[5.x] Augmentation performance improvements - #9636

Merged
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation
May 2, 2024
Merged

[5.x] Augmentation performance improvements#9636
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation

Conversation

@JohnathonKoster

@JohnathonKosterJohnathonKoster commented Mar 2, 2024

Copy link
Copy Markdown
Contributor

This PR is an extension of #9574, and reworks some more internal augmentation behaviors. The changes in this PR come together to allow us to reuse fields arrays and augmentation keys as much as possible in a new BulkAugmentor. This new class will work to reduce the overhead of resolving this information as much as possible. For example, if you want to augment 1,000 entries and they can all safely reuse the same fields/keys, the BulkAugmentor will only do those calls once. This PR only focuses on pages/entries for now.

  • Delay execution of "normal" Value objects
  • If a user calls get directly, they shouldn't need to care about materialization
  • Introduce Transient values - these will be passed around until something actually needs a value. This makes InvokableValue and DeferredValue even more lazy 🥳
  • Rework some shared logic across deferred values
  • Rework augmentation selection to support passing in a known list of fields
  • Add support for "bulk augmentation". Augmentor 50000 ™️
    • Augmentable items need a way to tell us "what" they are in general terms. i.e., Page/entry for collection.posts.blueprint_a, or page/entry for collection.posts.blueprint_b
  • Rework nav tag tree builder to bulk augment
  • Rework collection tag to use transient values will just handle this Antlers-side to cause less chaos
  • Rework internal Antlers variable logic to use transient values
  • Rework dump tag and modifier to resolve transient/deferred values

@JohnathonKoster
JohnathonKoster marked this pull request as draft March 2, 2024 16:02
@JohnathonKosterJohnathonKoster changed the title [5.x] Deferred all augmentation[5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustmentsMar 2, 2024
* Allow for implementors to return `null`, which disables the bulk process for that item
* Entry now takes data keys into consideration to prevent "locking" an entry into something if it has custom data
@JohnathonKosterJohnathonKoster mentioned this pull request Mar 2, 2024
37 tasks
@JohnathonKoster
JohnathonKoster marked this pull request as ready for review March 2, 2024 21:25
jasonvarga
jasonvarga previously requested changes Mar 7, 2024

@jasonvargajasonvarga left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please target master.

@JohnathonKoster
JohnathonKoster changed the base branch from 4.x to masterMarch 8, 2024 01:07
@jasonvarga
jasonvarga dismissed their stale reviewMarch 11, 2024 16:55

Requested changes were made.

jasonvargaand others added 6 commits March 12, 2024 12:37
…some just because ...
- Dedicated AbstractAugmented 'wrap' methods, to avoid passing around hard to follow booleans.
- Avoid a hard to follow boolean in InvokableValue
- Make InvokableValue easier to follow both paths in general.
- TransientValue and DeferredValue now pass their references when calling shallow(), which were missing, and was the reason for needing to check for nulls.
- Typehints and visibility
… ...
If you pass a closure as the raw value, it'll lazily evaluate them when resolving.
materialize() just called resolve() and returned a new Value instance.
Now that the subclasses are gone, there's no need to new up a separate Value instance. It can just return itself. With that gone, we might as well just call resolve since materialize would just be an alias.
@jasonvargajasonvarga changed the title [5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustments[5.x] Augmentation performance improvementsMay 1, 2024
jasonvarga added 15 commits May 1, 2024 16:44
- Add static factory methods to BulkAugmentor.
- Rename BulkAugmentor augmented() to toArray(). Seemed more inline with what it did. Looks like it makes sense when compared to the toDeferredAugmentedArray in the "else" where its used in RuntimeValues.
- Reduce visibility where possible
Since dd(), dump(), etc can have variable arguments, avoid making our own Dumper methods. Just use the class to resolve the values and pass them into the original functions.
Rename resolveValues to resolve.
Simplify map.
Remove unnecessary guard.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@JohnathonKoster@jasonvarga
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [5.x] Augmentation performance improvements by JohnathonKoster · Pull Request #9636 · statamic/cms · GitHub
Skip to content

[5.x] Augmentation performance improvements - #9636

Merged
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation
May 2, 2024
Merged

[5.x] Augmentation performance improvements#9636
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation

Conversation

@JohnathonKoster

@JohnathonKosterJohnathonKoster commented Mar 2, 2024

Copy link
Copy Markdown
Contributor

This PR is an extension of #9574, and reworks some more internal augmentation behaviors. The changes in this PR come together to allow us to reuse fields arrays and augmentation keys as much as possible in a new BulkAugmentor. This new class will work to reduce the overhead of resolving this information as much as possible. For example, if you want to augment 1,000 entries and they can all safely reuse the same fields/keys, the BulkAugmentor will only do those calls once. This PR only focuses on pages/entries for now.

  • Delay execution of "normal" Value objects
  • If a user calls get directly, they shouldn't need to care about materialization
  • Introduce Transient values - these will be passed around until something actually needs a value. This makes InvokableValue and DeferredValue even more lazy 🥳
  • Rework some shared logic across deferred values
  • Rework augmentation selection to support passing in a known list of fields
  • Add support for "bulk augmentation". Augmentor 50000 ™️
    • Augmentable items need a way to tell us "what" they are in general terms. i.e., Page/entry for collection.posts.blueprint_a, or page/entry for collection.posts.blueprint_b
  • Rework nav tag tree builder to bulk augment
  • Rework collection tag to use transient values will just handle this Antlers-side to cause less chaos
  • Rework internal Antlers variable logic to use transient values
  • Rework dump tag and modifier to resolve transient/deferred values

@JohnathonKoster
JohnathonKoster marked this pull request as draft March 2, 2024 16:02
@JohnathonKosterJohnathonKoster changed the title [5.x] Deferred all augmentation[5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustmentsMar 2, 2024
* Allow for implementors to return `null`, which disables the bulk process for that item
* Entry now takes data keys into consideration to prevent "locking" an entry into something if it has custom data
@JohnathonKosterJohnathonKoster mentioned this pull request Mar 2, 2024
37 tasks
@JohnathonKoster
JohnathonKoster marked this pull request as ready for review March 2, 2024 21:25
jasonvarga
jasonvarga previously requested changes Mar 7, 2024

@jasonvargajasonvarga left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please target master.

@JohnathonKoster
JohnathonKoster changed the base branch from 4.x to masterMarch 8, 2024 01:07
@jasonvarga
jasonvarga dismissed their stale reviewMarch 11, 2024 16:55

Requested changes were made.

jasonvargaand others added 6 commits March 12, 2024 12:37
…some just because ...
- Dedicated AbstractAugmented 'wrap' methods, to avoid passing around hard to follow booleans.
- Avoid a hard to follow boolean in InvokableValue
- Make InvokableValue easier to follow both paths in general.
- TransientValue and DeferredValue now pass their references when calling shallow(), which were missing, and was the reason for needing to check for nulls.
- Typehints and visibility
… ...
If you pass a closure as the raw value, it'll lazily evaluate them when resolving.
materialize() just called resolve() and returned a new Value instance.
Now that the subclasses are gone, there's no need to new up a separate Value instance. It can just return itself. With that gone, we might as well just call resolve since materialize would just be an alias.
@jasonvargajasonvarga changed the title [5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustments[5.x] Augmentation performance improvementsMay 1, 2024
jasonvarga added 15 commits May 1, 2024 16:44
- Add static factory methods to BulkAugmentor.
- Rename BulkAugmentor augmented() to toArray(). Seemed more inline with what it did. Looks like it makes sense when compared to the toDeferredAugmentedArray in the "else" where its used in RuntimeValues.
- Reduce visibility where possible
Since dd(), dump(), etc can have variable arguments, avoid making our own Dumper methods. Just use the class to resolve the values and pass them into the original functions.
Rename resolveValues to resolve.
Simplify map.
Remove unnecessary guard.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@JohnathonKoster@jasonvarga
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' [5.x] Augmentation performance improvements by JohnathonKoster · Pull Request #9636 · statamic/cms · GitHub
Skip to content

[5.x] Augmentation performance improvements - #9636

Merged
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation
May 2, 2024
Merged

[5.x] Augmentation performance improvements#9636
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation

Conversation

@JohnathonKoster

@JohnathonKosterJohnathonKoster commented Mar 2, 2024

Copy link
Copy Markdown
Contributor

This PR is an extension of #9574, and reworks some more internal augmentation behaviors. The changes in this PR come together to allow us to reuse fields arrays and augmentation keys as much as possible in a new BulkAugmentor. This new class will work to reduce the overhead of resolving this information as much as possible. For example, if you want to augment 1,000 entries and they can all safely reuse the same fields/keys, the BulkAugmentor will only do those calls once. This PR only focuses on pages/entries for now.

  • Delay execution of "normal" Value objects
  • If a user calls get directly, they shouldn't need to care about materialization
  • Introduce Transient values - these will be passed around until something actually needs a value. This makes InvokableValue and DeferredValue even more lazy 🥳
  • Rework some shared logic across deferred values
  • Rework augmentation selection to support passing in a known list of fields
  • Add support for "bulk augmentation". Augmentor 50000 ™️
    • Augmentable items need a way to tell us "what" they are in general terms. i.e., Page/entry for collection.posts.blueprint_a, or page/entry for collection.posts.blueprint_b
  • Rework nav tag tree builder to bulk augment
  • Rework collection tag to use transient values will just handle this Antlers-side to cause less chaos
  • Rework internal Antlers variable logic to use transient values
  • Rework dump tag and modifier to resolve transient/deferred values

@JohnathonKoster
JohnathonKoster marked this pull request as draft March 2, 2024 16:02
@JohnathonKosterJohnathonKoster changed the title [5.x] Deferred all augmentation[5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustmentsMar 2, 2024
* Allow for implementors to return `null`, which disables the bulk process for that item
* Entry now takes data keys into consideration to prevent "locking" an entry into something if it has custom data
@JohnathonKosterJohnathonKoster mentioned this pull request Mar 2, 2024
37 tasks
@JohnathonKoster
JohnathonKoster marked this pull request as ready for review March 2, 2024 21:25
jasonvarga
jasonvarga previously requested changes Mar 7, 2024

@jasonvargajasonvarga left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please target master.

@JohnathonKoster
JohnathonKoster changed the base branch from 4.x to masterMarch 8, 2024 01:07
@jasonvarga
jasonvarga dismissed their stale reviewMarch 11, 2024 16:55

Requested changes were made.

jasonvargaand others added 6 commits March 12, 2024 12:37
…some just because ...
- Dedicated AbstractAugmented 'wrap' methods, to avoid passing around hard to follow booleans.
- Avoid a hard to follow boolean in InvokableValue
- Make InvokableValue easier to follow both paths in general.
- TransientValue and DeferredValue now pass their references when calling shallow(), which were missing, and was the reason for needing to check for nulls.
- Typehints and visibility
… ...
If you pass a closure as the raw value, it'll lazily evaluate them when resolving.
materialize() just called resolve() and returned a new Value instance.
Now that the subclasses are gone, there's no need to new up a separate Value instance. It can just return itself. With that gone, we might as well just call resolve since materialize would just be an alias.
@jasonvargajasonvarga changed the title [5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustments[5.x] Augmentation performance improvementsMay 1, 2024
jasonvarga added 15 commits May 1, 2024 16:44
- Add static factory methods to BulkAugmentor.
- Rename BulkAugmentor augmented() to toArray(). Seemed more inline with what it did. Looks like it makes sense when compared to the toDeferredAugmentedArray in the "else" where its used in RuntimeValues.
- Reduce visibility where possible
Since dd(), dump(), etc can have variable arguments, avoid making our own Dumper methods. Just use the class to resolve the values and pass them into the original functions.
Rename resolveValues to resolve.
Simplify map.
Remove unnecessary guard.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@JohnathonKoster@jasonvarga
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [5.x] Augmentation performance improvements by JohnathonKoster · Pull Request #9636 · statamic/cms · GitHub
Skip to content

[5.x] Augmentation performance improvements - #9636

Merged
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation
May 2, 2024
Merged

[5.x] Augmentation performance improvements#9636
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation

Conversation

@JohnathonKoster

@JohnathonKosterJohnathonKoster commented Mar 2, 2024

Copy link
Copy Markdown
Contributor

This PR is an extension of #9574, and reworks some more internal augmentation behaviors. The changes in this PR come together to allow us to reuse fields arrays and augmentation keys as much as possible in a new BulkAugmentor. This new class will work to reduce the overhead of resolving this information as much as possible. For example, if you want to augment 1,000 entries and they can all safely reuse the same fields/keys, the BulkAugmentor will only do those calls once. This PR only focuses on pages/entries for now.

  • Delay execution of "normal" Value objects
  • If a user calls get directly, they shouldn't need to care about materialization
  • Introduce Transient values - these will be passed around until something actually needs a value. This makes InvokableValue and DeferredValue even more lazy 🥳
  • Rework some shared logic across deferred values
  • Rework augmentation selection to support passing in a known list of fields
  • Add support for "bulk augmentation". Augmentor 50000 ™️
    • Augmentable items need a way to tell us "what" they are in general terms. i.e., Page/entry for collection.posts.blueprint_a, or page/entry for collection.posts.blueprint_b
  • Rework nav tag tree builder to bulk augment
  • Rework collection tag to use transient values will just handle this Antlers-side to cause less chaos
  • Rework internal Antlers variable logic to use transient values
  • Rework dump tag and modifier to resolve transient/deferred values

@JohnathonKoster
JohnathonKoster marked this pull request as draft March 2, 2024 16:02
@JohnathonKosterJohnathonKoster changed the title [5.x] Deferred all augmentation[5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustmentsMar 2, 2024
* Allow for implementors to return `null`, which disables the bulk process for that item
* Entry now takes data keys into consideration to prevent "locking" an entry into something if it has custom data
@JohnathonKosterJohnathonKoster mentioned this pull request Mar 2, 2024
37 tasks
@JohnathonKoster
JohnathonKoster marked this pull request as ready for review March 2, 2024 21:25
jasonvarga
jasonvarga previously requested changes Mar 7, 2024

@jasonvargajasonvarga left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please target master.

@JohnathonKoster
JohnathonKoster changed the base branch from 4.x to masterMarch 8, 2024 01:07
@jasonvarga
jasonvarga dismissed their stale reviewMarch 11, 2024 16:55

Requested changes were made.

jasonvargaand others added 6 commits March 12, 2024 12:37
…some just because ...
- Dedicated AbstractAugmented 'wrap' methods, to avoid passing around hard to follow booleans.
- Avoid a hard to follow boolean in InvokableValue
- Make InvokableValue easier to follow both paths in general.
- TransientValue and DeferredValue now pass their references when calling shallow(), which were missing, and was the reason for needing to check for nulls.
- Typehints and visibility
… ...
If you pass a closure as the raw value, it'll lazily evaluate them when resolving.
materialize() just called resolve() and returned a new Value instance.
Now that the subclasses are gone, there's no need to new up a separate Value instance. It can just return itself. With that gone, we might as well just call resolve since materialize would just be an alias.
@jasonvargajasonvarga changed the title [5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustments[5.x] Augmentation performance improvementsMay 1, 2024
jasonvarga added 15 commits May 1, 2024 16:44
- Add static factory methods to BulkAugmentor.
- Rename BulkAugmentor augmented() to toArray(). Seemed more inline with what it did. Looks like it makes sense when compared to the toDeferredAugmentedArray in the "else" where its used in RuntimeValues.
- Reduce visibility where possible
Since dd(), dump(), etc can have variable arguments, avoid making our own Dumper methods. Just use the class to resolve the values and pass them into the original functions.
Rename resolveValues to resolve.
Simplify map.
Remove unnecessary guard.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@JohnathonKoster@jasonvarga
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [5.x] Augmentation performance improvements by JohnathonKoster · Pull Request #9636 · statamic/cms · GitHub
Skip to content

[5.x] Augmentation performance improvements - #9636

Merged
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation
May 2, 2024
Merged

[5.x] Augmentation performance improvements#9636
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation

Conversation

@JohnathonKoster

@JohnathonKosterJohnathonKoster commented Mar 2, 2024

Copy link
Copy Markdown
Contributor

This PR is an extension of #9574, and reworks some more internal augmentation behaviors. The changes in this PR come together to allow us to reuse fields arrays and augmentation keys as much as possible in a new BulkAugmentor. This new class will work to reduce the overhead of resolving this information as much as possible. For example, if you want to augment 1,000 entries and they can all safely reuse the same fields/keys, the BulkAugmentor will only do those calls once. This PR only focuses on pages/entries for now.

  • Delay execution of "normal" Value objects
  • If a user calls get directly, they shouldn't need to care about materialization
  • Introduce Transient values - these will be passed around until something actually needs a value. This makes InvokableValue and DeferredValue even more lazy 🥳
  • Rework some shared logic across deferred values
  • Rework augmentation selection to support passing in a known list of fields
  • Add support for "bulk augmentation". Augmentor 50000 ™️
    • Augmentable items need a way to tell us "what" they are in general terms. i.e., Page/entry for collection.posts.blueprint_a, or page/entry for collection.posts.blueprint_b
  • Rework nav tag tree builder to bulk augment
  • Rework collection tag to use transient values will just handle this Antlers-side to cause less chaos
  • Rework internal Antlers variable logic to use transient values
  • Rework dump tag and modifier to resolve transient/deferred values

@JohnathonKoster
JohnathonKoster marked this pull request as draft March 2, 2024 16:02
@JohnathonKosterJohnathonKoster changed the title [5.x] Deferred all augmentation[5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustmentsMar 2, 2024
* Allow for implementors to return `null`, which disables the bulk process for that item
* Entry now takes data keys into consideration to prevent "locking" an entry into something if it has custom data
@JohnathonKosterJohnathonKoster mentioned this pull request Mar 2, 2024
37 tasks
@JohnathonKoster
JohnathonKoster marked this pull request as ready for review March 2, 2024 21:25
jasonvarga
jasonvarga previously requested changes Mar 7, 2024

@jasonvargajasonvarga left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please target master.

@JohnathonKoster
JohnathonKoster changed the base branch from 4.x to masterMarch 8, 2024 01:07
@jasonvarga
jasonvarga dismissed their stale reviewMarch 11, 2024 16:55

Requested changes were made.

jasonvargaand others added 6 commits March 12, 2024 12:37
…some just because ...
- Dedicated AbstractAugmented 'wrap' methods, to avoid passing around hard to follow booleans.
- Avoid a hard to follow boolean in InvokableValue
- Make InvokableValue easier to follow both paths in general.
- TransientValue and DeferredValue now pass their references when calling shallow(), which were missing, and was the reason for needing to check for nulls.
- Typehints and visibility
… ...
If you pass a closure as the raw value, it'll lazily evaluate them when resolving.
materialize() just called resolve() and returned a new Value instance.
Now that the subclasses are gone, there's no need to new up a separate Value instance. It can just return itself. With that gone, we might as well just call resolve since materialize would just be an alias.
@jasonvargajasonvarga changed the title [5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustments[5.x] Augmentation performance improvementsMay 1, 2024
jasonvarga added 15 commits May 1, 2024 16:44
- Add static factory methods to BulkAugmentor.
- Rename BulkAugmentor augmented() to toArray(). Seemed more inline with what it did. Looks like it makes sense when compared to the toDeferredAugmentedArray in the "else" where its used in RuntimeValues.
- Reduce visibility where possible
Since dd(), dump(), etc can have variable arguments, avoid making our own Dumper methods. Just use the class to resolve the values and pass them into the original functions.
Rename resolveValues to resolve.
Simplify map.
Remove unnecessary guard.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@JohnathonKoster@jasonvarga
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); [5.x] Augmentation performance improvements by JohnathonKoster · Pull Request #9636 · statamic/cms · GitHub
Skip to content

[5.x] Augmentation performance improvements - #9636

Merged
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation
May 2, 2024
Merged

[5.x] Augmentation performance improvements#9636
jasonvarga merged 45 commits into
statamic:masterfrom
JohnathonKoster:deferred-all-augmentation

Conversation

@JohnathonKoster

@JohnathonKosterJohnathonKoster commented Mar 2, 2024

Copy link
Copy Markdown
Contributor

This PR is an extension of #9574, and reworks some more internal augmentation behaviors. The changes in this PR come together to allow us to reuse fields arrays and augmentation keys as much as possible in a new BulkAugmentor. This new class will work to reduce the overhead of resolving this information as much as possible. For example, if you want to augment 1,000 entries and they can all safely reuse the same fields/keys, the BulkAugmentor will only do those calls once. This PR only focuses on pages/entries for now.

  • Delay execution of "normal" Value objects
  • If a user calls get directly, they shouldn't need to care about materialization
  • Introduce Transient values - these will be passed around until something actually needs a value. This makes InvokableValue and DeferredValue even more lazy 🥳
  • Rework some shared logic across deferred values
  • Rework augmentation selection to support passing in a known list of fields
  • Add support for "bulk augmentation". Augmentor 50000 ™️
    • Augmentable items need a way to tell us "what" they are in general terms. i.e., Page/entry for collection.posts.blueprint_a, or page/entry for collection.posts.blueprint_b
  • Rework nav tag tree builder to bulk augment
  • Rework collection tag to use transient values will just handle this Antlers-side to cause less chaos
  • Rework internal Antlers variable logic to use transient values
  • Rework dump tag and modifier to resolve transient/deferred values

@JohnathonKoster
JohnathonKoster marked this pull request as draft March 2, 2024 16:02
@JohnathonKosterJohnathonKoster changed the title [5.x] Deferred all augmentation[5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustmentsMar 2, 2024
* Allow for implementors to return `null`, which disables the bulk process for that item
* Entry now takes data keys into consideration to prevent "locking" an entry into something if it has custom data
@JohnathonKosterJohnathonKoster mentioned this pull request Mar 2, 2024
37 tasks
@JohnathonKoster
JohnathonKoster marked this pull request as ready for review March 2, 2024 21:25
jasonvarga
jasonvarga previously requested changes Mar 7, 2024

@jasonvargajasonvarga left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please target master.

@JohnathonKoster
JohnathonKoster changed the base branch from 4.x to masterMarch 8, 2024 01:07
@jasonvarga
jasonvarga dismissed their stale reviewMarch 11, 2024 16:55

Requested changes were made.

jasonvargaand others added 6 commits March 12, 2024 12:37
…some just because ...
- Dedicated AbstractAugmented 'wrap' methods, to avoid passing around hard to follow booleans.
- Avoid a hard to follow boolean in InvokableValue
- Make InvokableValue easier to follow both paths in general.
- TransientValue and DeferredValue now pass their references when calling shallow(), which were missing, and was the reason for needing to check for nulls.
- Typehints and visibility
… ...
If you pass a closure as the raw value, it'll lazily evaluate them when resolving.
materialize() just called resolve() and returned a new Value instance.
Now that the subclasses are gone, there's no need to new up a separate Value instance. It can just return itself. With that gone, we might as well just call resolve since materialize would just be an alias.
@jasonvargajasonvarga changed the title [5.x] Defer all augmentation, allow for Bulk Augmentation (reuse key/fields across similar items), even more internal augmentation adjustments[5.x] Augmentation performance improvementsMay 1, 2024
jasonvarga added 15 commits May 1, 2024 16:44
- Add static factory methods to BulkAugmentor.
- Rename BulkAugmentor augmented() to toArray(). Seemed more inline with what it did. Looks like it makes sense when compared to the toDeferredAugmentedArray in the "else" where its used in RuntimeValues.
- Reduce visibility where possible
Since dd(), dump(), etc can have variable arguments, avoid making our own Dumper methods. Just use the class to resolve the values and pass them into the original functions.
Rename resolveValues to resolve.
Simplify map.
Remove unnecessary guard.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants

@JohnathonKoster@jasonvarga