Skip to content

Fix nunjucks path resolving - #1263

Merged
ang-zeyu merged 2 commits into
MarkBind:masterfrom
ang-zeyu:nunjucks-per-site
Jul 25, 2020
Merged

Fix nunjucks path resolving#1263
ang-zeyu merged 2 commits into
MarkBind:masterfrom
ang-zeyu:nunjucks-per-site

Conversation

@ang-zeyu

Copy link
Copy Markdown
Contributor

What is the purpose of this pull request? (put "X" next to an item, remove the rest)

• [x] Bug fix

Fixes#931

What is the rationale for this request?

What changes did you make? (Give an overview)
To be filled

Testing instructions:

  • npm run test should pass
  • nunjucks paths should now resolve from the closest parent (sub)site ( tests added )

Proposed commit message: (wrap lines at 72 characters)
Fix nunjucks path resolving

Nunjucks' features that use paths resolve from the specified template
directory by design.
In MarkBind's case, the template directories are the respective root
paths of the various sites and subsites of the project.

Hence, a separate nunjucks environment with the template directory
configured as such is needed for each site or subsite.

Let's introduce a wrapper abstraction for this, VariableRenderer, to
wrap over such a singular nunjucks environment.
The VariableRenderer instances will be managed by VariableProcessor
(previously VariablePreprocessor), which already has a framework set up
to render variables belonging to each site or subsite.

In addition, {% raw %} tags inside a layouts file do not work due to
two nunjucks passes being made.
Let's revert the removal of the helper functions to keep {% raw %} tags
in the first pass to fix this, while searching for a solution that
would allow a single pass for layouts.

@damithc

Copy link
Copy Markdown
Contributor

This will be a big step forward if we manage to pull it off. 👍
I believe sub-site support is one of our unique features and this was a vital missing piece in that feature.

@ang-zeyu

Copy link
Copy Markdown
ContributorAuthor

It's pretty much ready, just need to update some more test files. I'm putting it on hold as we are doing quite a bit of directory restructuring as suggested by @acjh here #1253 (comment). Would be easier to review then.

I believe sub-site support is one of our unique features and this was a vital missing piece in that feature.

The next step would be layouts such as header/footer/expressive layouts and such, I'm not sure if the current behaviour is intended though (layout files from the root site can currently be used in subsites, unlike variables)
And then we have other things like respecting configuration of individual site.jsons (not sure if the current behaviour is intended as well though).

All of this should be able to be easily achieved by building the sites individually (the work so far still being necessary even if so though), and "blocking parent sites from building sub site pages".

we should probably flesh out the requirements for sub sites first in an issue or document the current behaviour in ug (if intended)

@ang-zeyuang-zeyu changed the title [WIP] Fix nunjucks path resolvingFix nunjucks path resolvingJul 22, 2020

@marvinchinmarvinchin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine to me! Just to check - I assume this has been tested on sites with nunjucks (2103T maybe?)? I don't see anything that would change existing working behaviour, but just want to be sure 🙂

Just a suggestion regarding structuring commits - I think it would be nice to have renaming variablePreprocessor -> variableProcessor to be in its own commit. This way the reviewer can filter the commits to focus only on the material changes.

Nunjucks' features that use paths resolve from the specified template
directory by design.
In MarkBind's case, the template directories are the respective root
paths of the various sites and subsites of the project.
Hence, a separate nunjucks environment with the template directory
configured as such is needed for each site or subsite.
Let's introduce a wrapper abstraction for this, VariableRenderer, to
wrap over such a singular nunjucks environment.
The VariableRenderer instances will be managed by VariableProcessor
(previously VariablePreprocessor), which already has a framework set up
to render variables belonging to each site or subsite.
In addition, {% raw %} tags inside a layouts file do not work due to
two nunjucks passes being made.
Let's revert the removal of the helper functions to keep {% raw %} tags
in the first pass to fix this, while searching for a solution that
would allow a single pass for layouts.
@ang-zeyu

Copy link
Copy Markdown
ContributorAuthor

This looks fine to me! Just to check - I assume this has been tested on sites with nunjucks (2103T maybe?)? I don't see anything that would change existing working behaviour, but just want to be sure 🙂

yup!

Just a suggestion regarding structuring commits - I think it would be nice to have renaming variablePreprocessor -> variableProcessor to be in its own commit. This way the reviewer can filter the commits to focus only on the material changes.

That makes sense, my bad 😅 I reseparated it - although there's some renaming in the first commit as well. This should be tremendously easier to look at in the history still though.

I also moved the variable rendering portion in includeFile out of it so there's no need to pass the 2 intermediate parameters (additionalVariables and keepPercentRaw) to includeFile which shouldn't know about it.

Also more clearly separates the variable rendering from the "including" part of the process. (retested with 2103 site again too)

Render {{ MAIN_CONTENT_BODY }} and {% raw/endraw %} back to itself first,
which is then dealt with in the call below to {@link renderSiteVariables}.
*/
.then(result => this.variableProcessor.renderPage(layoutPagePath, result, {

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

variable rendering is moved out of includeFile, also in page.generate() / resolveDependency()

@ang-zeyuang-zeyu added this to the v2.15.0 milestone Jul 24, 2020
* @param content to render
* @param highestPriorityVariables to render with the highest priority if any.
* This is currently only used for the MAIN_CONTENT_BODY in layouts.
* @param keepPercentRaw whether to reoutput {% raw/endraw %} tags, also used only for layouts.

@marvinchinmarvinchinJul 25, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that variable rendering is moved out of include file, it seems clearer that there is some specific behaviour for layouts. Do you think it's worth having a renderLayout function instead of using a general purpose renderPage?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that makes sense! I think there's quite a bit of opportunity for refactor regarding layouts though, apart from this. (site nav, page layouts, header, footer takes up a sizeable portion of Page.js) Perhaps we could defer it to that pr?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds fine too, let's fix that in the next PR then 🚀

@ang-zeyu
ang-zeyu merged commit 82c5ff6 into MarkBind:masterJul 25, 2020
@ang-zeyuang-zeyu mentioned this pull request Aug 2, 2020
13 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot use nunjucks in a sub-site

3 participants

@ang-zeyu@damithc@marvinchin
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Fix nunjucks path resolving by ang-zeyu · Pull Request #1263 · MarkBind/markbind · GitHub
Skip to content

Fix nunjucks path resolving - #1263

Merged
ang-zeyu merged 2 commits into
MarkBind:masterfrom
ang-zeyu:nunjucks-per-site
Jul 25, 2020
Merged

Fix nunjucks path resolving#1263
ang-zeyu merged 2 commits into
MarkBind:masterfrom
ang-zeyu:nunjucks-per-site

Conversation

@ang-zeyu

Copy link
Copy Markdown
Contributor

What is the purpose of this pull request? (put "X" next to an item, remove the rest)

• [x] Bug fix

Fixes#931

What is the rationale for this request?

What changes did you make? (Give an overview)
To be filled

Testing instructions:

  • npm run test should pass
  • nunjucks paths should now resolve from the closest parent (sub)site ( tests added )

Proposed commit message: (wrap lines at 72 characters)
Fix nunjucks path resolving

Nunjucks' features that use paths resolve from the specified template
directory by design.
In MarkBind's case, the template directories are the respective root
paths of the various sites and subsites of the project.

Hence, a separate nunjucks environment with the template directory
configured as such is needed for each site or subsite.

Let's introduce a wrapper abstraction for this, VariableRenderer, to
wrap over such a singular nunjucks environment.
The VariableRenderer instances will be managed by VariableProcessor
(previously VariablePreprocessor), which already has a framework set up
to render variables belonging to each site or subsite.

In addition, {% raw %} tags inside a layouts file do not work due to
two nunjucks passes being made.
Let's revert the removal of the helper functions to keep {% raw %} tags
in the first pass to fix this, while searching for a solution that
would allow a single pass for layouts.

@damithc

Copy link
Copy Markdown
Contributor

This will be a big step forward if we manage to pull it off. 👍
I believe sub-site support is one of our unique features and this was a vital missing piece in that feature.

@ang-zeyu

Copy link
Copy Markdown
ContributorAuthor

It's pretty much ready, just need to update some more test files. I'm putting it on hold as we are doing quite a bit of directory restructuring as suggested by @acjh here #1253 (comment). Would be easier to review then.

I believe sub-site support is one of our unique features and this was a vital missing piece in that feature.

The next step would be layouts such as header/footer/expressive layouts and such, I'm not sure if the current behaviour is intended though (layout files from the root site can currently be used in subsites, unlike variables)
And then we have other things like respecting configuration of individual site.jsons (not sure if the current behaviour is intended as well though).

All of this should be able to be easily achieved by building the sites individually (the work so far still being necessary even if so though), and "blocking parent sites from building sub site pages".

we should probably flesh out the requirements for sub sites first in an issue or document the current behaviour in ug (if intended)

@ang-zeyuang-zeyu changed the title [WIP] Fix nunjucks path resolvingFix nunjucks path resolvingJul 22, 2020

@marvinchinmarvinchin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine to me! Just to check - I assume this has been tested on sites with nunjucks (2103T maybe?)? I don't see anything that would change existing working behaviour, but just want to be sure 🙂

Just a suggestion regarding structuring commits - I think it would be nice to have renaming variablePreprocessor -> variableProcessor to be in its own commit. This way the reviewer can filter the commits to focus only on the material changes.

Nunjucks' features that use paths resolve from the specified template
directory by design.
In MarkBind's case, the template directories are the respective root
paths of the various sites and subsites of the project.
Hence, a separate nunjucks environment with the template directory
configured as such is needed for each site or subsite.
Let's introduce a wrapper abstraction for this, VariableRenderer, to
wrap over such a singular nunjucks environment.
The VariableRenderer instances will be managed by VariableProcessor
(previously VariablePreprocessor), which already has a framework set up
to render variables belonging to each site or subsite.
In addition, {% raw %} tags inside a layouts file do not work due to
two nunjucks passes being made.
Let's revert the removal of the helper functions to keep {% raw %} tags
in the first pass to fix this, while searching for a solution that
would allow a single pass for layouts.
@ang-zeyu

Copy link
Copy Markdown
ContributorAuthor

This looks fine to me! Just to check - I assume this has been tested on sites with nunjucks (2103T maybe?)? I don't see anything that would change existing working behaviour, but just want to be sure 🙂

yup!

Just a suggestion regarding structuring commits - I think it would be nice to have renaming variablePreprocessor -> variableProcessor to be in its own commit. This way the reviewer can filter the commits to focus only on the material changes.

That makes sense, my bad 😅 I reseparated it - although there's some renaming in the first commit as well. This should be tremendously easier to look at in the history still though.

I also moved the variable rendering portion in includeFile out of it so there's no need to pass the 2 intermediate parameters (additionalVariables and keepPercentRaw) to includeFile which shouldn't know about it.

Also more clearly separates the variable rendering from the "including" part of the process. (retested with 2103 site again too)

Render {{ MAIN_CONTENT_BODY }} and {% raw/endraw %} back to itself first,
which is then dealt with in the call below to {@link renderSiteVariables}.
*/
.then(result => this.variableProcessor.renderPage(layoutPagePath, result, {

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

variable rendering is moved out of includeFile, also in page.generate() / resolveDependency()

@ang-zeyuang-zeyu added this to the v2.15.0 milestone Jul 24, 2020
* @param content to render
* @param highestPriorityVariables to render with the highest priority if any.
* This is currently only used for the MAIN_CONTENT_BODY in layouts.
* @param keepPercentRaw whether to reoutput {% raw/endraw %} tags, also used only for layouts.

@marvinchinmarvinchinJul 25, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that variable rendering is moved out of include file, it seems clearer that there is some specific behaviour for layouts. Do you think it's worth having a renderLayout function instead of using a general purpose renderPage?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that makes sense! I think there's quite a bit of opportunity for refactor regarding layouts though, apart from this. (site nav, page layouts, header, footer takes up a sizeable portion of Page.js) Perhaps we could defer it to that pr?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds fine too, let's fix that in the next PR then 🚀

@ang-zeyu
ang-zeyu merged commit 82c5ff6 into MarkBind:masterJul 25, 2020
@ang-zeyuang-zeyu mentioned this pull request Aug 2, 2020
13 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot use nunjucks in a sub-site

3 participants

@ang-zeyu@damithc@marvinchin
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Fix nunjucks path resolving by ang-zeyu · Pull Request #1263 · MarkBind/markbind · GitHub
Skip to content

Fix nunjucks path resolving - #1263

Merged
ang-zeyu merged 2 commits into
MarkBind:masterfrom
ang-zeyu:nunjucks-per-site
Jul 25, 2020
Merged

Fix nunjucks path resolving#1263
ang-zeyu merged 2 commits into
MarkBind:masterfrom
ang-zeyu:nunjucks-per-site

Conversation

@ang-zeyu

Copy link
Copy Markdown
Contributor

What is the purpose of this pull request? (put "X" next to an item, remove the rest)

• [x] Bug fix

Fixes#931

What is the rationale for this request?

What changes did you make? (Give an overview)
To be filled

Testing instructions:

  • npm run test should pass
  • nunjucks paths should now resolve from the closest parent (sub)site ( tests added )

Proposed commit message: (wrap lines at 72 characters)
Fix nunjucks path resolving

Nunjucks' features that use paths resolve from the specified template
directory by design.
In MarkBind's case, the template directories are the respective root
paths of the various sites and subsites of the project.

Hence, a separate nunjucks environment with the template directory
configured as such is needed for each site or subsite.

Let's introduce a wrapper abstraction for this, VariableRenderer, to
wrap over such a singular nunjucks environment.
The VariableRenderer instances will be managed by VariableProcessor
(previously VariablePreprocessor), which already has a framework set up
to render variables belonging to each site or subsite.

In addition, {% raw %} tags inside a layouts file do not work due to
two nunjucks passes being made.
Let's revert the removal of the helper functions to keep {% raw %} tags
in the first pass to fix this, while searching for a solution that
would allow a single pass for layouts.

@damithc

Copy link
Copy Markdown
Contributor

This will be a big step forward if we manage to pull it off. 👍
I believe sub-site support is one of our unique features and this was a vital missing piece in that feature.

@ang-zeyu

Copy link
Copy Markdown
ContributorAuthor

It's pretty much ready, just need to update some more test files. I'm putting it on hold as we are doing quite a bit of directory restructuring as suggested by @acjh here #1253 (comment). Would be easier to review then.

I believe sub-site support is one of our unique features and this was a vital missing piece in that feature.

The next step would be layouts such as header/footer/expressive layouts and such, I'm not sure if the current behaviour is intended though (layout files from the root site can currently be used in subsites, unlike variables)
And then we have other things like respecting configuration of individual site.jsons (not sure if the current behaviour is intended as well though).

All of this should be able to be easily achieved by building the sites individually (the work so far still being necessary even if so though), and "blocking parent sites from building sub site pages".

we should probably flesh out the requirements for sub sites first in an issue or document the current behaviour in ug (if intended)

@ang-zeyuang-zeyu changed the title [WIP] Fix nunjucks path resolvingFix nunjucks path resolvingJul 22, 2020

@marvinchinmarvinchin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine to me! Just to check - I assume this has been tested on sites with nunjucks (2103T maybe?)? I don't see anything that would change existing working behaviour, but just want to be sure 🙂

Just a suggestion regarding structuring commits - I think it would be nice to have renaming variablePreprocessor -> variableProcessor to be in its own commit. This way the reviewer can filter the commits to focus only on the material changes.

Nunjucks' features that use paths resolve from the specified template
directory by design.
In MarkBind's case, the template directories are the respective root
paths of the various sites and subsites of the project.
Hence, a separate nunjucks environment with the template directory
configured as such is needed for each site or subsite.
Let's introduce a wrapper abstraction for this, VariableRenderer, to
wrap over such a singular nunjucks environment.
The VariableRenderer instances will be managed by VariableProcessor
(previously VariablePreprocessor), which already has a framework set up
to render variables belonging to each site or subsite.
In addition, {% raw %} tags inside a layouts file do not work due to
two nunjucks passes being made.
Let's revert the removal of the helper functions to keep {% raw %} tags
in the first pass to fix this, while searching for a solution that
would allow a single pass for layouts.
@ang-zeyu

Copy link
Copy Markdown
ContributorAuthor

This looks fine to me! Just to check - I assume this has been tested on sites with nunjucks (2103T maybe?)? I don't see anything that would change existing working behaviour, but just want to be sure 🙂

yup!

Just a suggestion regarding structuring commits - I think it would be nice to have renaming variablePreprocessor -> variableProcessor to be in its own commit. This way the reviewer can filter the commits to focus only on the material changes.

That makes sense, my bad 😅 I reseparated it - although there's some renaming in the first commit as well. This should be tremendously easier to look at in the history still though.

I also moved the variable rendering portion in includeFile out of it so there's no need to pass the 2 intermediate parameters (additionalVariables and keepPercentRaw) to includeFile which shouldn't know about it.

Also more clearly separates the variable rendering from the "including" part of the process. (retested with 2103 site again too)

Render {{ MAIN_CONTENT_BODY }} and {% raw/endraw %} back to itself first,
which is then dealt with in the call below to {@link renderSiteVariables}.
*/
.then(result => this.variableProcessor.renderPage(layoutPagePath, result, {

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

variable rendering is moved out of includeFile, also in page.generate() / resolveDependency()

@ang-zeyuang-zeyu added this to the v2.15.0 milestone Jul 24, 2020
* @param content to render
* @param highestPriorityVariables to render with the highest priority if any.
* This is currently only used for the MAIN_CONTENT_BODY in layouts.
* @param keepPercentRaw whether to reoutput {% raw/endraw %} tags, also used only for layouts.

@marvinchinmarvinchinJul 25, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that variable rendering is moved out of include file, it seems clearer that there is some specific behaviour for layouts. Do you think it's worth having a renderLayout function instead of using a general purpose renderPage?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that makes sense! I think there's quite a bit of opportunity for refactor regarding layouts though, apart from this. (site nav, page layouts, header, footer takes up a sizeable portion of Page.js) Perhaps we could defer it to that pr?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds fine too, let's fix that in the next PR then 🚀

@ang-zeyu
ang-zeyu merged commit 82c5ff6 into MarkBind:masterJul 25, 2020
@ang-zeyuang-zeyu mentioned this pull request Aug 2, 2020
13 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot use nunjucks in a sub-site

3 participants

@ang-zeyu@damithc@marvinchin
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Fix nunjucks path resolving by ang-zeyu · Pull Request #1263 · MarkBind/markbind · GitHub
Skip to content

Fix nunjucks path resolving - #1263

Merged
ang-zeyu merged 2 commits into
MarkBind:masterfrom
ang-zeyu:nunjucks-per-site
Jul 25, 2020
Merged

Fix nunjucks path resolving#1263
ang-zeyu merged 2 commits into
MarkBind:masterfrom
ang-zeyu:nunjucks-per-site

Conversation

@ang-zeyu

Copy link
Copy Markdown
Contributor

What is the purpose of this pull request? (put "X" next to an item, remove the rest)

• [x] Bug fix

Fixes#931

What is the rationale for this request?

What changes did you make? (Give an overview)
To be filled

Testing instructions:

  • npm run test should pass
  • nunjucks paths should now resolve from the closest parent (sub)site ( tests added )

Proposed commit message: (wrap lines at 72 characters)
Fix nunjucks path resolving

Nunjucks' features that use paths resolve from the specified template
directory by design.
In MarkBind's case, the template directories are the respective root
paths of the various sites and subsites of the project.

Hence, a separate nunjucks environment with the template directory
configured as such is needed for each site or subsite.

Let's introduce a wrapper abstraction for this, VariableRenderer, to
wrap over such a singular nunjucks environment.
The VariableRenderer instances will be managed by VariableProcessor
(previously VariablePreprocessor), which already has a framework set up
to render variables belonging to each site or subsite.

In addition, {% raw %} tags inside a layouts file do not work due to
two nunjucks passes being made.
Let's revert the removal of the helper functions to keep {% raw %} tags
in the first pass to fix this, while searching for a solution that
would allow a single pass for layouts.

@damithc

Copy link
Copy Markdown
Contributor

This will be a big step forward if we manage to pull it off. 👍
I believe sub-site support is one of our unique features and this was a vital missing piece in that feature.

@ang-zeyu

Copy link
Copy Markdown
ContributorAuthor

It's pretty much ready, just need to update some more test files. I'm putting it on hold as we are doing quite a bit of directory restructuring as suggested by @acjh here #1253 (comment). Would be easier to review then.

I believe sub-site support is one of our unique features and this was a vital missing piece in that feature.

The next step would be layouts such as header/footer/expressive layouts and such, I'm not sure if the current behaviour is intended though (layout files from the root site can currently be used in subsites, unlike variables)
And then we have other things like respecting configuration of individual site.jsons (not sure if the current behaviour is intended as well though).

All of this should be able to be easily achieved by building the sites individually (the work so far still being necessary even if so though), and "blocking parent sites from building sub site pages".

we should probably flesh out the requirements for sub sites first in an issue or document the current behaviour in ug (if intended)

@ang-zeyuang-zeyu changed the title [WIP] Fix nunjucks path resolvingFix nunjucks path resolvingJul 22, 2020

@marvinchinmarvinchin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine to me! Just to check - I assume this has been tested on sites with nunjucks (2103T maybe?)? I don't see anything that would change existing working behaviour, but just want to be sure 🙂

Just a suggestion regarding structuring commits - I think it would be nice to have renaming variablePreprocessor -> variableProcessor to be in its own commit. This way the reviewer can filter the commits to focus only on the material changes.

Nunjucks' features that use paths resolve from the specified template
directory by design.
In MarkBind's case, the template directories are the respective root
paths of the various sites and subsites of the project.
Hence, a separate nunjucks environment with the template directory
configured as such is needed for each site or subsite.
Let's introduce a wrapper abstraction for this, VariableRenderer, to
wrap over such a singular nunjucks environment.
The VariableRenderer instances will be managed by VariableProcessor
(previously VariablePreprocessor), which already has a framework set up
to render variables belonging to each site or subsite.
In addition, {% raw %} tags inside a layouts file do not work due to
two nunjucks passes being made.
Let's revert the removal of the helper functions to keep {% raw %} tags
in the first pass to fix this, while searching for a solution that
would allow a single pass for layouts.
@ang-zeyu

Copy link
Copy Markdown
ContributorAuthor

This looks fine to me! Just to check - I assume this has been tested on sites with nunjucks (2103T maybe?)? I don't see anything that would change existing working behaviour, but just want to be sure 🙂

yup!

Just a suggestion regarding structuring commits - I think it would be nice to have renaming variablePreprocessor -> variableProcessor to be in its own commit. This way the reviewer can filter the commits to focus only on the material changes.

That makes sense, my bad 😅 I reseparated it - although there's some renaming in the first commit as well. This should be tremendously easier to look at in the history still though.

I also moved the variable rendering portion in includeFile out of it so there's no need to pass the 2 intermediate parameters (additionalVariables and keepPercentRaw) to includeFile which shouldn't know about it.

Also more clearly separates the variable rendering from the "including" part of the process. (retested with 2103 site again too)

Render {{ MAIN_CONTENT_BODY }} and {% raw/endraw %} back to itself first,
which is then dealt with in the call below to {@link renderSiteVariables}.
*/
.then(result => this.variableProcessor.renderPage(layoutPagePath, result, {

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

variable rendering is moved out of includeFile, also in page.generate() / resolveDependency()

@ang-zeyuang-zeyu added this to the v2.15.0 milestone Jul 24, 2020
* @param content to render
* @param highestPriorityVariables to render with the highest priority if any.
* This is currently only used for the MAIN_CONTENT_BODY in layouts.
* @param keepPercentRaw whether to reoutput {% raw/endraw %} tags, also used only for layouts.

@marvinchinmarvinchinJul 25, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that variable rendering is moved out of include file, it seems clearer that there is some specific behaviour for layouts. Do you think it's worth having a renderLayout function instead of using a general purpose renderPage?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that makes sense! I think there's quite a bit of opportunity for refactor regarding layouts though, apart from this. (site nav, page layouts, header, footer takes up a sizeable portion of Page.js) Perhaps we could defer it to that pr?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds fine too, let's fix that in the next PR then 🚀

@ang-zeyu
ang-zeyu merged commit 82c5ff6 into MarkBind:masterJul 25, 2020
@ang-zeyuang-zeyu mentioned this pull request Aug 2, 2020
13 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot use nunjucks in a sub-site

3 participants

@ang-zeyu@damithc@marvinchin
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' Fix nunjucks path resolving by ang-zeyu · Pull Request #1263 · MarkBind/markbind · GitHub
Skip to content

Fix nunjucks path resolving - #1263

Merged
ang-zeyu merged 2 commits into
MarkBind:masterfrom
ang-zeyu:nunjucks-per-site
Jul 25, 2020
Merged

Fix nunjucks path resolving#1263
ang-zeyu merged 2 commits into
MarkBind:masterfrom
ang-zeyu:nunjucks-per-site

Conversation

@ang-zeyu

Copy link
Copy Markdown
Contributor

What is the purpose of this pull request? (put "X" next to an item, remove the rest)

• [x] Bug fix

Fixes#931

What is the rationale for this request?

What changes did you make? (Give an overview)
To be filled

Testing instructions:

  • npm run test should pass
  • nunjucks paths should now resolve from the closest parent (sub)site ( tests added )

Proposed commit message: (wrap lines at 72 characters)
Fix nunjucks path resolving

Nunjucks' features that use paths resolve from the specified template
directory by design.
In MarkBind's case, the template directories are the respective root
paths of the various sites and subsites of the project.

Hence, a separate nunjucks environment with the template directory
configured as such is needed for each site or subsite.

Let's introduce a wrapper abstraction for this, VariableRenderer, to
wrap over such a singular nunjucks environment.
The VariableRenderer instances will be managed by VariableProcessor
(previously VariablePreprocessor), which already has a framework set up
to render variables belonging to each site or subsite.

In addition, {% raw %} tags inside a layouts file do not work due to
two nunjucks passes being made.
Let's revert the removal of the helper functions to keep {% raw %} tags
in the first pass to fix this, while searching for a solution that
would allow a single pass for layouts.

@damithc

Copy link
Copy Markdown
Contributor

This will be a big step forward if we manage to pull it off. 👍
I believe sub-site support is one of our unique features and this was a vital missing piece in that feature.

@ang-zeyu

Copy link
Copy Markdown
ContributorAuthor

It's pretty much ready, just need to update some more test files. I'm putting it on hold as we are doing quite a bit of directory restructuring as suggested by @acjh here #1253 (comment). Would be easier to review then.

I believe sub-site support is one of our unique features and this was a vital missing piece in that feature.

The next step would be layouts such as header/footer/expressive layouts and such, I'm not sure if the current behaviour is intended though (layout files from the root site can currently be used in subsites, unlike variables)
And then we have other things like respecting configuration of individual site.jsons (not sure if the current behaviour is intended as well though).

All of this should be able to be easily achieved by building the sites individually (the work so far still being necessary even if so though), and "blocking parent sites from building sub site pages".

we should probably flesh out the requirements for sub sites first in an issue or document the current behaviour in ug (if intended)

@ang-zeyuang-zeyu changed the title [WIP] Fix nunjucks path resolvingFix nunjucks path resolvingJul 22, 2020

@marvinchinmarvinchin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine to me! Just to check - I assume this has been tested on sites with nunjucks (2103T maybe?)? I don't see anything that would change existing working behaviour, but just want to be sure 🙂

Just a suggestion regarding structuring commits - I think it would be nice to have renaming variablePreprocessor -> variableProcessor to be in its own commit. This way the reviewer can filter the commits to focus only on the material changes.

Nunjucks' features that use paths resolve from the specified template
directory by design.
In MarkBind's case, the template directories are the respective root
paths of the various sites and subsites of the project.
Hence, a separate nunjucks environment with the template directory
configured as such is needed for each site or subsite.
Let's introduce a wrapper abstraction for this, VariableRenderer, to
wrap over such a singular nunjucks environment.
The VariableRenderer instances will be managed by VariableProcessor
(previously VariablePreprocessor), which already has a framework set up
to render variables belonging to each site or subsite.
In addition, {% raw %} tags inside a layouts file do not work due to
two nunjucks passes being made.
Let's revert the removal of the helper functions to keep {% raw %} tags
in the first pass to fix this, while searching for a solution that
would allow a single pass for layouts.
@ang-zeyu

Copy link
Copy Markdown
ContributorAuthor

This looks fine to me! Just to check - I assume this has been tested on sites with nunjucks (2103T maybe?)? I don't see anything that would change existing working behaviour, but just want to be sure 🙂

yup!

Just a suggestion regarding structuring commits - I think it would be nice to have renaming variablePreprocessor -> variableProcessor to be in its own commit. This way the reviewer can filter the commits to focus only on the material changes.

That makes sense, my bad 😅 I reseparated it - although there's some renaming in the first commit as well. This should be tremendously easier to look at in the history still though.

I also moved the variable rendering portion in includeFile out of it so there's no need to pass the 2 intermediate parameters (additionalVariables and keepPercentRaw) to includeFile which shouldn't know about it.

Also more clearly separates the variable rendering from the "including" part of the process. (retested with 2103 site again too)

Render {{ MAIN_CONTENT_BODY }} and {% raw/endraw %} back to itself first,
which is then dealt with in the call below to {@link renderSiteVariables}.
*/
.then(result => this.variableProcessor.renderPage(layoutPagePath, result, {

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

variable rendering is moved out of includeFile, also in page.generate() / resolveDependency()

@ang-zeyuang-zeyu added this to the v2.15.0 milestone Jul 24, 2020
* @param content to render
* @param highestPriorityVariables to render with the highest priority if any.
* This is currently only used for the MAIN_CONTENT_BODY in layouts.
* @param keepPercentRaw whether to reoutput {% raw/endraw %} tags, also used only for layouts.

@marvinchinmarvinchinJul 25, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that variable rendering is moved out of include file, it seems clearer that there is some specific behaviour for layouts. Do you think it's worth having a renderLayout function instead of using a general purpose renderPage?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that makes sense! I think there's quite a bit of opportunity for refactor regarding layouts though, apart from this. (site nav, page layouts, header, footer takes up a sizeable portion of Page.js) Perhaps we could defer it to that pr?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds fine too, let's fix that in the next PR then 🚀

@ang-zeyu
ang-zeyu merged commit 82c5ff6 into MarkBind:masterJul 25, 2020
@ang-zeyuang-zeyu mentioned this pull request Aug 2, 2020
13 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot use nunjucks in a sub-site

3 participants

@ang-zeyu@damithc@marvinchin
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Fix nunjucks path resolving by ang-zeyu · Pull Request #1263 · MarkBind/markbind · GitHub
Skip to content

Fix nunjucks path resolving - #1263

Merged
ang-zeyu merged 2 commits into
MarkBind:masterfrom
ang-zeyu:nunjucks-per-site
Jul 25, 2020
Merged

Fix nunjucks path resolving#1263
ang-zeyu merged 2 commits into
MarkBind:masterfrom
ang-zeyu:nunjucks-per-site

Conversation

@ang-zeyu

Copy link
Copy Markdown
Contributor

What is the purpose of this pull request? (put "X" next to an item, remove the rest)

• [x] Bug fix

Fixes#931

What is the rationale for this request?

What changes did you make? (Give an overview)
To be filled

Testing instructions:

  • npm run test should pass
  • nunjucks paths should now resolve from the closest parent (sub)site ( tests added )

Proposed commit message: (wrap lines at 72 characters)
Fix nunjucks path resolving

Nunjucks' features that use paths resolve from the specified template
directory by design.
In MarkBind's case, the template directories are the respective root
paths of the various sites and subsites of the project.

Hence, a separate nunjucks environment with the template directory
configured as such is needed for each site or subsite.

Let's introduce a wrapper abstraction for this, VariableRenderer, to
wrap over such a singular nunjucks environment.
The VariableRenderer instances will be managed by VariableProcessor
(previously VariablePreprocessor), which already has a framework set up
to render variables belonging to each site or subsite.

In addition, {% raw %} tags inside a layouts file do not work due to
two nunjucks passes being made.
Let's revert the removal of the helper functions to keep {% raw %} tags
in the first pass to fix this, while searching for a solution that
would allow a single pass for layouts.

@damithc

Copy link
Copy Markdown
Contributor

This will be a big step forward if we manage to pull it off. 👍
I believe sub-site support is one of our unique features and this was a vital missing piece in that feature.

@ang-zeyu

Copy link
Copy Markdown
ContributorAuthor

It's pretty much ready, just need to update some more test files. I'm putting it on hold as we are doing quite a bit of directory restructuring as suggested by @acjh here #1253 (comment). Would be easier to review then.

I believe sub-site support is one of our unique features and this was a vital missing piece in that feature.

The next step would be layouts such as header/footer/expressive layouts and such, I'm not sure if the current behaviour is intended though (layout files from the root site can currently be used in subsites, unlike variables)
And then we have other things like respecting configuration of individual site.jsons (not sure if the current behaviour is intended as well though).

All of this should be able to be easily achieved by building the sites individually (the work so far still being necessary even if so though), and "blocking parent sites from building sub site pages".

we should probably flesh out the requirements for sub sites first in an issue or document the current behaviour in ug (if intended)

@ang-zeyuang-zeyu changed the title [WIP] Fix nunjucks path resolvingFix nunjucks path resolvingJul 22, 2020

@marvinchinmarvinchin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine to me! Just to check - I assume this has been tested on sites with nunjucks (2103T maybe?)? I don't see anything that would change existing working behaviour, but just want to be sure 🙂

Just a suggestion regarding structuring commits - I think it would be nice to have renaming variablePreprocessor -> variableProcessor to be in its own commit. This way the reviewer can filter the commits to focus only on the material changes.

Nunjucks' features that use paths resolve from the specified template
directory by design.
In MarkBind's case, the template directories are the respective root
paths of the various sites and subsites of the project.
Hence, a separate nunjucks environment with the template directory
configured as such is needed for each site or subsite.
Let's introduce a wrapper abstraction for this, VariableRenderer, to
wrap over such a singular nunjucks environment.
The VariableRenderer instances will be managed by VariableProcessor
(previously VariablePreprocessor), which already has a framework set up
to render variables belonging to each site or subsite.
In addition, {% raw %} tags inside a layouts file do not work due to
two nunjucks passes being made.
Let's revert the removal of the helper functions to keep {% raw %} tags
in the first pass to fix this, while searching for a solution that
would allow a single pass for layouts.
@ang-zeyu

Copy link
Copy Markdown
ContributorAuthor

This looks fine to me! Just to check - I assume this has been tested on sites with nunjucks (2103T maybe?)? I don't see anything that would change existing working behaviour, but just want to be sure 🙂

yup!

Just a suggestion regarding structuring commits - I think it would be nice to have renaming variablePreprocessor -> variableProcessor to be in its own commit. This way the reviewer can filter the commits to focus only on the material changes.

That makes sense, my bad 😅 I reseparated it - although there's some renaming in the first commit as well. This should be tremendously easier to look at in the history still though.

I also moved the variable rendering portion in includeFile out of it so there's no need to pass the 2 intermediate parameters (additionalVariables and keepPercentRaw) to includeFile which shouldn't know about it.

Also more clearly separates the variable rendering from the "including" part of the process. (retested with 2103 site again too)

Render {{ MAIN_CONTENT_BODY }} and {% raw/endraw %} back to itself first,
which is then dealt with in the call below to {@link renderSiteVariables}.
*/
.then(result => this.variableProcessor.renderPage(layoutPagePath, result, {

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

variable rendering is moved out of includeFile, also in page.generate() / resolveDependency()

@ang-zeyuang-zeyu added this to the v2.15.0 milestone Jul 24, 2020
* @param content to render
* @param highestPriorityVariables to render with the highest priority if any.
* This is currently only used for the MAIN_CONTENT_BODY in layouts.
* @param keepPercentRaw whether to reoutput {% raw/endraw %} tags, also used only for layouts.

@marvinchinmarvinchinJul 25, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that variable rendering is moved out of include file, it seems clearer that there is some specific behaviour for layouts. Do you think it's worth having a renderLayout function instead of using a general purpose renderPage?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that makes sense! I think there's quite a bit of opportunity for refactor regarding layouts though, apart from this. (site nav, page layouts, header, footer takes up a sizeable portion of Page.js) Perhaps we could defer it to that pr?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds fine too, let's fix that in the next PR then 🚀

@ang-zeyu
ang-zeyu merged commit 82c5ff6 into MarkBind:masterJul 25, 2020
@ang-zeyuang-zeyu mentioned this pull request Aug 2, 2020
13 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot use nunjucks in a sub-site

3 participants

@ang-zeyu@damithc@marvinchin
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); })(); Fix nunjucks path resolving by ang-zeyu · Pull Request #1263 · MarkBind/markbind · GitHub
Skip to content

Fix nunjucks path resolving - #1263

Merged
ang-zeyu merged 2 commits into
MarkBind:masterfrom
ang-zeyu:nunjucks-per-site
Jul 25, 2020
Merged

Fix nunjucks path resolving#1263
ang-zeyu merged 2 commits into
MarkBind:masterfrom
ang-zeyu:nunjucks-per-site

Conversation

@ang-zeyu

Copy link
Copy Markdown
Contributor

What is the purpose of this pull request? (put "X" next to an item, remove the rest)

• [x] Bug fix

Fixes#931

What is the rationale for this request?

What changes did you make? (Give an overview)
To be filled

Testing instructions:

  • npm run test should pass
  • nunjucks paths should now resolve from the closest parent (sub)site ( tests added )

Proposed commit message: (wrap lines at 72 characters)
Fix nunjucks path resolving

Nunjucks' features that use paths resolve from the specified template
directory by design.
In MarkBind's case, the template directories are the respective root
paths of the various sites and subsites of the project.

Hence, a separate nunjucks environment with the template directory
configured as such is needed for each site or subsite.

Let's introduce a wrapper abstraction for this, VariableRenderer, to
wrap over such a singular nunjucks environment.
The VariableRenderer instances will be managed by VariableProcessor
(previously VariablePreprocessor), which already has a framework set up
to render variables belonging to each site or subsite.

In addition, {% raw %} tags inside a layouts file do not work due to
two nunjucks passes being made.
Let's revert the removal of the helper functions to keep {% raw %} tags
in the first pass to fix this, while searching for a solution that
would allow a single pass for layouts.

@damithc

Copy link
Copy Markdown
Contributor

This will be a big step forward if we manage to pull it off. 👍
I believe sub-site support is one of our unique features and this was a vital missing piece in that feature.

@ang-zeyu

Copy link
Copy Markdown
ContributorAuthor

It's pretty much ready, just need to update some more test files. I'm putting it on hold as we are doing quite a bit of directory restructuring as suggested by @acjh here #1253 (comment). Would be easier to review then.

I believe sub-site support is one of our unique features and this was a vital missing piece in that feature.

The next step would be layouts such as header/footer/expressive layouts and such, I'm not sure if the current behaviour is intended though (layout files from the root site can currently be used in subsites, unlike variables)
And then we have other things like respecting configuration of individual site.jsons (not sure if the current behaviour is intended as well though).

All of this should be able to be easily achieved by building the sites individually (the work so far still being necessary even if so though), and "blocking parent sites from building sub site pages".

we should probably flesh out the requirements for sub sites first in an issue or document the current behaviour in ug (if intended)

@ang-zeyuang-zeyu changed the title [WIP] Fix nunjucks path resolvingFix nunjucks path resolvingJul 22, 2020

@marvinchinmarvinchin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine to me! Just to check - I assume this has been tested on sites with nunjucks (2103T maybe?)? I don't see anything that would change existing working behaviour, but just want to be sure 🙂

Just a suggestion regarding structuring commits - I think it would be nice to have renaming variablePreprocessor -> variableProcessor to be in its own commit. This way the reviewer can filter the commits to focus only on the material changes.

Nunjucks' features that use paths resolve from the specified template
directory by design.
In MarkBind's case, the template directories are the respective root
paths of the various sites and subsites of the project.
Hence, a separate nunjucks environment with the template directory
configured as such is needed for each site or subsite.
Let's introduce a wrapper abstraction for this, VariableRenderer, to
wrap over such a singular nunjucks environment.
The VariableRenderer instances will be managed by VariableProcessor
(previously VariablePreprocessor), which already has a framework set up
to render variables belonging to each site or subsite.
In addition, {% raw %} tags inside a layouts file do not work due to
two nunjucks passes being made.
Let's revert the removal of the helper functions to keep {% raw %} tags
in the first pass to fix this, while searching for a solution that
would allow a single pass for layouts.
@ang-zeyu

Copy link
Copy Markdown
ContributorAuthor

This looks fine to me! Just to check - I assume this has been tested on sites with nunjucks (2103T maybe?)? I don't see anything that would change existing working behaviour, but just want to be sure 🙂

yup!

Just a suggestion regarding structuring commits - I think it would be nice to have renaming variablePreprocessor -> variableProcessor to be in its own commit. This way the reviewer can filter the commits to focus only on the material changes.

That makes sense, my bad 😅 I reseparated it - although there's some renaming in the first commit as well. This should be tremendously easier to look at in the history still though.

I also moved the variable rendering portion in includeFile out of it so there's no need to pass the 2 intermediate parameters (additionalVariables and keepPercentRaw) to includeFile which shouldn't know about it.

Also more clearly separates the variable rendering from the "including" part of the process. (retested with 2103 site again too)

Render {{ MAIN_CONTENT_BODY }} and {% raw/endraw %} back to itself first,
which is then dealt with in the call below to {@link renderSiteVariables}.
*/
.then(result => this.variableProcessor.renderPage(layoutPagePath, result, {

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

variable rendering is moved out of includeFile, also in page.generate() / resolveDependency()

@ang-zeyuang-zeyu added this to the v2.15.0 milestone Jul 24, 2020
* @param content to render
* @param highestPriorityVariables to render with the highest priority if any.
* This is currently only used for the MAIN_CONTENT_BODY in layouts.
* @param keepPercentRaw whether to reoutput {% raw/endraw %} tags, also used only for layouts.

@marvinchinmarvinchinJul 25, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that variable rendering is moved out of include file, it seems clearer that there is some specific behaviour for layouts. Do you think it's worth having a renderLayout function instead of using a general purpose renderPage?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that makes sense! I think there's quite a bit of opportunity for refactor regarding layouts though, apart from this. (site nav, page layouts, header, footer takes up a sizeable portion of Page.js) Perhaps we could defer it to that pr?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds fine too, let's fix that in the next PR then 🚀

@ang-zeyu
ang-zeyu merged commit 82c5ff6 into MarkBind:masterJul 25, 2020
@ang-zeyuang-zeyu mentioned this pull request Aug 2, 2020
13 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot use nunjucks in a sub-site

3 participants

@ang-zeyu@damithc@marvinchin