tools: build all.json by combining generated JSON by rubys · Pull Request #21637 · nodejs/node

rubys · 2018-07-03T16:55:48Z

Notes:

Removed a number of root properties that did not seem relevant: source,
desc, and introduced_in. There no longer is a source, and the other two are
from the first include and do not reflect the entire API.
As with doc: all.html is seriously broken link-wise #20100, the current "desc"
properties sometimes contained in-page links, other times referenced another
page, and often did not match the links in the original HTML or JSON file.
I chose to standardize on external links as "desc" values are isolated
snippets as opposed to all.html which can be viewed as a standalone and self
contained document.
Eliminated preprocessing for @include entirely, including the test case
for this function.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are updated
commit message follows commit guidelines

rubys · 2018-07-03T17:06:08Z

See #21568 (comment) for the original request.

Trott · 2018-07-03T17:18:05Z

Not sure if @tolmasky or any of their RunKit colleagues will be able to test this ahead of time, but pinging anyway as their the one significant consumer of our docs-in-JSON-format that I'm aware of. (I don't know if this would help or not, but maybe it would help them if the resulting JSON could be stashed somewhere, maybe in a gist or something.)

vsemozhetbyt · 2018-07-03T17:22:52Z

What will happen with https://github.com/nodejs/node/blob/master/doc/api/index.md? It also contains one include directive.

rubys · 2018-07-03T17:30:17Z

@Trott @tolmasky gist containing both what all.json will look like after this change, and a diff between what it looks like after vs what it looked like before.

@vsemozhetbyt good catch! Initial thoughts: _toc.md should be renamed to _index.md (overwriting this file). allhtml.js and alljson.js would need to be updated to read this data from its new location.

If you agree, I'll make this change and squash the commit.

vsemozhetbyt · 2018-07-03T17:43:07Z

Seems good, but we should also check mentions like this one:

node/tools/doc/html.js

Line 39 in 6e16ad7

constgtocPath=path.join(docPath,'api','_toc.md');

vsemozhetbyt · 2018-07-03T19:43:12Z

I hope I will be able to review at the weekend. If anybody feels confident to review and land sooner please do)

Me1000 · 2018-07-03T19:58:01Z

Hey all! I work on the RunKit team, thanks for the ping @Trott.

The new output looks like it contains all the existing data. If that is indeed the case (the output is quite large, so maybe I missed something) then it should be fairly easy for us consume the new format. :)

vsemozhetbyt · 2018-07-07T11:21:31Z

One breaking change in the index.html: previously, preprocess.js not only processed @include directives but also stripped the special comments (see commentExpr mentions in preprocess.js). Maybe we should make these comments formatted as HTML comments — these are just two lines in the index.md. Otherwise, they are rendered in the index.html:

vsemozhetbyt · 2018-07-07T10:55:36Z

It seems the parentheses are redundant now.

vsemozhetbyt · 2018-07-07T11:49:08Z

global -> globals?

vsemozhetbyt · 2018-07-07T11:49:42Z

initialize -> Initialize.
Extra space between the sentences.

vsemozhetbyt · 2018-07-07T13:29:36Z

.replace('.html', '.json') seems erroneous here?

vsemozhetbyt · 2018-07-07T13:31:11Z

It seems this should be /<a href=\\"#/g, `<a href=\\"${href}#`, as double quotes are escaped in .json sources.

vsemozhetbyt · 2018-07-07T13:34:17Z

seen[href] -> seen[json]?

vsemozhetbyt · 2018-07-07T16:56:33Z

Sorry, it seems this line also need to be updated so that the new HTML comment from index.md could be excluded from all the docs except the index.html.

I mean the .replace(/^@\/\/.*$/gm, '') part.

vsemozhetbyt · 2018-07-07T19:13:04Z

(Just to prevent doubts: dotAll flag is supported since V8 6.2, so this line is backportable for Node.js 8 LTS.)

vsemozhetbyt · 2018-07-07T19:45:58Z

Just one more significant nit I've found while diffing the old and new results: new all.json lacks the \n at the end of the file, just after the last }. So maybe it is worth to replace JSON.stringify(results, null, 2) with `${JSON.stringify(results, null, 2)}\n` to be on the safe side)

vsemozhetbyt · 2018-07-07T19:50:06Z

A note: amended commits are not reported to the people that are subscribed to a PR, so maybe it is worth to let everybody know about a new commit version by a comment)

@include

Notes: 1) Removed a number of root properties that did not seem relevant: source, desc, and introduced_in. There no longer is a source, and the other two are from the first include and do not reflect the entire API. 2) As with nodejs#20100, the current "desc" properties sometimes contained in-page links, other times referenced another page, and often did not match the links in the original HTML or JSON file. I chose to standardize on external links as "desc" values are isolated snippets as opposed to all.html which can be viewed as a standalone and self contained document. 3) Eliminated preprocessing for @include entirely, including the test case for this function. 4) _toc.md was renamed to index.md. 5) index comments no longer appear in embedded TOCs (left hand side column in the generated documentation.

rubys · 2018-07-07T20:08:32Z

Perhaps I should just stop squashing commits.

vsemozhetbyt · 2018-07-07T20:23:15Z

CI-lite: https://ci.nodejs.org/job/node-test-pull-request-lite/904/

vsemozhetbyt · 2018-07-07T22:13:10Z

Let's land this on Monday, maybe somebody from @nodejs/documentation will want to chime in.

@include

Notes: 1) Removed a number of root properties that did not seem relevant: source, desc, and introduced_in. There no longer is a source, and the other two are from the first include and do not reflect the entire API. 2) As with #20100, the current "desc" properties sometimes contained in-page links, other times referenced another page, and often did not match the links in the original HTML or JSON file. I chose to standardize on external links as "desc" values are isolated snippets as opposed to all.html which can be viewed as a standalone and self contained document. 3) Eliminated preprocessing for @include entirely, including the test case for this function. 4) _toc.md was renamed to index.md. 5) index comments no longer appear in embedded TOCs (left hand side column in the generated documentation. PR-URL: #21637 Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

vsemozhetbyt · 2018-07-09T20:02:11Z

Landed in 0c743b5
Thank you!

@include

Notes: 1) Removed a number of root properties that did not seem relevant: source, desc, and introduced_in. There no longer is a source, and the other two are from the first include and do not reflect the entire API. 2) As with #20100, the current "desc" properties sometimes contained in-page links, other times referenced another page, and often did not match the links in the original HTML or JSON file. I chose to standardize on external links as "desc" values are isolated snippets as opposed to all.html which can be viewed as a standalone and self contained document. 3) Eliminated preprocessing for @include entirely, including the test case for this function. 4) _toc.md was renamed to index.md. 5) index comments no longer appear in embedded TOCs (left hand side column in the generated documentation. PR-URL: #21637 Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

nodejs-github-bot added build Issues and PRs related to build files or the CI. doc Issues and PRs related to the documentations. tools Issues and PRs related to the tools directory. labels Jul 3, 2018

rubys force-pushed the alljson branch 2 times, most recently from 95ad110 to 511f403Compare July 3, 2018 19:04

vsemozhetbyt reviewed Jul 7, 2018
View reviewed changes

rubys force-pushed the alljson branch from 511f403 to f2308dcCompare July 7, 2018 15:03

vsemozhetbyt reviewed Jul 7, 2018
View reviewed changes

rubys force-pushed the alljson branch from f2308dc to f54c875Compare July 7, 2018 17:26

vsemozhetbyt reviewed Jul 7, 2018
View reviewed changes

rubys force-pushed the alljson branch from f54c875 to 7ac03daCompare July 7, 2018 19:58

vsemozhetbyt approved these changes Jul 7, 2018
View reviewed changes

vsemozhetbyt added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jul 7, 2018

Trott approved these changes Jul 7, 2018
View reviewed changes

vsemozhetbyt closed this Jul 9, 2018

vsemozhetbyt mentioned this pull request Jul 9, 2018
Creating a branch of the Docs using RunKit #21723
Closed

targos removed the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jul 12, 2018

vsemozhetbyt mentioned this pull request Jul 14, 2018
test/doctool/test-make-doc.js fails if .DS_Store exists #21519
Closed

targos mentioned this pull request Jul 17, 2018
v10.7.0 proposal #21851
Merged

mcollina mentioned this pull request Jul 20, 2018
test: unclear error with _toc.html #21912
Closed

Uh oh!

Uh oh!

Conversation

rubys commented Jul 3, 2018

Checklist

Uh oh!

rubys commented Jul 3, 2018

Uh oh!

Trott commented Jul 3, 2018

Uh oh!

vsemozhetbyt commented Jul 3, 2018

Uh oh!

rubys commented Jul 3, 2018

Uh oh!

vsemozhetbyt commented Jul 3, 2018• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vsemozhetbyt commented Jul 3, 2018

Uh oh!

Me1000 commented Jul 3, 2018

Uh oh!

vsemozhetbyt commented Jul 7, 2018• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vsemozhetbyt commented Jul 7, 2018

Uh oh!

rubys commented Jul 7, 2018

Uh oh!

vsemozhetbyt commented Jul 7, 2018

Uh oh!

vsemozhetbyt commented Jul 7, 2018

Uh oh!

vsemozhetbyt commented Jul 9, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

vsemozhetbyt commented Jul 3, 2018•
edited
Loading

vsemozhetbyt commented Jul 7, 2018•
edited
Loading