src: add /json/protocol endpoint to inspector by bnoordhuis · Pull Request #7491 · nodejs/node

bnoordhuis · 2016-06-29T23:24:41Z

Embed the compressed and minified protocol.json from the bundled
v8_inspector and make it available through the /json/protocol endpoint.

Refs: nodejs/diagnostics#52
CI: https://ci.nodejs.org/job/node-test-pull-request/3133/
EDIT: https://ci.nodejs.org/job/node-test-pull-request/3135/

addaleax · 2016-06-29T23:35:57Z

0x10000u?

Indeed. I'll fix that.

jasnell · 2016-06-30T00:05:08Z

LGTM but can you add a quick statement to the commit log about why? :-)

addaleax · 2016-06-30T00:13:50Z

Adding thread_id * 100 to the port number and requiring the port to match common.PORT seem to be kind of contradictory?

I didn't make it up, exports.PORT += process.env.TEST_THREAD_ID * 100; is what test/common.js does.

Oh, sorry, didn’t see that value is changed later in test/common.js.

addaleax · 2016-06-30T00:25:26Z

This CI failure looks related

joshgav · 2016-06-30T02:34:03Z

Does this commit intend to canonicalize V8's embedded protocol.json as Node's protocol.json? I don't think we should finalize that yet, that's the goal of the discussion in nodejs/diagnostics#52.

bnoordhuis · 2016-06-30T07:33:25Z

This CI failure looks related

Looks like I got the dependency chain wrong. It also looks like we're doing that wrong in more places. I added a fix-up that makes the inspector build depend explicitly on the generated header.

New CI: https://ci.nodejs.org/job/node-test-pull-request/3137/

Does this commit intend to canonicalize V8's embedded protocol.json as Node's protocol.json?

No.

saghul · 2016-06-30T08:35:57Z

not sure if important for this kind of script, but fp is not explicitly closed. Since we depend on 2.6-2.7, maybe use a with statement instead?

The script is short-lived so I went for simplicity.

addaleax · 2016-06-30T08:44:55Z

Apparently, that did not do the trick… https://ci.nodejs.org/job/node-test-commit-arm/3865/nodes=armv7-wheezy/console

bnoordhuis · 2016-06-30T08:45:09Z

I'm starting to think the bug might not be in my changes to node.gyp. Other buildbots do this:

 LD_LIBRARY_PATH=/home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-ubuntu1404/out/Release/lib.host:/home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-ubuntu1404/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../.; mkdir -p /home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-ubuntu1404/out/Release/obj/gen; python tools/js2c.py "/home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-ubuntu1404/out/Release/obj/gen/node_natives.h" lib/internal/bootstrap_node.js lib/_debug_agent.js lib/_debugger.js lib/assert.js lib/buffer.js lib/child_process.js lib/console.js lib/constants.js lib/crypto.js lib/cluster.js lib/dgram.js lib/dns.js lib/domain.js lib/events.js lib/fs.js lib/http.js lib/_http_agent.js lib/_http_client.js lib/_http_common.js lib/_http_incoming.js lib/_http_outgoing.js lib/_http_server.js lib/https.js lib/_linklist.js lib/module.js lib/net.js lib/os.js lib/path.js lib/process.js lib/punycode.js lib/querystring.js lib/readline.js lib/repl.js lib/stream.js lib/_stream_readable.js lib/_stream_writable.js lib/_stream_duplex.js lib/_stream_transform.js lib/_stream_passthrough.js lib/_stream_wrap.js lib/string_decoder.js lib/sys.js lib/timers.js lib/tls.js lib/_tls_common.js lib/_tls_legacy.js lib/_tls_wrap.js lib/tty.js lib/url.js lib/util.js lib/v8.js lib/vm.js lib/zlib.js lib/internal/child_process.js lib/internal/cluster.js lib/internal/freelist.js lib/internal/linkedlist.js lib/internal/net.js lib/internal/module.js lib/internal/process/next_tick.js lib/internal/process/promises.js lib/internal/process/stdio.js lib/internal/process/warning.js lib/internal/process.js lib/internal/readline.js lib/internal/repl.js lib/internal/socket_list.js lib/internal/util.js lib/internal/v8_prof_polyfill.js lib/internal/v8_prof_processor.js lib/internal/streams/lazy_transform.js lib/internal/streams/BufferList.js deps/v8/tools/splaytree.js deps/v8/tools/codemap.js deps/v8/tools/consarray.js deps/v8/tools/csvparser.js deps/v8/tools/profile.js deps/v8/tools/profile_view.js deps/v8/tools/logreader.js deps/v8/tools/tickprocessor.js deps/v8/tools/SourceMap.js deps/v8/tools/tickprocessor-driver.js ./config.gypi src/notrace_macros.py src/nolttng_macros.py src/perfctr_macros.py
touch /home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-ubuntu1404/out/Release/obj.target/node_dtrace_header.stamp
touch /home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-ubuntu1404/out/Release/obj.target/node_dtrace_provider.stamp
touch /home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-ubuntu1404/out/Release/obj.target/node_dtrace_ustack.stamp
touch /home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-ubuntu1404/out/Release/obj.target/node_etw.stamp
touch /home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-ubuntu1404/out/Release/obj.target/node_perfctr.stamp
touch /home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-ubuntu1404/out/Release/obj.target/specialize_node_d.stamp
LD_LIBRARY_PATH=/home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-ubuntu1404/out/Release/lib.host:/home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-ubuntu1404/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../.; mkdir -p /home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-ubuntu1404/out/Release/obj/gen; python tools/compress_json.py deps/v8_inspector/platform/v8_inspector/js_protocol.json "/home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-ubuntu1404/out/Release/obj/gen/v8_inspector_protocol_json.h"

But the armv7-wheezy buildbot only does this:

 LD_LIBRARY_PATH=/home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-wheezy/out/Release/lib.host:/home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-wheezy/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../.; mkdir -p /home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-wheezy/out/Release/obj/gen; python tools/js2c.py "/home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-wheezy/out/Release/obj/gen/node_natives.h" lib/internal/bootstrap_node.js lib/_debug_agent.js lib/_debugger.js lib/assert.js lib/buffer.js lib/child_process.js lib/console.js lib/constants.js lib/crypto.js lib/cluster.js lib/dgram.js lib/dns.js lib/domain.js lib/events.js lib/fs.js lib/http.js lib/_http_agent.js lib/_http_client.js lib/_http_common.js lib/_http_incoming.js lib/_http_outgoing.js lib/_http_server.js lib/https.js lib/_linklist.js lib/module.js lib/net.js lib/os.js lib/path.js lib/process.js lib/punycode.js lib/querystring.js lib/readline.js lib/repl.js lib/stream.js lib/_stream_readable.js lib/_stream_writable.js lib/_stream_duplex.js lib/_stream_transform.js lib/_stream_passthrough.js lib/_stream_wrap.js lib/string_decoder.js lib/sys.js lib/timers.js lib/tls.js lib/_tls_common.js lib/_tls_legacy.js lib/_tls_wrap.js lib/tty.js lib/url.js lib/util.js lib/v8.js lib/vm.js lib/zlib.js lib/internal/child_process.js lib/internal/cluster.js lib/internal/freelist.js lib/internal/linkedlist.js lib/internal/net.js lib/internal/module.js lib/internal/process/next_tick.js lib/internal/process/promises.js lib/internal/process/stdio.js lib/internal/process/warning.js lib/internal/process.js lib/internal/readline.js lib/internal/repl.js lib/internal/socket_list.js lib/internal/util.js lib/internal/v8_prof_polyfill.js lib/internal/v8_prof_processor.js lib/internal/streams/lazy_transform.js lib/internal/streams/BufferList.js deps/v8/tools/splaytree.js deps/v8/tools/codemap.js deps/v8/tools/consarray.js deps/v8/tools/csvparser.js deps/v8/tools/profile.js deps/v8/tools/profile_view.js deps/v8/tools/logreader.js deps/v8/tools/tickprocessor.js deps/v8/tools/SourceMap.js deps/v8/tools/tickprocessor-driver.js ./config.gypi src/notrace_macros.py src/nolttng_macros.py src/perfctr_macros.py
touch /home/iojs/build/workspace/node-test-commit-arm/nodes/armv7-wheezy/out/Release/obj.host/node_js2c.stamp

The output of ./configure on the two buildbots isn't hugely different. arm_fpu and arm_version are only used by V8.

--- a.txt 2016-06-30 10:39:49.475993197 +0200+++ b.txt 2016-06-30 10:40:04.387993197 +0200@@ -1,7 +1,7 @@
'variables': { 'arm_float_abi': 'hard',
- 'arm_fpu': 'vfpv3',+ 'arm_fpu': 'vfp',
'arm_thumb': 0,
- 'arm_version': '7',+ 'arm_version': '6',
'asan': 0,
'debug_devtools': 'node',
'gas_version': '2.24',

cc @nodejs/build - ideas on what might be causing that?

EDIT: To be clear, CI runs for other pull requests show the exact same behavior. It's not isolated to this pull request.

bnoordhuis · 2016-07-06T14:34:35Z

Ping @nodejs/build. Even just a 'no clue' would be appreciated. :-)

pavelfeldman · 2016-07-06T19:27:45Z

It might make sense to expose this via the protocol itself i.e. Protocol.getJSON method would return the protocol definition. That way clients could reuse same web socket connection and hence save on CORS, etc. for the separate request.

joaocgreis · 2016-07-07T11:07:34Z

On my local machine this compiles with make -j10 but fails with make -j1. make -n shows that making v8_inspector_protocol_json.h is one of the last things that make tries to do, after compiling inspector_agent.cc. What about adding 'v8_inspector_compress_protocol_json' to the dependencies in line 306 instead of the last fixup?

bnoordhuis · 2016-07-09T12:02:21Z

@joaocgreis Good suggestion, updated. CI: https://ci.nodejs.org/job/node-test-pull-request/3227/

It might make sense to expose this via the protocol itself i.e. Protocol.getJSON method would return the protocol definition. That way clients could reuse same web socket connection and hence save on CORS, etc. for the separate request.

@pavelfeldman Correct me if I'm wrong but doesn't that introduce a chicken-and-egg problem? The client won't know what protocol to speak to request the protocol JSON.

pavelfeldman · 2016-07-11T21:03:09Z

Correct me if I'm wrong but doesn't that introduce a chicken-and-egg problem?

It does, but it is manageable. There could be runtimes with different sets of capabilities available via the same endpoint. If we do use json/protocol to expose those capabilities, then it should be on the connection level in a form of RTTI. We have not implemented these capabilities yet, so it is up for debate.

jasnell · 2016-08-09T21:56:24Z

ping @bnoordhuis :-)

bnoordhuis · 2016-08-10T13:07:48Z

Let's take the debian approach: when faced with several options, do 'em all. Does publishing the protocol over both WS and HTTP sound good?

pavelfeldman · 2016-08-10T15:36:25Z

@bnoordhuis we can do /json/protocol first to cover the immediate needs. we'll follow up with the same in chromium for consistency.

bnoordhuis · 2016-08-15T17:33:04Z

Rebased, new CI: https://ci.nodejs.org/job/node-test-pull-request/3676/

Can I get a LGTM or two?

MylesBorins · 2016-08-15T17:40:54Z

nit: is there a reason for the inconsistent indent between this and the two lines above?

The previous line is a continuation of the one before it. Note how the first line doesn't have a comma at the end.

bnoordhuis · 2016-08-23T19:06:16Z

Rebased against master. Can someone sign off on this?

CI: https://ci.nodejs.org/job/node-test-pull-request/3808/

jasnell · 2016-08-23T19:08:45Z

LGTM

cjihrig · 2016-08-23T23:59:53Z

LGTM

bnoordhuis · 2016-09-11T11:06:21Z

Rebased. New CI run: https://ci.nodejs.org/job/node-test-pull-request/4008/

Barring nays or CI failures, I'll merge this tomorrow or the day after.

EDIT: Again, because "stderr: fatal: unable to access 'https://github.com/nodejs/node.git/': gnutls_handshake() failed: A TLS packet with unexpected length was received."

https://ci.nodejs.org/job/node-test-pull-request/4009/

joshgav · 2016-09-15T22:46:10Z

CI again so we can get this merged: https://ci.nodejs.org/job/node-test-pull-request/4064

EDIT: Problems on ARM and AIX which are out of my realm to troubleshoot 😢

jasnell · 2016-09-20T16:48:51Z

Ping @bnoordhuis

bnoordhuis · 2016-09-23T16:11:30Z

Rebased. CI again: https://ci.nodejs.org/job/node-test-pull-request/4240/

bnoordhuis · 2016-09-23T16:51:27Z

Known flake parallel/test-tick-processor-unknown on smartos, see #8725; everything else is green/yellow. I'm going to land this.

Embed the compressed and minified protocol.json from the bundled v8_inspector and make it available through the /json/protocol endpoint. Refs: nodejs/diagnostics#52 PR-URL: nodejs#7491 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

Embed the compressed and minified protocol.json from the bundled v8_inspector and make it available through the /json/protocol endpoint. Refs: nodejs/diagnostics#52 PR-URL: #7491 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

According to nodejs/node#7491

bnoordhuis added c++ Issues and PRs that require attention from people who are familiar with C++. inspector Issues and PRs related to the V8 inspector protocol labels Jun 29, 2016

nodejs-github-bot added the zlib Issues and PRs related to the zlib subsystem. label Jun 29, 2016

bnoordhuis mentioned this pull request Jun 29, 2016
Subset of CDP as profile for Node.js tools and VMs nodejs/diagnostics#52
Closed
1 task

addaleax reviewed Jun 29, 2016
View reviewed changes

addaleax reviewed Jun 30, 2016
View reviewed changes

saghul reviewed Jun 30, 2016
View reviewed changes

bnoordhuis force-pushed the diag52 branch from 0fec31a to 4a92a61Compare July 9, 2016 11:59

xbolshe mentioned this pull request Jul 17, 2016
arm and LD_LIBRARY_PATH (ld-linux-armhf.so.3: No such file or directory) #7770
Closed

bnoordhuis force-pushed the diag52 branch from 4a92a61 to 25e30b6Compare August 15, 2016 17:31

MylesBorins reviewed Aug 15, 2016
View reviewed changes

bnoordhuis force-pushed the diag52 branch 2 times, most recently from 68206a9 to 2655fb9Compare August 23, 2016 19:05

bnoordhuis force-pushed the diag52 branch from 2655fb9 to fe76066Compare September 11, 2016 11:03

Trott force-pushed the master branch from b0df363 to c5ce7f4Compare September 21, 2016 00:09

bnoordhuis force-pushed the diag52 branch from fe76066 to 0e54827Compare September 23, 2016 16:10

bnoordhuis force-pushed the diag52 branch from 0e54827 to 782620fCompare September 23, 2016 16:54

bnoordhuis closed this Sep 23, 2016

bnoordhuis deleted the diag52 branch September 23, 2016 16:54

bnoordhuis merged commit 782620f into nodejs:masterSep 23, 2016

MylesBorins added the dont-land-on-v4.x label Sep 30, 2016

cyrus-and added a commit to cyrus-and/chrome-remote-interface that referenced this pull request Oct 5, 2016
Add the Node.js protocol fetcher
1444a46
According to nodejs/node#7491

Uh oh!

Uh oh!

Conversation

bnoordhuis commented Jun 29, 2016• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jasnell commented Jun 30, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

addaleax commented Jun 30, 2016

Uh oh!

joshgav commented Jun 30, 2016

Uh oh!

bnoordhuis commented Jun 30, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

addaleax commented Jun 30, 2016

Uh oh!

bnoordhuis commented Jun 30, 2016• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bnoordhuis commented Jul 6, 2016

Uh oh!

pavelfeldman commented Jul 6, 2016

Uh oh!

joaocgreis commented Jul 7, 2016

Uh oh!

bnoordhuis commented Jul 9, 2016

Uh oh!

pavelfeldman commented Jul 11, 2016

Uh oh!

jasnell commented Aug 9, 2016

Uh oh!

bnoordhuis commented Aug 10, 2016

Uh oh!

pavelfeldman commented Aug 10, 2016

Uh oh!

bnoordhuis commented Aug 15, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bnoordhuis commented Aug 23, 2016

Uh oh!

jasnell commented Aug 23, 2016

Uh oh!

cjihrig commented Aug 23, 2016

Uh oh!

bnoordhuis commented Sep 11, 2016• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joshgav commented Sep 15, 2016• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jasnell commented Sep 20, 2016

Uh oh!

bnoordhuis commented Sep 23, 2016

Uh oh!

bnoordhuis commented Sep 23, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

bnoordhuis commented Jun 29, 2016•
edited
Loading

bnoordhuis commented Jun 30, 2016•
edited
Loading

bnoordhuis commented Sep 11, 2016•
edited
Loading

joshgav commented Sep 15, 2016•
edited
Loading