crypto: implement randomuuid by jasnell · Pull Request #36729 · nodejs/node

jasnell · 2021-01-02T06:01:36Z

/cc @bcoe

Signed-off-by: James M Snell jasnell@gmail.com

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

targos · 2021-01-02T06:09:40Z

Is it being added to the WebCrypto spec? I can't find any reference to this method.

jasnell · 2021-01-02T16:20:45Z

@targos:

Is it being added to the WebCrypto spec?

This is based on a proposal by @bcoe being discussed here: https://discourse.wicg.io/t/proposal-rfc4122-v4-uuids/4986. The actual proposal as it currently stands is introduction of a new global function but there's also discussion about it being exposed via Web Crypto.

ChALkeR

data[6]/data[8] is incorrect, otherwise big +1 on this!

jasnell · 2021-01-02T17:39:56Z

Updated to resolve the comments. Also batched the random data generation to cover generating up to 64 uuid's before having to generate the random data again -- yields a significant performance bump (from ~200k ops/sec up to ~1.8 million / ops/sec) at a minor memory cost.

Fixed

jasnell · 2021-01-02T17:46:46Z

CI: https://ci.nodejs.org/job/node-test-pull-request/35203/

panva · 2021-01-02T18:06:43Z

Adding to the SubtleCrypto interface is imho premature. Otherwise it's a nice addition.

ChALkeR · 2021-01-02T18:08:09Z

@jasnell how about this:

consthexDigits=[48,49,50,51,52,53,54,55,56,57,97,98,99,100,101,102];// Implements an RFC 4122 version 4 random UUID.// To improve performance, random data is generated in batches// large enough to cover 64 UUID's at a time.// The uuidData array and uuid buffer are reused for each.// Each uuid consumes 16 bytes of random data from the array.constuuidBatches=64;// a buffer for this many uuids is prefilledletuuidBatch=0;letuuidData;letuuid;functionrandomUUID(){if(uuidData===undefined){uuidData=newUint8Array(16*uuidBatches);uuid=Buffer.alloc(36,'-');}if(uuidBatch===0)getRandomValues(uuidData);constbase=16*uuidBatch;uuidBatch=(uuidBatch+1)%uuidBatches;// Version byte: 0100xxxx (version 4)uuidData[base+6]=(uuidData[base+6]&0xf)|0x40;// Variant byte: 10xxxxxx (variant 1)uuidData[base+8]=(uuidData[base+8]&0x3f)|0x80;letp=0;for(letn=base;n<base+16;n++){uuid[p++]=hexDigits[(uuidData[n]&0xf0)>>4];uuid[p++]=hexDigits[(uuidData[n]&0xf)];if(p===8||p===13||p===18||p===23)p++;}returnuuid.toString();}

Changes:

Split random data buffer from stringification buffer to improve clarity
Fixed 63 -> 64 and simplified the logic (starts with 0, use %).
I doubt there is any effect on performance, but that could be reverted if there is.
No need to set '-' each time, can pre-fill those and just skip those bytes.
No allocUnsafe()

jasnell · 2021-01-02T18:33:20Z

That works @ChALkeR and the performance is looking really good now also...

jasnell · 2021-01-02T18:36:14Z

New CI: https://ci.nodejs.org/job/node-test-pull-request/35204/

(but it looks like CI is still broken ... @nodejs/build)

jasnell · 2021-01-02T18:38:50Z

@panva:

Adding to the SubtleCrypto interface is imho premature

Well, it's exposed via require('crypto').webcrypto (at the same level as getRandomBytes() but is not on SubtleCrypto. We can mark it experimental for now giving room for us to move it around as necessary later.

panva · 2021-01-02T18:43:51Z

We can mark it experimental for now giving room for us to move it around as necessary later.

But that benefits noone - if I can have it stable in 'crypto' why would i ever require it from webcrypto, knowing it can get renamed/removed in the future. Just have it stable in crypto and if/when picked up by w3c it can be added to webcrypto.

ChALkeR · 2021-01-02T18:45:32Z

+1 to what @panva said, can we expose that on require('crypto') for now, and not on require('crypto').webcrypto, until it gets into upstream spec?

Implementation LGTM.

jasnell · 2021-01-02T18:46:24Z

But that benefits noone - if I can have it stable in 'crypto' why would i ever require it from webcrypto, knowing it can get renamed/removed in the future. Just have it stable in crypto and if/when picked up by w3c it can be added to webcrypto.

Partially because the API proposal is still being discussed and could end up being quite a bit different in the end. I'd rather not mark it stable prematurely.

panva · 2021-01-02T18:47:59Z

@jasnell, fine, but still - we can have that in node's crypto only. Imho it doesn't make sense in webcrypto until it gets picked up by the standards body maintaining that namespace.

PR-URL: #36889 Notable changes: * child_process: * add 'overlapped' stdio flag (Thiago Padilha) (#29412) * support AbortSignal in fork (Benjamin Gruenbaum) (#36603) * crypto: * implement basic secure heap support (James M Snell) (#36779) * fixup bug in keygen error handling (James M Snell) (#36779) * introduce X509Certificate API (James M Snell) (#36804) * implement randomuuid (James M Snell) (#36729) * doc: * update release key for Danielle Adams (Danielle Adams) (#36793) * add dnlup to collaborators (Daniele Belardi) (#36849) * add panva to collaborators (Filip Skokan) (#36802) * add yashLadha to collaborator (Yash Ladha) (#36666) * http: * set lifo as the default scheduling strategy in Agent (Matteo Collina) (#36685) * net: * support abortSignal in server.listen (Nitzan Uziely) (#36623) * process: * add direct access to rss without iterating pages (Adrien Maret) (#34291) * v8: * fix native constructors (ExE Boss) (#36549)

Signed-off-by: James M Snell <jasnell@gmail.com> Original-PR-URL: nodejs#36729

Signed-off-by: James M Snell <jasnell@gmail.com> Original-PR-URL: #36729 PR-URL: #36945 Reviewed-By: Ben Coe <bencoe@gmail.com>

PR-URL: nodejs#38074 Refs: nodejs#36729 Reviewed-By: Rich Trott <rtrott@gmail.com>

Signed-off-by: James M Snell <jasnell@gmail.com> PR-URL: nodejs#36729 Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Filip Skokan <panva.ip@gmail.com> Reviewed-By: Ben Coe <bencoe@gmail.com>

Signed-off-by: James M Snell <jasnell@gmail.com> PR-URL: #36729 Backport-PR-URL: #36945 Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Filip Skokan <panva.ip@gmail.com> Reviewed-By: Ben Coe <bencoe@gmail.com>

PR-URL: #38074 Refs: #36729 Reviewed-By: Rich Trott <rtrott@gmail.com>

PR-URL: #38507 Notable changes: * assert: * change status of legacy asserts (James M Snell) (#38113) * crypto: * implement randomuuid (James M Snell) (#36729) * doc: * apply style for legacy status (James M Snell) (#37784) * revoke deprecation of legacy url, change status to legacy (James M Snell) (#37784) * add legacy status to stability index (James M Snell) (#37784) * upgrade stability status of report API (Gireesh Punathil) (#35654) * deps: * V8: Backport various patches for Apple Silicon support (BoHong Li) (#38051) * update ICU to 68.1 (Michaël Zasso) (#36187) * upgrade to libuv 1.41.0 (Colin Ihrig) (#37360) * http: * add http.ClientRequest.getRawHeaderNames() (simov) (#37660) * report request start and end with diagnostics\_channel (Stephen Belanger) (#34895) * lib: * create diagnostics\_channel module (Stephen Belanger) (#34895) * implement AbortSignal.abort() (James M Snell) (#37693) * test: * update dom/abort tests (James M Snell) (#37693) * util: * add getSystemErrorMap() impl (eladkeyshawn) (#38101)

PR-URL: #38074 Refs: #36729 Reviewed-By: Rich Trott <rtrott@gmail.com>

PR-URL: #38507 Notable changes: * crypto: * implement randomuuid (James M Snell) (#36729) * doc: * apply style for legacy status (James M Snell) (#37784) * revoke deprecation of legacy url, change status to legacy (James M Snell) (#37784) * add legacy status to stability index (James M Snell) (#37784) * upgrade stability status of report API (Gireesh Punathil) (#35654) * deps: * V8: Backport various patches for Apple Silicon support (BoHong Li) (#38051) * update ICU to 68.1 (Michaël Zasso) (#36187) * upgrade to libuv 1.41.0 (Colin Ihrig) (#37360) * http: * add http.ClientRequest.getRawHeaderNames() (simov) (#37660) * report request start and end with diagnostics\_channel (Stephen Belanger) (#34895) * lib: * create diagnostics\_channel module (Stephen Belanger) (#34895) * implement AbortSignal.abort() (James M Snell) (#37693) * test: * update dom/abort tests (James M Snell) (#37693) * util: * add getSystemErrorMap() impl (eladkeyshawn) (#38101)

Notable changes: Diagnostics channel (experimental module): `diagnostics_channel` is a new experimental module that provides an API to create named channels to report arbitrary message data for diagnostics purposes. The module was initially introduced in Node.js v15.1.0 and is backported to v14.17.0 to enable testing it at a larger scale. With `diagnostics_channel`, Node.js core and module authors can publish contextual data about what they are doing at a given time. This could be the hostname and query string of a mysql query, for example. Just create a named channel with `dc.channel(name)` and call `channel.publish(data)` to send the data to any listeners to that channel. ```js const dc = require('diagnostics_channel'); const channel = dc.channel('mysql.query'); MySQL.prototype.query = function query(queryString, values, callback) { // Broadcast query information whenever a query is made channel.publish({ query: queryString, host: this.hostname, }); this.doQuery(queryString, values, callback); }; ``` Channels are like one big global event emitter but are split into separate objects to ensure they get the best performance. If nothing is listening to the channel, the publishing overhead should be as close to zero as possible. Consuming channel data is as easy as using `channel.subscribe(listener)` to run a function whenever a message is published to that channel. ```js const dc = require('diagnostics_channel'); const channel = dc.channel('mysql.query'); channel.subscribe(({ query, host }) => { console.log(`mysql query to ${host}: ${query}`); }); ``` The data captured can be used to provide context for what an app is doing at a given time. This can be used for things like augmenting tracing data, tracking network and filesystem activity, logging queries, and many other things. It's also a very useful data source for diagnostics tools to provide a clearer picture of exactly what the application is doing at a given point in the data they are presenting. Contributed by Stephen Belanger (#34895). UUID support in the crypto module: The new `crypto.randomUUID()` method now allows to generate random [RFC 4122](https://www.rfc-editor.org/rfc/rfc4122.txt) Version 4 UUID strings: ```js const { randomUUID } = require('crypto'); console.log(randomUUID()); // 'aa7c91a1-f8fc-4339-b9db-f93fc7233429' ``` Contributed by James M Snell (#36729. Experimental support for `AbortController` and `AbortSignal`: Node.js 14.17.0 adds experimental partial support for `AbortController` and `AbortSignal`. Both constructors can be enabled globally using the `--experimental-abortcontroller` flag. Additionally, several Node.js APIs have been updated to support `AbortSignal` for cancellation. It is not mandatory to use the built-in constructors with them. Any spec-compliant third-party alternatives should be compatible. `AbortSignal` support was added to the following methods: * `child_process.exec` * `child_process.execFile` * `child_process.fork` * `child_process.spawn` * `dgram.createSocket` * `events.on` * `events.once` * `fs.readFile` * `fs.watch` * `fs.writeFile` * `http.request` * `https.request` * `http2Session.request` * The promisified variants of `setImmediate` and `setTimeout` Other notable changes: * doc: * revoke deprecation of legacy url, change status to legacy (James M Snell) (#37784) * add legacy status to stability index (James M Snell) (#37784) * upgrade stability status of report API (Gireesh Punathil) (#35654) * deps: * V8: Backport various patches for Apple Silicon support (BoHong Li) (#38051) * update ICU to 68.1 (Michaël Zasso) (#36187) * upgrade to libuv 1.41.0 (Colin Ihrig) (#37360) * http: * add http.ClientRequest.getRawHeaderNames() (simov) (#37660) * report request start and end with diagnostics\_channel (Stephen Belanger) (#34895) * util: * add getSystemErrorMap() impl (eladkeyshawn) (#38101) PR-URL: #38507

Notable Changes: Diagnostics channel (experimental module): `diagnostics_channel` is a new experimental module that provides an API to create named channels to report arbitrary message data for diagnostics purposes. The module was initially introduced in Node.js v15.1.0 and is backported to v14.17.0 to enable testing it at a larger scale. With `diagnostics_channel`, Node.js core and module authors can publish contextual data about what they are doing at a given time. This could be the hostname and query string of a mysql query, for example. Just create a named channel with `dc.channel(name)` and call `channel.publish(data)` to send the data to any listeners to that channel. ```js const dc = require('diagnostics_channel'); const channel = dc.channel('mysql.query'); MySQL.prototype.query = function query(queryString, values, callback) { // Broadcast query information whenever a query is made channel.publish({ query: queryString, host: this.hostname, }); this.doQuery(queryString, values, callback); }; ``` Channels are like one big global event emitter but are split into separate objects to ensure they get the best performance. If nothing is listening to the channel, the publishing overhead should be as close to zero as possible. Consuming channel data is as easy as using `channel.subscribe(listener)` to run a function whenever a message is published to that channel. ```js const dc = require('diagnostics_channel'); const channel = dc.channel('mysql.query'); channel.subscribe(({ query, host }) => { console.log(`mysql query to ${host}: ${query}`); }); ``` The data captured can be used to provide context for what an app is doing at a given time. This can be used for things like augmenting tracing data, tracking network and filesystem activity, logging queries, and many other things. It's also a very useful data source for diagnostics tools to provide a clearer picture of exactly what the application is doing at a given point in the data they are presenting. Contributed by Stephen Belanger (#34895). UUID support in the crypto module: The new `crypto.randomUUID()` method now allows to generate random [RFC 4122](https://www.rfc-editor.org/rfc/rfc4122.txt) Version 4 UUID strings: ```js const { randomUUID } = require('crypto'); console.log(randomUUID()); // 'aa7c91a1-f8fc-4339-b9db-f93fc7233429' ``` Contributed by James M Snell (#36729). Experimental support for `AbortController` and `AbortSignal`: Node.js 14.17.0 adds experimental partial support for `AbortController` and `AbortSignal`. Both constructors can be enabled globally using the `--experimental-abortcontroller` flag. Additionally, several Node.js APIs have been updated to support `AbortSignal` for cancellation. It is not mandatory to use the built-in constructors with them. Any spec-compliant third-party alternatives should be compatible. `AbortSignal` support was added to the following methods: * `child_process.exec` * `child_process.execFile` * `child_process.fork` * `child_process.spawn` * `dgram.createSocket` * `events.on` * `events.once` * `fs.readFile` * `fs.watch` * `fs.writeFile` * `http.request` * `https.request` * `http2Session.request` * The promisified variants of `setImmediate` and `setTimeout` Other notable changes: * doc: * revoke deprecation of legacy url, change status to legacy (James M Snell) (#37784) * add legacy status to stability index (James M Snell) (#37784) * upgrade stability status of report API (Gireesh Punathil) (#35654) * deps: * V8: Backport various patches for Apple Silicon support (BoHong Li) (#38051) * update ICU to 68.1 (Michaël Zasso) (#36187) * upgrade to libuv 1.41.0 (Colin Ihrig) (#37360) * http: * add http.ClientRequest.getRawHeaderNames() (simov) (#37660) * report request start and end with diagnostics\_channel (Stephen Belanger) (#34895) * util: * add getSystemErrorMap() impl (eladkeyshawn) (#38101) PR-URL: #38507

nodejs/node#36729

nodejs-github-bot added the crypto Issues and PRs related to the crypto subsystem. label Jan 2, 2021

mscdex reviewed Jan 2, 2021
View reviewed changes

Comment threadlib/internal/crypto/random.js Outdated