Fixed "publish --force" not correctly publishing for already published packages

[MasterCassim]

Currently "publish --force" does not work correctly when the deployed package is already in the registry. This is a regression from v10 where this worked correctly. The following error is thrown in line 138 of publish.js:

TypeError [ERR_INVALID_ARG_VALUE]: The argument 'path' must be a string or Uint8Array without null bytes. Received <Buffer 1f 8b 08 00 4d 45 ea 5d 04 00 ed 19 f9 af d2 30 d8 9f 4d fc 1f 2a 1a 07 ca 36 f0 8a 22 a0 c6 db 78 45 3c e3 95 b9 75 50 ...
at readFile (fs.js:295:10)
at go$readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:110:14)
at readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:107:12)
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at ret (eval at makeNodePromisifiedEval (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promisify.js:184:12), :14:23)
at upload (C:\Program Files\nodejs\node_modules\npm\lib\publish.js:138:12)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:156:22
at C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:16:12
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Function.Promise.attempt.Promise.try (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\method.js:39:29)
at otplease (C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:14:16)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:154:20
at PassThroughHandlerContext.finallyHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\finally.js:56:23)
at PassThroughHandlerContext.tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Promise._settlePromiseFromHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:517:31)
at Promise._settlePromise (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:574:18) {
code: 'ERR_INVALID_ARG_VALUE'
}

The BB.promisify(require('graceful-fs').readFile) throws an error when used with an input buffer which is used in line 156 of publish.js. Using the file path again (PR) fixes the problem and is the same behavior as in v10.

What / Why

• It is no longer possible to publish already published packages with the --force option.

References

[ljharb]

I'm confused; published versions are immutable and can't be overwritten. When does this apply?

[MasterCassim]

publish --force internally does (see source): publish which fails unpublish publish But because of the mentioned Bug, the last publish is not executed. Jordan Harband <notifications@github.com> schrieb am Fr., 6. Dez. 2019, 21:00:

…

I'm confused; published versions are immutable and can't be overwritten. When does this apply? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#565?email_source=notifications&email_token=AAIRKF6VCN5V6IBBN6WM5KLQXKVNDA5CNFSM4JWWLFEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGFGECY#issuecomment-562717195>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIRKFYCAVKW5UNQWZIAHW3QXKVNDANCNFSM4JWWLFEA> .

[ljharb]

Right, but even when unpublishing succeeds, because it's within the short window where that's possible, you can never republish the same version ever again even after it's unpublished.

[MasterCassim]

But isn't that exactly what the code in publish,js does? Call libpub -> In case of EPUBLISHCONFLICT calls libunpub, then calls upload again which then calls libpub. However, the second libpub is never called because of the mentioned bug (which worked in v10) although the second upload is actually called.

function upload (pkg, isRetry, cached, opts) {
if (!opts.dryRun) {
return readFileAsync(cached).then(tarball => {
return otplease(opts, opts => {
return libpub(pkg, tarball, opts)
}).catch(err => {
if (
err.code === 'EPUBLISHCONFLICT' &&
opts.force &&
!isRetry
) {
log.warn('publish', 'Forced publish over ' + pkg._id)
return otplease(opts, opts => libunpub(
npa.resolve(pkg.name, pkg.version), opts
)).finally(() => {
// ignore errors. Use the force. Reach out with your feelings.
return otplease(opts, opts => {
return upload(pkg, true, cached, opts)
}).catch(() => {
// but if it fails again, then report the first error.
throw err
})
})
} else {
throw err
}
})
})
} else {
return opts.Promise.resolve(true)
}
}

Even if not supported by the default npm registry; we are using nexus repository which allows republishing of the same version. Although other local repositories (verdaccio) also allow republish of the same version.

[ljharb]

Gotcha; that's probably why nobody noticed, since i think most people just take it for granted that published things are immutable now (there's lots of security and maintenance reasons to have them so). Sorry for the confusion.

[MasterCassim]

No problem. We have a pretty complicated setup which just recently included npm modules as well. Because we are coming from maven (java) world where our version number only changes for external releases; we implemented the same for our new npm modules.

There were some hurdles but in the end, this worked correctly with nodejs 10 but broke once we upgraded to nodejs 12.

[isaacs]

I think this should be replacing upload with libpub, not replacing tarball with cached. Note that the function it's retrying is libpub, not upload?

We need a test in any event, and probably some more investigation.

[MasterCassim]

Currently "publish --force" does not work correctly when the deployed package is already in the registry. This is a regression from v10 where this worked correctly. The following error is thrown in line 138 of publish.js:

TypeError [ERR_INVALID_ARG_VALUE]: The argument 'path' must be a string or Uint8Array without null bytes. Received <Buffer 1f 8b 08 00 4d 45 ea 5d 04 00 ed 19 f9 af d2 30 d8 9f 4d fc 1f 2a 1a 07 ca 36 f0 8a 22 a0 c6 db 78 45 3c e3 95 b9 75 50 ...
at readFile (fs.js:295:10)
at go$readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:110:14)
at readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:107:12)
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at ret (eval at makeNodePromisifiedEval (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promisify.js:184:12), :14:23)
at upload (C:\Program Files\nodejs\node_modules\npm\lib\publish.js:138:12)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:156:22
at C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:16:12
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Function.Promise.attempt.Promise.try (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\method.js:39:29)
at otplease (C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:14:16)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:154:20
at PassThroughHandlerContext.finallyHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\finally.js:56:23)
at PassThroughHandlerContext.tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Promise._settlePromiseFromHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:517:31)
at Promise._settlePromise (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:574:18) {
code: 'ERR_INVALID_ARG_VALUE'
}

The BB.promisify(require('graceful-fs').readFile) throws an error when used with an input buffer which is used in line 156 of publish.js. Using the file path again (PR) fixes the problem and is the same behavior as in v10.

What / Why

• It is no longer possible to publish already published packages with the --force option.

References

[ljharb]

I'm confused; published versions are immutable and can't be overwritten. When does this apply?

[MasterCassim]

publish --force internally does (see source): publish which fails unpublish publish But because of the mentioned Bug, the last publish is not executed. Jordan Harband <notifications@github.com> schrieb am Fr., 6. Dez. 2019, 21:00:

…

I'm confused; published versions are immutable and can't be overwritten. When does this apply? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#565?email_source=notifications&email_token=AAIRKF6VCN5V6IBBN6WM5KLQXKVNDA5CNFSM4JWWLFEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGFGECY#issuecomment-562717195>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIRKFYCAVKW5UNQWZIAHW3QXKVNDANCNFSM4JWWLFEA> .

[ljharb]

Right, but even when unpublishing succeeds, because it's within the short window where that's possible, you can never republish the same version ever again even after it's unpublished.

[MasterCassim]

But isn't that exactly what the code in publish,js does? Call libpub -> In case of EPUBLISHCONFLICT calls libunpub, then calls upload again which then calls libpub. However, the second libpub is never called because of the mentioned bug (which worked in v10) although the second upload is actually called.

function upload (pkg, isRetry, cached, opts) {
if (!opts.dryRun) {
return readFileAsync(cached).then(tarball => {
return otplease(opts, opts => {
return libpub(pkg, tarball, opts)
}).catch(err => {
if (
err.code === 'EPUBLISHCONFLICT' &&
opts.force &&
!isRetry
) {
log.warn('publish', 'Forced publish over ' + pkg._id)
return otplease(opts, opts => libunpub(
npa.resolve(pkg.name, pkg.version), opts
)).finally(() => {
// ignore errors. Use the force. Reach out with your feelings.
return otplease(opts, opts => {
return upload(pkg, true, cached, opts)
}).catch(() => {
// but if it fails again, then report the first error.
throw err
})
})
} else {
throw err
}
})
})
} else {
return opts.Promise.resolve(true)
}
}

Even if not supported by the default npm registry; we are using nexus repository which allows republishing of the same version. Although other local repositories (verdaccio) also allow republish of the same version.

[ljharb]

Gotcha; that's probably why nobody noticed, since i think most people just take it for granted that published things are immutable now (there's lots of security and maintenance reasons to have them so). Sorry for the confusion.

[MasterCassim]

No problem. We have a pretty complicated setup which just recently included npm modules as well. Because we are coming from maven (java) world where our version number only changes for external releases; we implemented the same for our new npm modules.

There were some hurdles but in the end, this worked correctly with nodejs 10 but broke once we upgraded to nodejs 12.

[isaacs]

I think this should be replacing upload with libpub, not replacing tarball with cached. Note that the function it's retrying is libpub, not upload?

We need a test in any event, and probably some more investigation.

[MasterCassim]

Currently "publish --force" does not work correctly when the deployed package is already in the registry. This is a regression from v10 where this worked correctly. The following error is thrown in line 138 of publish.js:

TypeError [ERR_INVALID_ARG_VALUE]: The argument 'path' must be a string or Uint8Array without null bytes. Received <Buffer 1f 8b 08 00 4d 45 ea 5d 04 00 ed 19 f9 af d2 30 d8 9f 4d fc 1f 2a 1a 07 ca 36 f0 8a 22 a0 c6 db 78 45 3c e3 95 b9 75 50 ...
at readFile (fs.js:295:10)
at go$readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:110:14)
at readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:107:12)
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at ret (eval at makeNodePromisifiedEval (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promisify.js:184:12), :14:23)
at upload (C:\Program Files\nodejs\node_modules\npm\lib\publish.js:138:12)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:156:22
at C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:16:12
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Function.Promise.attempt.Promise.try (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\method.js:39:29)
at otplease (C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:14:16)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:154:20
at PassThroughHandlerContext.finallyHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\finally.js:56:23)
at PassThroughHandlerContext.tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Promise._settlePromiseFromHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:517:31)
at Promise._settlePromise (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:574:18) {
code: 'ERR_INVALID_ARG_VALUE'
}

The BB.promisify(require('graceful-fs').readFile) throws an error when used with an input buffer which is used in line 156 of publish.js. Using the file path again (PR) fixes the problem and is the same behavior as in v10.

What / Why

• It is no longer possible to publish already published packages with the --force option.

References

[ljharb]

I'm confused; published versions are immutable and can't be overwritten. When does this apply?

[MasterCassim]

publish --force internally does (see source): publish which fails unpublish publish But because of the mentioned Bug, the last publish is not executed. Jordan Harband <notifications@github.com> schrieb am Fr., 6. Dez. 2019, 21:00:

…

I'm confused; published versions are immutable and can't be overwritten. When does this apply? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#565?email_source=notifications&email_token=AAIRKF6VCN5V6IBBN6WM5KLQXKVNDA5CNFSM4JWWLFEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGFGECY#issuecomment-562717195>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIRKFYCAVKW5UNQWZIAHW3QXKVNDANCNFSM4JWWLFEA> .

[ljharb]

Right, but even when unpublishing succeeds, because it's within the short window where that's possible, you can never republish the same version ever again even after it's unpublished.

[MasterCassim]

But isn't that exactly what the code in publish,js does? Call libpub -> In case of EPUBLISHCONFLICT calls libunpub, then calls upload again which then calls libpub. However, the second libpub is never called because of the mentioned bug (which worked in v10) although the second upload is actually called.

function upload (pkg, isRetry, cached, opts) {
if (!opts.dryRun) {
return readFileAsync(cached).then(tarball => {
return otplease(opts, opts => {
return libpub(pkg, tarball, opts)
}).catch(err => {
if (
err.code === 'EPUBLISHCONFLICT' &&
opts.force &&
!isRetry
) {
log.warn('publish', 'Forced publish over ' + pkg._id)
return otplease(opts, opts => libunpub(
npa.resolve(pkg.name, pkg.version), opts
)).finally(() => {
// ignore errors. Use the force. Reach out with your feelings.
return otplease(opts, opts => {
return upload(pkg, true, cached, opts)
}).catch(() => {
// but if it fails again, then report the first error.
throw err
})
})
} else {
throw err
}
})
})
} else {
return opts.Promise.resolve(true)
}
}

Even if not supported by the default npm registry; we are using nexus repository which allows republishing of the same version. Although other local repositories (verdaccio) also allow republish of the same version.

[ljharb]

Gotcha; that's probably why nobody noticed, since i think most people just take it for granted that published things are immutable now (there's lots of security and maintenance reasons to have them so). Sorry for the confusion.

[MasterCassim]

No problem. We have a pretty complicated setup which just recently included npm modules as well. Because we are coming from maven (java) world where our version number only changes for external releases; we implemented the same for our new npm modules.

There were some hurdles but in the end, this worked correctly with nodejs 10 but broke once we upgraded to nodejs 12.

[isaacs]

I think this should be replacing upload with libpub, not replacing tarball with cached. Note that the function it's retrying is libpub, not upload?

We need a test in any event, and probably some more investigation.

[MasterCassim]

Currently "publish --force" does not work correctly when the deployed package is already in the registry. This is a regression from v10 where this worked correctly. The following error is thrown in line 138 of publish.js:

TypeError [ERR_INVALID_ARG_VALUE]: The argument 'path' must be a string or Uint8Array without null bytes. Received <Buffer 1f 8b 08 00 4d 45 ea 5d 04 00 ed 19 f9 af d2 30 d8 9f 4d fc 1f 2a 1a 07 ca 36 f0 8a 22 a0 c6 db 78 45 3c e3 95 b9 75 50 ...
at readFile (fs.js:295:10)
at go$readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:110:14)
at readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:107:12)
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at ret (eval at makeNodePromisifiedEval (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promisify.js:184:12), :14:23)
at upload (C:\Program Files\nodejs\node_modules\npm\lib\publish.js:138:12)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:156:22
at C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:16:12
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Function.Promise.attempt.Promise.try (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\method.js:39:29)
at otplease (C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:14:16)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:154:20
at PassThroughHandlerContext.finallyHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\finally.js:56:23)
at PassThroughHandlerContext.tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Promise._settlePromiseFromHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:517:31)
at Promise._settlePromise (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:574:18) {
code: 'ERR_INVALID_ARG_VALUE'
}

The BB.promisify(require('graceful-fs').readFile) throws an error when used with an input buffer which is used in line 156 of publish.js. Using the file path again (PR) fixes the problem and is the same behavior as in v10.

What / Why

• It is no longer possible to publish already published packages with the --force option.

References

[ljharb]

I'm confused; published versions are immutable and can't be overwritten. When does this apply?

[MasterCassim]

publish --force internally does (see source): publish which fails unpublish publish But because of the mentioned Bug, the last publish is not executed. Jordan Harband <notifications@github.com> schrieb am Fr., 6. Dez. 2019, 21:00:

…

I'm confused; published versions are immutable and can't be overwritten. When does this apply? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#565?email_source=notifications&email_token=AAIRKF6VCN5V6IBBN6WM5KLQXKVNDA5CNFSM4JWWLFEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGFGECY#issuecomment-562717195>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIRKFYCAVKW5UNQWZIAHW3QXKVNDANCNFSM4JWWLFEA> .

[ljharb]

Right, but even when unpublishing succeeds, because it's within the short window where that's possible, you can never republish the same version ever again even after it's unpublished.

[MasterCassim]

But isn't that exactly what the code in publish,js does? Call libpub -> In case of EPUBLISHCONFLICT calls libunpub, then calls upload again which then calls libpub. However, the second libpub is never called because of the mentioned bug (which worked in v10) although the second upload is actually called.

function upload (pkg, isRetry, cached, opts) {
if (!opts.dryRun) {
return readFileAsync(cached).then(tarball => {
return otplease(opts, opts => {
return libpub(pkg, tarball, opts)
}).catch(err => {
if (
err.code === 'EPUBLISHCONFLICT' &&
opts.force &&
!isRetry
) {
log.warn('publish', 'Forced publish over ' + pkg._id)
return otplease(opts, opts => libunpub(
npa.resolve(pkg.name, pkg.version), opts
)).finally(() => {
// ignore errors. Use the force. Reach out with your feelings.
return otplease(opts, opts => {
return upload(pkg, true, cached, opts)
}).catch(() => {
// but if it fails again, then report the first error.
throw err
})
})
} else {
throw err
}
})
})
} else {
return opts.Promise.resolve(true)
}
}

Even if not supported by the default npm registry; we are using nexus repository which allows republishing of the same version. Although other local repositories (verdaccio) also allow republish of the same version.

[ljharb]

Gotcha; that's probably why nobody noticed, since i think most people just take it for granted that published things are immutable now (there's lots of security and maintenance reasons to have them so). Sorry for the confusion.

[MasterCassim]

No problem. We have a pretty complicated setup which just recently included npm modules as well. Because we are coming from maven (java) world where our version number only changes for external releases; we implemented the same for our new npm modules.

There were some hurdles but in the end, this worked correctly with nodejs 10 but broke once we upgraded to nodejs 12.

[isaacs]

I think this should be replacing upload with libpub, not replacing tarball with cached. Note that the function it's retrying is libpub, not upload?

We need a test in any event, and probably some more investigation.

[MasterCassim]

Currently "publish --force" does not work correctly when the deployed package is already in the registry. This is a regression from v10 where this worked correctly. The following error is thrown in line 138 of publish.js:

TypeError [ERR_INVALID_ARG_VALUE]: The argument 'path' must be a string or Uint8Array without null bytes. Received <Buffer 1f 8b 08 00 4d 45 ea 5d 04 00 ed 19 f9 af d2 30 d8 9f 4d fc 1f 2a 1a 07 ca 36 f0 8a 22 a0 c6 db 78 45 3c e3 95 b9 75 50 ...
at readFile (fs.js:295:10)
at go$readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:110:14)
at readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:107:12)
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at ret (eval at makeNodePromisifiedEval (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promisify.js:184:12), :14:23)
at upload (C:\Program Files\nodejs\node_modules\npm\lib\publish.js:138:12)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:156:22
at C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:16:12
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Function.Promise.attempt.Promise.try (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\method.js:39:29)
at otplease (C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:14:16)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:154:20
at PassThroughHandlerContext.finallyHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\finally.js:56:23)
at PassThroughHandlerContext.tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Promise._settlePromiseFromHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:517:31)
at Promise._settlePromise (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:574:18) {
code: 'ERR_INVALID_ARG_VALUE'
}

The BB.promisify(require('graceful-fs').readFile) throws an error when used with an input buffer which is used in line 156 of publish.js. Using the file path again (PR) fixes the problem and is the same behavior as in v10.

What / Why

• It is no longer possible to publish already published packages with the --force option.

References

[ljharb]

I'm confused; published versions are immutable and can't be overwritten. When does this apply?

[MasterCassim]

publish --force internally does (see source): publish which fails unpublish publish But because of the mentioned Bug, the last publish is not executed. Jordan Harband <notifications@github.com> schrieb am Fr., 6. Dez. 2019, 21:00:

…

I'm confused; published versions are immutable and can't be overwritten. When does this apply? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#565?email_source=notifications&email_token=AAIRKF6VCN5V6IBBN6WM5KLQXKVNDA5CNFSM4JWWLFEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGFGECY#issuecomment-562717195>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIRKFYCAVKW5UNQWZIAHW3QXKVNDANCNFSM4JWWLFEA> .

[ljharb]

Right, but even when unpublishing succeeds, because it's within the short window where that's possible, you can never republish the same version ever again even after it's unpublished.

[MasterCassim]

But isn't that exactly what the code in publish,js does? Call libpub -> In case of EPUBLISHCONFLICT calls libunpub, then calls upload again which then calls libpub. However, the second libpub is never called because of the mentioned bug (which worked in v10) although the second upload is actually called.

function upload (pkg, isRetry, cached, opts) {
if (!opts.dryRun) {
return readFileAsync(cached).then(tarball => {
return otplease(opts, opts => {
return libpub(pkg, tarball, opts)
}).catch(err => {
if (
err.code === 'EPUBLISHCONFLICT' &&
opts.force &&
!isRetry
) {
log.warn('publish', 'Forced publish over ' + pkg._id)
return otplease(opts, opts => libunpub(
npa.resolve(pkg.name, pkg.version), opts
)).finally(() => {
// ignore errors. Use the force. Reach out with your feelings.
return otplease(opts, opts => {
return upload(pkg, true, cached, opts)
}).catch(() => {
// but if it fails again, then report the first error.
throw err
})
})
} else {
throw err
}
})
})
} else {
return opts.Promise.resolve(true)
}
}

Even if not supported by the default npm registry; we are using nexus repository which allows republishing of the same version. Although other local repositories (verdaccio) also allow republish of the same version.

[ljharb]

Gotcha; that's probably why nobody noticed, since i think most people just take it for granted that published things are immutable now (there's lots of security and maintenance reasons to have them so). Sorry for the confusion.

[MasterCassim]

No problem. We have a pretty complicated setup which just recently included npm modules as well. Because we are coming from maven (java) world where our version number only changes for external releases; we implemented the same for our new npm modules.

There were some hurdles but in the end, this worked correctly with nodejs 10 but broke once we upgraded to nodejs 12.

[isaacs]

I think this should be replacing upload with libpub, not replacing tarball with cached. Note that the function it's retrying is libpub, not upload?

We need a test in any event, and probably some more investigation.

[MasterCassim]

Currently "publish --force" does not work correctly when the deployed package is already in the registry. This is a regression from v10 where this worked correctly. The following error is thrown in line 138 of publish.js:

TypeError [ERR_INVALID_ARG_VALUE]: The argument 'path' must be a string or Uint8Array without null bytes. Received <Buffer 1f 8b 08 00 4d 45 ea 5d 04 00 ed 19 f9 af d2 30 d8 9f 4d fc 1f 2a 1a 07 ca 36 f0 8a 22 a0 c6 db 78 45 3c e3 95 b9 75 50 ...
at readFile (fs.js:295:10)
at go$readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:110:14)
at readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:107:12)
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at ret (eval at makeNodePromisifiedEval (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promisify.js:184:12), :14:23)
at upload (C:\Program Files\nodejs\node_modules\npm\lib\publish.js:138:12)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:156:22
at C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:16:12
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Function.Promise.attempt.Promise.try (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\method.js:39:29)
at otplease (C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:14:16)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:154:20
at PassThroughHandlerContext.finallyHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\finally.js:56:23)
at PassThroughHandlerContext.tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Promise._settlePromiseFromHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:517:31)
at Promise._settlePromise (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:574:18) {
code: 'ERR_INVALID_ARG_VALUE'
}

The BB.promisify(require('graceful-fs').readFile) throws an error when used with an input buffer which is used in line 156 of publish.js. Using the file path again (PR) fixes the problem and is the same behavior as in v10.

What / Why

• It is no longer possible to publish already published packages with the --force option.

References

[ljharb]

I'm confused; published versions are immutable and can't be overwritten. When does this apply?

[MasterCassim]

publish --force internally does (see source): publish which fails unpublish publish But because of the mentioned Bug, the last publish is not executed. Jordan Harband <notifications@github.com> schrieb am Fr., 6. Dez. 2019, 21:00:

…

I'm confused; published versions are immutable and can't be overwritten. When does this apply? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#565?email_source=notifications&email_token=AAIRKF6VCN5V6IBBN6WM5KLQXKVNDA5CNFSM4JWWLFEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGFGECY#issuecomment-562717195>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIRKFYCAVKW5UNQWZIAHW3QXKVNDANCNFSM4JWWLFEA> .

[ljharb]

Right, but even when unpublishing succeeds, because it's within the short window where that's possible, you can never republish the same version ever again even after it's unpublished.

[MasterCassim]

But isn't that exactly what the code in publish,js does? Call libpub -> In case of EPUBLISHCONFLICT calls libunpub, then calls upload again which then calls libpub. However, the second libpub is never called because of the mentioned bug (which worked in v10) although the second upload is actually called.

function upload (pkg, isRetry, cached, opts) {
if (!opts.dryRun) {
return readFileAsync(cached).then(tarball => {
return otplease(opts, opts => {
return libpub(pkg, tarball, opts)
}).catch(err => {
if (
err.code === 'EPUBLISHCONFLICT' &&
opts.force &&
!isRetry
) {
log.warn('publish', 'Forced publish over ' + pkg._id)
return otplease(opts, opts => libunpub(
npa.resolve(pkg.name, pkg.version), opts
)).finally(() => {
// ignore errors. Use the force. Reach out with your feelings.
return otplease(opts, opts => {
return upload(pkg, true, cached, opts)
}).catch(() => {
// but if it fails again, then report the first error.
throw err
})
})
} else {
throw err
}
})
})
} else {
return opts.Promise.resolve(true)
}
}

Even if not supported by the default npm registry; we are using nexus repository which allows republishing of the same version. Although other local repositories (verdaccio) also allow republish of the same version.

[ljharb]

Gotcha; that's probably why nobody noticed, since i think most people just take it for granted that published things are immutable now (there's lots of security and maintenance reasons to have them so). Sorry for the confusion.

[MasterCassim]

No problem. We have a pretty complicated setup which just recently included npm modules as well. Because we are coming from maven (java) world where our version number only changes for external releases; we implemented the same for our new npm modules.

There were some hurdles but in the end, this worked correctly with nodejs 10 but broke once we upgraded to nodejs 12.

[isaacs]

I think this should be replacing upload with libpub, not replacing tarball with cached. Note that the function it's retrying is libpub, not upload?

We need a test in any event, and probably some more investigation.

[MasterCassim]

Currently "publish --force" does not work correctly when the deployed package is already in the registry. This is a regression from v10 where this worked correctly. The following error is thrown in line 138 of publish.js:

TypeError [ERR_INVALID_ARG_VALUE]: The argument 'path' must be a string or Uint8Array without null bytes. Received <Buffer 1f 8b 08 00 4d 45 ea 5d 04 00 ed 19 f9 af d2 30 d8 9f 4d fc 1f 2a 1a 07 ca 36 f0 8a 22 a0 c6 db 78 45 3c e3 95 b9 75 50 ...
at readFile (fs.js:295:10)
at go$readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:110:14)
at readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:107:12)
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at ret (eval at makeNodePromisifiedEval (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promisify.js:184:12), :14:23)
at upload (C:\Program Files\nodejs\node_modules\npm\lib\publish.js:138:12)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:156:22
at C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:16:12
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Function.Promise.attempt.Promise.try (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\method.js:39:29)
at otplease (C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:14:16)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:154:20
at PassThroughHandlerContext.finallyHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\finally.js:56:23)
at PassThroughHandlerContext.tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Promise._settlePromiseFromHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:517:31)
at Promise._settlePromise (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:574:18) {
code: 'ERR_INVALID_ARG_VALUE'
}

The BB.promisify(require('graceful-fs').readFile) throws an error when used with an input buffer which is used in line 156 of publish.js. Using the file path again (PR) fixes the problem and is the same behavior as in v10.

What / Why

• It is no longer possible to publish already published packages with the --force option.

References

[ljharb]

I'm confused; published versions are immutable and can't be overwritten. When does this apply?

[MasterCassim]

publish --force internally does (see source): publish which fails unpublish publish But because of the mentioned Bug, the last publish is not executed. Jordan Harband <notifications@github.com> schrieb am Fr., 6. Dez. 2019, 21:00:

…

I'm confused; published versions are immutable and can't be overwritten. When does this apply? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#565?email_source=notifications&email_token=AAIRKF6VCN5V6IBBN6WM5KLQXKVNDA5CNFSM4JWWLFEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGFGECY#issuecomment-562717195>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIRKFYCAVKW5UNQWZIAHW3QXKVNDANCNFSM4JWWLFEA> .

[ljharb]

Right, but even when unpublishing succeeds, because it's within the short window where that's possible, you can never republish the same version ever again even after it's unpublished.

[MasterCassim]

But isn't that exactly what the code in publish,js does? Call libpub -> In case of EPUBLISHCONFLICT calls libunpub, then calls upload again which then calls libpub. However, the second libpub is never called because of the mentioned bug (which worked in v10) although the second upload is actually called.

function upload (pkg, isRetry, cached, opts) {
if (!opts.dryRun) {
return readFileAsync(cached).then(tarball => {
return otplease(opts, opts => {
return libpub(pkg, tarball, opts)
}).catch(err => {
if (
err.code === 'EPUBLISHCONFLICT' &&
opts.force &&
!isRetry
) {
log.warn('publish', 'Forced publish over ' + pkg._id)
return otplease(opts, opts => libunpub(
npa.resolve(pkg.name, pkg.version), opts
)).finally(() => {
// ignore errors. Use the force. Reach out with your feelings.
return otplease(opts, opts => {
return upload(pkg, true, cached, opts)
}).catch(() => {
// but if it fails again, then report the first error.
throw err
})
})
} else {
throw err
}
})
})
} else {
return opts.Promise.resolve(true)
}
}

Even if not supported by the default npm registry; we are using nexus repository which allows republishing of the same version. Although other local repositories (verdaccio) also allow republish of the same version.

[ljharb]

Gotcha; that's probably why nobody noticed, since i think most people just take it for granted that published things are immutable now (there's lots of security and maintenance reasons to have them so). Sorry for the confusion.

[MasterCassim]

No problem. We have a pretty complicated setup which just recently included npm modules as well. Because we are coming from maven (java) world where our version number only changes for external releases; we implemented the same for our new npm modules.

There were some hurdles but in the end, this worked correctly with nodejs 10 but broke once we upgraded to nodejs 12.

[isaacs]

I think this should be replacing upload with libpub, not replacing tarball with cached. Note that the function it's retrying is libpub, not upload?

We need a test in any event, and probably some more investigation.

[MasterCassim]

Currently "publish --force" does not work correctly when the deployed package is already in the registry. This is a regression from v10 where this worked correctly. The following error is thrown in line 138 of publish.js:

TypeError [ERR_INVALID_ARG_VALUE]: The argument 'path' must be a string or Uint8Array without null bytes. Received <Buffer 1f 8b 08 00 4d 45 ea 5d 04 00 ed 19 f9 af d2 30 d8 9f 4d fc 1f 2a 1a 07 ca 36 f0 8a 22 a0 c6 db 78 45 3c e3 95 b9 75 50 ...
at readFile (fs.js:295:10)
at go$readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:110:14)
at readFile (C:\Program Files\nodejs\node_modules\npm\node_modules\graceful-fs\graceful-fs.js:107:12)
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at ret (eval at makeNodePromisifiedEval (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promisify.js:184:12), :14:23)
at upload (C:\Program Files\nodejs\node_modules\npm\lib\publish.js:138:12)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:156:22
at C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:16:12
at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Function.Promise.attempt.Promise.try (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\method.js:39:29)
at otplease (C:\Program Files\nodejs\node_modules\npm\lib\utils\otplease.js:14:16)
at C:\Program Files\nodejs\node_modules\npm\lib\publish.js:154:20
at PassThroughHandlerContext.finallyHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\finally.js:56:23)
at PassThroughHandlerContext.tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
at Promise._settlePromiseFromHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:517:31)
at Promise._settlePromise (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:574:18) {
code: 'ERR_INVALID_ARG_VALUE'
}

The BB.promisify(require('graceful-fs').readFile) throws an error when used with an input buffer which is used in line 156 of publish.js. Using the file path again (PR) fixes the problem and is the same behavior as in v10.

What / Why

• It is no longer possible to publish already published packages with the --force option.

References

[ljharb]

I'm confused; published versions are immutable and can't be overwritten. When does this apply?

[MasterCassim]

publish --force internally does (see source): publish which fails unpublish publish But because of the mentioned Bug, the last publish is not executed. Jordan Harband <notifications@github.com> schrieb am Fr., 6. Dez. 2019, 21:00:

…

I'm confused; published versions are immutable and can't be overwritten. When does this apply? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#565?email_source=notifications&email_token=AAIRKF6VCN5V6IBBN6WM5KLQXKVNDA5CNFSM4JWWLFEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGFGECY#issuecomment-562717195>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIRKFYCAVKW5UNQWZIAHW3QXKVNDANCNFSM4JWWLFEA> .

[ljharb]

Right, but even when unpublishing succeeds, because it's within the short window where that's possible, you can never republish the same version ever again even after it's unpublished.

[MasterCassim]

But isn't that exactly what the code in publish,js does? Call libpub -> In case of EPUBLISHCONFLICT calls libunpub, then calls upload again which then calls libpub. However, the second libpub is never called because of the mentioned bug (which worked in v10) although the second upload is actually called.

function upload (pkg, isRetry, cached, opts) {
if (!opts.dryRun) {
return readFileAsync(cached).then(tarball => {
return otplease(opts, opts => {
return libpub(pkg, tarball, opts)
}).catch(err => {
if (
err.code === 'EPUBLISHCONFLICT' &&
opts.force &&
!isRetry
) {
log.warn('publish', 'Forced publish over ' + pkg._id)
return otplease(opts, opts => libunpub(
npa.resolve(pkg.name, pkg.version), opts
)).finally(() => {
// ignore errors. Use the force. Reach out with your feelings.
return otplease(opts, opts => {
return upload(pkg, true, cached, opts)
}).catch(() => {
// but if it fails again, then report the first error.
throw err
})
})
} else {
throw err
}
})
})
} else {
return opts.Promise.resolve(true)
}
}

Even if not supported by the default npm registry; we are using nexus repository which allows republishing of the same version. Although other local repositories (verdaccio) also allow republish of the same version.

[ljharb]

Gotcha; that's probably why nobody noticed, since i think most people just take it for granted that published things are immutable now (there's lots of security and maintenance reasons to have them so). Sorry for the confusion.

[MasterCassim]

No problem. We have a pretty complicated setup which just recently included npm modules as well. Because we are coming from maven (java) world where our version number only changes for external releases; we implemented the same for our new npm modules.

There were some hurdles but in the end, this worked correctly with nodejs 10 but broke once we upgraded to nodejs 12.

[isaacs]

I think this should be replacing upload with libpub, not replacing tarball with cached. Note that the function it's retrying is libpub, not upload?

We need a test in any event, and probably some more investigation.