[v8.x] https: revert "refactor to use http internals"

[MylesBorins]

This reverts commit 5118f31 on the v8.x release line

It is breaking code in the wild that depends on the original behavior
to do tracing.

I don't think we need to necessarily fix this in 9.x but we might want
to reclassify the original commit as Semver Major

Refs: #16395

/cc @nodejs/lts @bengl@gibfahn@ofrobots

[MylesBorins]

ci: https://ci.nodejs.org/job/node-test-pull-request/11123/

[addaleax]

Is there a bug report of some kind somewhere?

Also, our release tooling is designed to handle git’s built-in revert commit messages and explicitly calls out the commits as reverts if it is used.

[MylesBorins]

@addaleax There are two comments found at #16395 (comment)

lmk if commit message looks good

[addaleax]

Yup, seems good for v8.x!

[refack]

I don't think we need to necessarily fix this in 8.x but we might want
to reclassify the original commit as Semver Major

I'm assuming you meant 9?

[MylesBorins]

@refack yeah, I just updated that in the commit message 😅

Waiting for CI to run before pushing because I don't want to restart it again

[refack]

P.S. any good candidate for CitGM in the failures?

[MylesBorins]

@addaleax@bengl are you cool with the original commit being relabeled semver major? I'll update changelogs

[bnoordhuis]

No reason to revert, IMO. We don't promise that internals won't change. People monkey-patching core APIs know and accept that.

[MylesBorins]

@bnoordhuis does it make sense to knowlingly break people in a patch for such a small optimization? Seems reasonable to give people some time to catch up

[bnoordhuis]

They'll need to update anyway, that's the responsibility you take on when you start monkey-patching core.

There's also the principle of the matter. A hard-line stance avoids future debate.

[apapirovski]

Kinda -1 on reverting this for the same reasons @bnoordhuis is. Also, I don't think semver-major really applies to internal implementation details.

[jasnell]

Our documented guidelines specifically call out the fact that we can change internals in ways that potentially break. So, yes, in this case semver-major would not apply technically. We have used in the the past, however, out of an abundance of caution to try to avoid breakage but I definitely have to say that I have sympathy for @bnoordhuis' position.

[evanlucas]

While I understand the sentiment, I disagree that we shouldn't land this. It doesn't really cost us anything and will keep us from having people that cannot update to use the LTS version of node 8 now. Just my 2 cents though.

[ofrobots]

Normally we get feedback for a change on Current for a duration before potentially breaking changes land in LTS. That didn't happen here, which means that there was no warning that this was coming down the pipe before it broke folks that are building diagnostic tooling.

On the flip side, I don't think there is a great argument that this change should go into LTS to begin with. The code review already identified that this is potentially breaking. Restricting this to 9.x+ seems appropriate to me.

[jasnell]

I'm fine with reverting in 8.x only

[cjihrig]

In most cases I strongly agree with core being able to change its own internals. In this case, I'm OK with reverting because this change was identified as possibly breaking and went straight into an LTS release with no baking period in Current.

[ofrobots]

@bnoordhuis@apapirovski This revert isn't about preventing core from being able to change internals. It is rather about not putting these changes directly into LTS without a baking period. Reverting it now helps avoid people running into breakages across the ecosystem. Does that formulation help remove your -1?

[bnoordhuis]

@ofrobots I get your point but it doesn't change mine. I won't rehash my arguments and I won't fight it to death but I feel it sets a bad precedent.

As to the argument that a revert doesn't really cost anything: it makes back-porting fixes from the master branch harder. It's not free.

[refack]

@MylesBorins ack on semver level.
The list of precedents makes a good case for formalizing this ad-hoc policy.
IMHO once we have a formal policy it's easier to draw the line in the sand on principal.

[MylesBorins]

@refack I don't believe that we should make a formal policy around reverts aside from revert often and quickly.

[MylesBorins]

Also for clarity, with the added information from this thread I see no reason for this change to be treated as Semver-Major. Semver-Patch makes the most sense

[bengl]

I'm -1 on calling this a semver-major, but +1 on the revert. I think the revert will probably help enough folks that it's worth it, but I certainly don't want to set a precedent that internal API changes are semver-major if they only break folks who are doing things with API internals.

Again: +1 on revert, simply for let's-unbreak-people purposes.

[joyeecheung]

Making it explicit: I am +1 on reverting in v8.x if users are willing to fix other their side, but we should have a better channel to coordinate these changes, before async_hooks are leaves experimental stage there will still be apm vendors sensitive to the internal changes. cc @nodejs/diagnostics

[gireeshpunathil]

I like the idea of: when arguments saturate, favor ecosystem and existing deployments.

monkey-patching seems to originate partly from the language semantics, and partly from the need for monitoring as an essential embodiment of modern workloads. I propose to address monitoring capability in the core as an important focus area.

[hybrist]

Afaik this change breaks the only reliable way to instrument outgoing HTTP request, so +1 on reverting and I'd actually say we shouldn't have this change anywhere w/o an official API to achieve the same thing.

[bnoordhuis]

Afaik this change breaks the only reliable way to instrument outgoing HTTP request

It doesn't, you just need to update your code if you were monkey-patching that method.

Matter of fact, if this change broke your tracer it wasn't catching everything in the first place - requests made directly through the ClientRequest interface.

[hybrist]

Matter of fact, if this change broke your tracer it wasn't catching everything in the first place - requests made directly through the ClientRequest interface.

Okay, maybe I should've said "most reliable way in practice I'm aware of". There's little if any code I ever ran across that was using the ClientRequest constructor directly. Anyhow, the point here is: Landing this directly in an LTS release isn't a great precedent. Thus my 👍 for a revert.

[gibfahn]

@refack I don't believe that we should make a formal policy around reverts aside from revert often and quickly.

@MylesBorins why?

[NatalieWolfe]

Matter of fact, if this change broke your tracer it wasn't catching everything in the first place - requests made directly through the ClientRequest interface.

Not necessarily. In our case we were explicitly not instrumenting the https module because the http instrumentation caught everything. The solution was very simple though, and it matters not to us if this is reverted. Heck, it's less work for me if it isn't. :)

[MylesBorins]

landed in 08b75c1

[danielkhan]

Sorry for being late to the party. I like @MylesBorins pragmatic suggestion to add tooling that removes the need to monkey patch core. I also agree with @NatalieWolfe that async hooks should and could solve that going forward. We should take that as an use case for improvement in nodejs/diagnostics#117