feat: Autoload Database Integrations in Node environment by kamilogorek · Pull Request #3483 · getsentry/sentry-javascript

kamilogorek · 2021-04-29T14:21:45Z

This feature will auto-load DB tracing integrations whenever a related package is available in the environment.

Postgres testing snippets: #3064
Mongo testing snippets: #3072
Mysql testing snippets: #3088

github-actions · 2021-04-29T14:34:31Z

size-limit report

Path	Size
@sentry/browser - CDN Bundle (gzipped)	20.65 KB (+0.09% 🔺)
@sentry/browser - Webpack	21.52 KB (+0.1% 🔺)
@sentry/react - Webpack	21.55 KB (+0.09% 🔺)
@sentry/browser + @sentry/tracing - CDN Bundle (gzipped)	28.14 KB (+0.8% 🔺)

HazAT · 2021-05-05T10:53:13Z

I am going to pick up the PR and carry it to the 🏁

…bject

HazAT

👍

rhcarvalho

Is this going to work for all clients (the default one from init and custom clients)?
Can users disable those DB integrations the same way they can disable other integrations?

rhcarvalho · 2021-05-10T13:06:55Z

+ try {
+ mod = dynamicRequire(module, moduleName);
+ } catch (e) {
+ // no-empty
+ }
+
+ try {
+ const { cwd } = dynamicRequire(module, 'process');
+ mod = dynamicRequire(module, `${cwd()}/node_modules/${moduleName}`) as T;
+ } catch (e) {
+ // no-empty
+ }


Why does this try to assign mod twice? If the first dynamicRequire threw no exception, why don't we return mod there?

Because T | undefined return type requires us to return on every branch anyway, so the best we could do is
try{mod=dynamicRequire(module,moduleName);}catch(e){// no-empty}try{const{ cwd }=dynamicRequire(module,'process');mod=dynamicRequire(module,`${cwd()}/node_modules/${moduleName}`)asT;}catch(e){// no-empty}returnundefined;
In which case the current solution is imo more readable.

Maybe I'm reading it wrong, but if there are no exceptions, this will execute linearly like this:
mod=dynamicRequire(module,moduleName);const{ cwd }=dynamicRequire(module,'process');mod=dynamicRequire(module,`${cwd()}/node_modules/${moduleName}`)asT;
In which case mod = dynamicRequire(module, moduleName); is overwritten by mod = dynamicRequire(module, ${cwd()}/node_modules/${moduleName}) as T;.
I'd understand it better if it was
try{mod=dynamicRequire(module,moduleName);returnmod;// NEW CODE}catch(e){// no-empty}try{const{ cwd }=dynamicRequire(module,'process');mod=dynamicRequire(module,`${cwd()}/node_modules/${moduleName}`)asT;returnmod;// NEW CODE}catch(e){// no-empty}returnundefined;

We do want to override it, as if possible, we want the second value. This is to mimick 1:1 require.resolve behavior.

Ah, I see. So why don't we start with what we want first, and fallback later?
The code as is is non-obvious.
Suggested change
try{
mod=dynamicRequire(module,moduleName);
}catch(e){
// no-empty
}
try{
const{ cwd }=dynamicRequire(module,'process');
mod=dynamicRequire(module,`${cwd()}/node_modules/${moduleName}`)asT;
}catch(e){
// no-empty
}
try{
const{ cwd }=dynamicRequire(module,'process');
mod=dynamicRequire(module,`${cwd()}/node_modules/${moduleName}`)asT;
returnmod
}catch(e){
// ignore
}
try{
mod=dynamicRequire(module,moduleName);
returnmod
}catch(e){
// ignore
}

This is to mimick 1:1 require.resolve behavior.
Just saw this in the JSdoc; if we are trying to mimic also the side-effects of importing, then ignore my suggestion.

It's stylistic thing already, and I don't want to run CI over and over again :P

We’re changing in this same PR some unrelated code to return early, and here choose to do an import potentially twice :-)
The import is not only a style preference, it also contains potential side-effects. We are not profiling the code nor explicitly trying to make SDK startup as fast as possible, but we can also say that the double import also makes it overall slower.
No need to block on this, we've got 2 review approvals mine included!

kamilogorek · 2021-05-10T13:50:38Z

Is this going to work for all clients (the default one from init and custom clients)?

No, it won't work for custom clients, the same way as default integrations doesn't work for them.

Can users disable those DB integrations the same way they can disable other integrations?

Yes, with the same defaultIntegrations: false setting. Or filter them out as with regular integrations.

Co-authored-by: Rodolfo Carvalho <rodolfo.carvalho@sentry.io>

x5a · 2021-05-18T19:26:53Z

we're still investigating, but this change unexpectedly broke our apps in a very insidious and subtle way that caused a production outage - I'd encourage you to release this sort of monkey-patching-database-connectors change in a major version rather than a minor version.

HazAT · 2021-05-18T21:40:37Z

@x5a Can you provide a bit more info; How does your stack look like / how can we repro this?

x5a · 2021-05-19T02:49:17Z

@HazAT we're using postgres (pg@6.4.2) specifically in concert with pg-cursor@1.3.0. Because you monkeypatch Client.prototype.query and assume that the return value is thenable (it's not when working with https://node-postgres.com/api/cursor), we started getting an exception:

TypeError: orig.call(...).then is not a function\n' + ' at Client.query (/opt/app/node_modules/@sentry/tracing/dist/integrations/postgres.js:49:56)

which maps to: https://github.com/getsentry/sentry-javascript/blob/master/packages/tracing/src/integrations/postgres.ts#L63

Kikobeats · 2021-05-19T08:33:33Z

I opened a new issue to warn users the latest version is broken:

#3566

elliotttf · 2021-05-19T13:47:30Z

I can also confirm that this change has broken my application's postgres integration. In my case, the failure occurs when using a QueryStream via knex and pg-query-stream. The failure is due to the exact same reason that @x5a has pointed out. For example:

conststream=knex('posts').select('*').stream();stream.pipe(someWritableStream);

The resulting query object is a QueryStream and just like the cursor is not thenable. The assumption in the sentry code about the original query method being thenable fails but the error is swallowed in a not-obvious way.

kamilogorek requested review from a team, lobsterkatie and rhcarvalho and removed request for a teamApril 29, 2021 14:21

kamilogorek force-pushed the autoload-tracing-integrations branch from 147a445 to 6035456Compare April 29, 2021 14:22

kamilogorek requested a review from HazAT April 29, 2021 14:23

kamilogorekand others added 3 commits May 10, 2021 12:41

feat: Autoload Database Integrations in Node environment
fe09ce2

Remove value before deleting __SENTRY__ key
1f1ba83

ref: Attach autoloaded integrations to defaultIntegrations
5561844

kamilogorek force-pushed the autoload-tracing-integrations branch from e12e808 to 5561844Compare May 10, 2021 11:30

Remove unnecessary comment and fix attaching integrations to global o…
3cd0cdd
…bject

HazAT approved these changes May 10, 2021
View reviewed changes

rhcarvalho reviewed May 10, 2021
View reviewed changes

Comment threadpackages/hub/src/hub.ts

rhcarvalho reviewed May 10, 2021
View reviewed changes

kamilogorek commented May 10, 2021
View reviewed changes

Comment threadpackages/tracing/src/hubextensions.ts Outdated

kamilogorekand others added 2 commits May 10, 2021 15:59

Use packageToIntegrationMapping keys for iteration
869e4fe
Co-authored-by: Rodolfo Carvalho <rodolfo.carvalho@sentry.io>

Update FIXME comment for getMainCarrier and loadModule
3cc4516

rhcarvalho approved these changes May 10, 2021
View reviewed changes

kamilogorek enabled auto-merge (squash) May 10, 2021 15:19

Merge branch 'master' into autoload-tracing-integrations
3fee493

kamilogorek disabled auto-merge May 10, 2021 15:20

kamilogorek merged commit 117bc99 into masterMay 10, 2021

kamilogorek deleted the autoload-tracing-integrations branch May 10, 2021 15:41

kamilogorek mentioned this pull request May 11, 2021
ref: Node DB auto-instrumentation docs getsentry/sentry-docs#3563
Merged

Kikobeats mentioned this pull request May 19, 2021
TypeError: orig.call(...).then is not a function #3566
Closed

This was referenced Jun 10, 2021
feat: @sentry/integration-node-mongodb package #3473
Closed
feat: Auto-discover integrations #3472
Closed

andreialecu mentioned this pull request Oct 19, 2021
fix(utils): Fix loadModule when using PnP #4077
Closed
2 tasks

-try{
-mod=dynamicRequire(module,moduleName);
-}catch(e){
-// no-empty
-}
-try{
-const{ cwd }=dynamicRequire(module,'process');
-mod=dynamicRequire(module,`${cwd()}/node_modules/${moduleName}`)asT;
-}catch(e){
-// no-empty
-}
+try{
+const{ cwd }=dynamicRequire(module,'process');
+mod=dynamicRequire(module,`${cwd()}/node_modules/${moduleName}`)asT;
+returnmod
+}catch(e){
+// ignore
+}
+try{
+mod=dynamicRequire(module,moduleName);
+returnmod
+}catch(e){
+// ignore
+}

Uh oh!

Uh oh!

Conversation

kamilogorek commented Apr 29, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Apr 29, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

size-limit report

Uh oh!

HazAT commented May 5, 2021

Uh oh!

HazAT left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rhcarvalho left a comment

Choose a reason for hiding this comment

Uh oh!

rhcarvalhoMay 10, 2021

Choose a reason for hiding this comment

Uh oh!

kamilogorekMay 10, 2021

Choose a reason for hiding this comment

Uh oh!

rhcarvalhoMay 10, 2021

Choose a reason for hiding this comment

Uh oh!

kamilogorekMay 10, 2021

Choose a reason for hiding this comment

Uh oh!

rhcarvalhoMay 10, 2021

Choose a reason for hiding this comment

Uh oh!

rhcarvalhoMay 10, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kamilogorekMay 10, 2021

Choose a reason for hiding this comment

Uh oh!

rhcarvalhoMay 10, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kamilogorek commented May 10, 2021

Uh oh!

Uh oh!

x5a commented May 18, 2021

Uh oh!

HazAT commented May 18, 2021

Uh oh!

x5a commented May 19, 2021

Uh oh!

Kikobeats commented May 19, 2021

Uh oh!

elliotttf commented May 19, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

kamilogorek commented Apr 29, 2021•
edited
Loading

github-actionsBot commented Apr 29, 2021•
edited
Loading

rhcarvalhoMay 10, 2021•
edited
Loading