Build packages with `tsup` by Mrtenz · Pull Request #2120 · MetaMask/snaps

Mrtenz · 2024-01-22T14:55:31Z

This changes all packages to be built with tsup, instead of SWC. tsup uses esbuild under the hood, so performance should be comparable.

More context here: MetaMask/utils#144.

socket-security · 2024-01-22T14:56:42Z

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package	New capabilities	Transitives	Size	Publisher
npm/@esbuild/aix-ppc64@0.19.12	None	`0`	10.4 MB	evanw
npm/@esbuild/android-arm@0.19.12	None	`0`	11.5 MB	evanw
npm/@esbuild/android-arm64@0.19.12	None	`0`	9.83 MB	evanw
npm/@esbuild/android-x64@0.19.12	None	`0`	11.5 MB	evanw
npm/@esbuild/darwin-arm64@0.19.12	None	`0`	9.76 MB	evanw
npm/@esbuild/darwin-x64@0.19.12	None	`0`	10.2 MB	evanw
npm/@esbuild/freebsd-arm64@0.19.12	None	`0`	8.85 MB	evanw
npm/@esbuild/freebsd-x64@0.19.12	None	`0`	9.61 MB	evanw
npm/@esbuild/linux-arm@0.19.12	None	`0`	9.18 MB	evanw
npm/@esbuild/linux-arm64@0.19.12	None	`0`	8.85 MB	evanw
npm/@esbuild/linux-ia32@0.19.12	None	`0`	9.14 MB	evanw
npm/@esbuild/linux-loong64@0.19.12	None	`0`	9.37 MB	evanw
npm/@esbuild/linux-mips64el@0.19.12	None	`0`	10.4 MB	evanw
npm/@esbuild/linux-ppc64@0.19.12	None	`0`	9.18 MB	evanw
npm/@esbuild/linux-riscv64@0.19.12	None	`0`	9.11 MB	evanw
npm/@esbuild/linux-s390x@0.19.12	None	`0`	10 MB	evanw
npm/@esbuild/linux-x64@0.19.12	None	`0`	9.6 MB	evanw
npm/@esbuild/netbsd-x64@0.19.12	None	`0`	9.59 MB	evanw
npm/@esbuild/openbsd-x64@0.19.12	None	`0`	9.62 MB	evanw
npm/@esbuild/sunos-x64@0.19.12	None	`0`	9.59 MB	evanw
npm/@esbuild/win32-arm64@0.19.12	None	`0`	8.98 MB	evanw
npm/@esbuild/win32-ia32@0.19.12	None	`0`	9.45 MB	evanw
npm/@esbuild/win32-x64@0.19.12	None	`0`	9.81 MB	evanw
npm/@rollup/rollup-android-arm-eabi@4.12.0	None	`0`	1.63 MB	lukastaegert
npm/@rollup/rollup-android-arm64@4.12.0	None	`0`	2.49 MB	lukastaegert
npm/@rollup/rollup-darwin-arm64@4.12.0	None	`0`	2.41 MB	lukastaegert
npm/@rollup/rollup-darwin-x64@4.12.0	None	`0`	2.58 MB	lukastaegert
npm/@rollup/rollup-linux-arm-gnueabihf@4.12.0	None	`0`	2.43 MB	lukastaegert
npm/@rollup/rollup-linux-arm64-gnu@4.12.0	None	`0`	2.47 MB	lukastaegert
npm/@rollup/rollup-linux-arm64-musl@4.12.0	None	`0`	2.37 MB	lukastaegert
npm/@rollup/rollup-linux-riscv64-gnu@4.12.0	None	`0`	2.54 MB	lukastaegert
npm/@rollup/rollup-linux-x64-gnu@4.12.0	None	`0`	2.73 MB	lukastaegert
npm/@rollup/rollup-linux-x64-musl@4.12.0	None	`0`	2.73 MB	lukastaegert
npm/@rollup/rollup-win32-arm64-msvc@4.12.0	None	`0`	2.93 MB	lukastaegert
npm/@rollup/rollup-win32-ia32-msvc@4.12.0	None	`0`	2.64 MB	lukastaegert
npm/@rollup/rollup-win32-x64-msvc@4.12.0	None	`0`	3.44 MB	lukastaegert
npm/@sindresorhus/is@5.3.0	None	`0`	58.9 kB	sindresorhus
npm/any-promise@1.3.0	None	`0`	22.2 kB	kevinbeaty
npm/bundle-require@4.0.2	environment, filesystem, unsafe Transitive: network, shell	`+25`	224 MB	egoist
npm/cac@6.7.14	None	`0`	81.8 kB	egoist
npm/chokidar@3.6.0	environment, filesystem	`+14`	531 kB	paulmillr
npm/esbuild@0.19.12	environment, filesystem, network, shell	`+23`	224 MB	evanw
npm/joycon@3.1.1	environment, filesystem	`0`	14 kB	egoist
npm/load-tsconfig@0.2.5	None	`0`	17.1 kB	egoist
npm/lodash.sortby@4.7.0	None	`0`	75.8 kB	jdalton
npm/mz@2.7.0	filesystem, network, shell	`+4`	52 kB	jongleberry
npm/postcss-load-config@4.0.2	environment, unsafe Transitive: filesystem, shell	`+38`	466 MB	ai
npm/rollup@4.12.0	environment, filesystem	`+15`	35.8 MB	lukastaegert
npm/sucrase@3.35.0	Transitive: environment, filesystem, network, shell, unsafe	`+38`	3.74 MB	alangpierce
npm/thenify-all@1.6.0	None	`+2`	36.7 kB	dead_horse
npm/thenify@3.3.1	None	`+1`	30.1 kB	dead_horse
npm/tr46@1.0.1	None	`+1`	356 kB	sebmaster
npm/tree-kill@1.2.2	shell	`0`	7.82 kB	wmhilton
npm/ts-interface-checker@0.1.13	None	`0`	68 kB	dsagal2
npm/tsup@8.0.2	environment, eval, filesystem Transitive: network, shell, unsafe	`+169`	731 MB	egoist
npm/webidl-conversions@4.0.2	None	`0`	19.3 kB	domenic
npm/whatwg-url@7.1.0	None	`+4`	529 kB	domenic

View full report↗︎

socket-security · 2024-01-22T14:56:45Z

👍 Dependency issues cleared. Learn more about Socket for GitHub ↗︎

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

Ignoring: npm/mz@2.7.0, npm/source-map@0.8.0-beta.0, npm/tree-kill@1.2.2

View full report↗︎

Next steps

Take a deeper look at the dependency

Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support [AT] socket [DOT] dev.

Remove the package

If you happen to install a dependency that Socket reports as Known Malware you should immediately remove it and select a different dependency. For other alert types, you may may wish to investigate alternative packages or consider if there are other ways to mitigate the specific risk posed by the dependency.

Mark a package as acceptable risk

To ignore an alert, reply with a comment starting with @SocketSecurity ignore followed by a space separated list of ecosystem/package-name@version specifiers. e.g. @SocketSecurity ignore npm/foo@1.0.0 or ignore all packages with @SocketSecurity ignore-all

rygine · 2024-01-25T17:56:58Z

just a general comment on tsup. if the treeshake option is ever used, the source maps will be inaccurate. more info here.

it can make debugging a huge pain. we've since moved away from tsup in favor of rollup.

Mrtenz · 2024-01-26T10:06:06Z

just a general comment on tsup. if the treeshake option is ever used, the source maps will be inaccurate. more info here.
it can make debugging a huge pain. we've since moved away from tsup in favor of rollup.

Thanks for the heads up! I'll see how big of a difference treeshake makes in our case. It seems like there's a couple open PRs to fix it as well, so potentially we can patch tsup until one of those is merged.

Mrtenz · 2024-02-01T14:15:38Z

@SocketSecurity ignore npm/tree-kill@1.2.2
@SocketSecurity ignore npm/source-map@0.8.0-beta.0

Mrtenz · 2024-02-01T14:18:26Z

tsup moves everything to separate chunk files (when code splitting is enabled), which breaks code relying on relative imports, in our case, the Webpack loader. As a workaround I added this new loader, which executes a function. This works since the worker path points to __filename, so regardless of where the file is placed, it will load properly.

codecov · 2024-02-01T14:19:16Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (e682522) 96.59% compared to head (1eba380) 96.60%.

Additional details and impacted files

@@ Coverage Diff @@## tsup #2120 +/- ##
=======================================
Coverage 96.59% 96.60% =======================================
Files 332 334 +2 Lines 7582 7595 +13 Branches 1175 1175 =======================================
+ Hits 7324 7337 +13 
Misses 258 258

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Mrtenz · 2024-02-18T22:36:55Z

@SocketSecurity ignore npm/mz@2.7.0

Mrtenz · 2024-02-19T09:42:51Z

 {
 "name": "@metamask/snaps-simulator",
 "version": "2.4.3",
+ "private": true,


Note that I added private: true here. We previously published this to NPM for snaps-jest, but we no longer use it.

Mrtenz · 2024-02-19T09:44:13Z

 exit 1
 fi

- post-build:


I've removed all post-tsc scripts for simplicity.

FrederikBolding · 2024-02-21T09:15:25Z

+ languageName: node
+ linkType: hard
+
 "@esbuild/linux-arm64@npm:0.18.20":


Can we dedupe our esbuild deps or does that require bumping WDIO again?

Looks like it requires bumping WDIO yeah.

This changes all packages to be built with `tsup`, instead of SWC. `tsup` uses `esbuild` under the hood, so performance should be comparable. More context here: MetaMask/utils#144.

This swaps out the build system for `tsup`, and adds a proper ESM build to each package. In addition to that, the following changes have been made: - All packages with Node.js-specific code have been refactored to make them browser compatible. Node.js exports were moved to a separate `/node` export, i.e., `@metamask/snaps-utils/node`. - This also applies to the React Native webview service in `snaps-controllers`, which can now be imported from `@metamask/snaps-controllers/react-native`. - The `snaps-simulator` package is no longer published to NPM. We were previously using it for `snaps-jest`, but it's no longer used now. --- This pull request consists of the following pull requests: - #2120. - #2211.

This changes all packages to be built with `tsup`, instead of SWC. `tsup` uses `esbuild` under the hood, so performance should be comparable. More context here: MetaMask/utils#144.

Mrtenz force-pushed the mrtenz/tsup branch from e5a06d0 to 0c6cf3aCompare January 23, 2024 13:05

Mrtenz force-pushed the mrtenz/tsup branch from d08e2f6 to ec8cc03Compare February 1, 2024 14:13

Mrtenz commented Feb 1, 2024
View reviewed changes

Mrtenz force-pushed the mrtenz/tsup branch from 39e3976 to e683f30Compare February 18, 2024 21:45

Mrtenz commented Feb 19, 2024
View reviewed changes

FrederikBolding reviewed Feb 19, 2024
View reviewed changes

Comment threadpackages/snaps-controllers/package.json Outdated

Mrtenz marked this pull request as ready for review February 19, 2024 13:58

Mrtenz requested a review from a team as a code owner February 19, 2024 13:58

FrederikBolding mentioned this pull request Feb 20, 2024
Bump snaps packages and support dynamic interfaces MetaMask/metamask-extension#22828
Merged

FrederikBolding reviewed Feb 21, 2024
View reviewed changes

Comment threadpackages/snaps-webpack-plugin/package.json

FrederikBolding reviewed Feb 21, 2024
View reviewed changes

Comment threadpackages/snaps-browserify-plugin/package.json Outdated

FrederikBolding reviewed Feb 21, 2024
View reviewed changes

Comment threadpackages/snaps-cli/package.json

Mrtenz changed the base branch from main to tsupFebruary 21, 2024 15:21

Mrtenz added 8 commits February 21, 2024 16:23

Build packages with tsup
1a654cb

Update CI and remove unused dependencies
caf6fc5

Fix CI dependencies
6f68cbf

Fix CLI tests
36dedb3

Fix CLI tests
e81bc55

Exclude Snaps Simulator from build
f0dd6fb

Fix typo
276b6c4

Potentially fix build
f103978

Mrtenz added 10 commits February 21, 2024 16:25

Add missing script
8f18c3d

Fix jest-preset import
4d981a4

Add run support to plugin-workspaces-filter
67902d5

Clean up some of the scripts
d48ba56

Fix LavaMoat policies
9ecd9eb

Fix lint error
9a4753d

More script changes
2e9573c

Update example shasums
d2e809e

Fix package.json and externals
770c137

Fix snaps-browserify-plugin package.json
1eba380

Mrtenz force-pushed the mrtenz/tsup branch from 957c12c to 1eba380Compare February 21, 2024 15:31

FrederikBolding approved these changes Feb 22, 2024
View reviewed changes

Mrtenz merged commit a785f62 into tsupFeb 22, 2024

Mrtenz deleted the mrtenz/tsup branch February 22, 2024 09:48

Mrtenz added a commit that referenced this pull request Feb 22, 2024
Build packages withtsup(#2120)
9617754
This changes all packages to be built with `tsup`, instead of SWC. `tsup` uses `esbuild` under the hood, so performance should be comparable. More context here: MetaMask/utils#144.

Mrtenz added a commit that referenced this pull request Feb 23, 2024
Build packages withtsup(#2120)
b4b8bc0
This changes all packages to be built with `tsup`, instead of SWC. `tsup` uses `esbuild` under the hood, so performance should be comparable. More context here: MetaMask/utils#144.

Mrtenz mentioned this pull request Feb 23, 2024
BREAKING: Use tsup and refactor exports #2210
Merged

Mrtenz added a commit that referenced this pull request Feb 26, 2024
Build packages withtsup(#2120)
2724a2a
This changes all packages to be built with `tsup`, instead of SWC. `tsup` uses `esbuild` under the hood, so performance should be comparable. More context here: MetaMask/utils#144.

naugtur pushed a commit that referenced this pull request Mar 28, 2024
Build packages withtsup(#2120)
ab1e622
This changes all packages to be built with `tsup`, instead of SWC. `tsup` uses `esbuild` under the hood, so performance should be comparable. More context here: MetaMask/utils#144.

Uh oh!

Conversation

Mrtenz commented Jan 22, 2024

Uh oh!

socket-securityBot commented Jan 22, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

socket-securityBot commented Jan 22, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Next steps

Uh oh!

rygine commented Jan 25, 2024

Uh oh!

Mrtenz commented Jan 26, 2024

Uh oh!

Mrtenz commented Feb 1, 2024

Uh oh!

MrtenzFeb 1, 2024

Choose a reason for hiding this comment

Uh oh!

codecovBot commented Feb 1, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Mrtenz commented Feb 18, 2024

Uh oh!

MrtenzFeb 19, 2024

Choose a reason for hiding this comment

Uh oh!

MrtenzFeb 19, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

FrederikBoldingFeb 21, 2024

Choose a reason for hiding this comment

Uh oh!

MrtenzFeb 21, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

socket-securityBot commented Jan 22, 2024•
edited
Loading

socket-securityBot commented Jan 22, 2024•
edited
Loading

codecovBot commented Feb 1, 2024•
edited
Loading