We have a good deal of code which concerns the creation of build artifacts, but none of it is tested in an end-to-end way.
(Right now I'm thinking about all of our rollup config code and the withSentryConfig code in the nextjs SDK, which modifies the user's webpack config in order to change what's in their bundles, but there may be other places as well. The latter does have unit tests, but no integration tests, and the former has no tests at all.)
We should consider having integration tests which run the code and check the emitted artifacts for certain characteristics. For example, do the debug bundles actually contain logging statements? Do the non-debug bundles correctly omit them? Do artifacts end up where we think they should? etc etc.
While we'd have to restrict it to fairly broad characterizations (both to prevent brittleness and to make sure that we're testing things which it's easy to check programmatically), it could at least protect against obvious mistakes.
We have a good deal of code which concerns the creation of build artifacts, but none of it is tested in an end-to-end way.
(Right now I'm thinking about all of our rollup config code and the
withSentryConfigcode in the nextjs SDK, which modifies the user's webpack config in order to change what's in their bundles, but there may be other places as well. The latter does have unit tests, but no integration tests, and the former has no tests at all.)We should consider having integration tests which run the code and check the emitted artifacts for certain characteristics. For example, do the debug bundles actually contain logging statements? Do the non-debug bundles correctly omit them? Do artifacts end up where we think they should? etc etc.
While we'd have to restrict it to fairly broad characterizations (both to prevent brittleness and to make sure that we're testing things which it's easy to check programmatically), it could at least protect against obvious mistakes.