acc: Use BundleConfig to define bundle.name once

[denik]

Changes

Using BundleConfig option (#2809), specify boilerplate bundle.name once and clean up individual test cases.

The tests with diagnostics (lineno, column) are left alone.

Test acceptance/bundle/volume_path is split into invidual tests, one databricks.yml per test (since BundleConfig only updates databricks.yml[.tmpl] in the root of the test.

Why

Simpler to read and write, make each test focussed on what's important.

Tests

Existing tests.

Benchmarked acceptance test suite (local) to ensure this does not add a lot of overhead (due to parsing/marshalling every config):

This branch: Time (mean ± σ): 18.364 s ± 0.857 s [User: 32.707 s, System: 27.281 s]
Range (min … max): 17.002 s … 19.922 s 10 runs
Main:
Time (mean ± σ): 18.337 s ± 0.789 s [User: 32.613 s, System: 27.235 s]
Range (min … max): 17.078 s … 19.342 s 10 runs

[denik]

The configuration for the bind tests includes a UUID for the bundle name. This is not done for other tests. If they run with Cloud = true and run in parallel, they can interfere with each other. We can ensure they never interfere by including a UUID for all of them.

Have you observed this failure mode?

I haven't seen any failures, but I also don't see why not default to test-bundle-$UNIQUE_NAME everywhere, even if tests are local-only. That way we won't miss it if tests are updated to run from local to local+cloud.

See 0ccf171

[pietern]

@denik I believe this only works when used in combination with .tmpl and envsubst, not out of the box.

You can see the output for some of these include $UNIQUE_ID instead of the replacement value [UNIQUE_ID].

[denik]

@denik I believe this only works when used in combination with .tmpl and envsubst, not out of the box.

You can see the output for some of these include $UNIQUE_ID instead of the replacement value [UNIQUE_ID].

Good catch, I dropped that commit. I got confused with Env / EnvMatrix which do process environment variables in the test runner.

We can implement something similar here (and thus drop the need for envsubst completely) but that would have to be a follow up.