Skip to content

Fix a bug in artifacts where glob was applied on non-abs path - #2540

Merged
denik merged 9 commits into
mainfrom
denik/artifacts-double-test
Mar 21, 2025
Merged

Fix a bug in artifacts where glob was applied on non-abs path#2540
denik merged 9 commits into
mainfrom
denik/artifacts-double-test

Conversation

@denik

@denikdenik commented Mar 20, 2025

Copy link
Copy Markdown
Contributor

Changes

Bug introduced in #2526.

When calling expandGlobs as a function without mutator entry/exit copy, we did not copy typed structure into dynamic structure. Thus expandGlobs saw an old version of config, without absolute paths introduced by code before it. Since it calls filepath.Glob which depends on CWD, it stopped doing the matching.

Tests

New regression test to catch this issues.

@denik
deniktemporarily deployed to test-trigger-is March 20, 2025 15:58 — with GitHub Actions Inactive
@denik
denikforce-pushed the denik/artifacts-double-test branch from f5d4b5f to bcc702bCompareMarch 20, 2025 15:59
@denik
deniktemporarily deployed to test-trigger-is March 20, 2025 15:59 — with GitHub Actions Inactive
@denik
denikforce-pushed the denik/artifacts-double-test branch from bcc702b to 1a8dc5dCompareMarch 20, 2025 16:01
@denik
deniktemporarily deployed to test-trigger-is March 20, 2025 16:01 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is March 20, 2025 16:30 — with GitHub Actions Inactive
@denikdenik changed the title Add a test that show the wheel being uploaded twiceFix a bug where wheel is uploaded twiceMar 20, 2025
@denik
deniktemporarily deployed to test-trigger-is March 20, 2025 16:33 — with GitHub Actions Inactive
@denik
denik enabled auto-merge March 20, 2025 16:34
@denik
deniktemporarily deployed to test-trigger-is March 20, 2025 16:35 — with GitHub Actions Inactive
cmdio.LogString(ctx, fmt.Sprintf("Uploading %s...", relPath))
cmdio.LogString(ctx, fmt.Sprintf("Uploading %s...", source))
errs.Go(func() error {
return UploadFile(errCtx, source, u.client)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before UploadFile was taking absolute path if I'm not mistaken and now it takes normalised one (relative + forward slashes). How does it make UploadFile open the file later for upload?

Comment threadbundle/bundle.go Outdated

func (b *Bundle) NormalizePath(path string) (string, error) {
var err error
if filepath.IsAbs(path) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not entirely sure about making absolute paths relative ones. Some of the code (like UploadFile for artifacts or some dashboard code later) relies on such paths to be absolute

@denik
denik marked this pull request as draft March 20, 2025 21:01
auto-merge was automatically disabled March 20, 2025 21:01

Pull request was converted to draft

@denik
denikforce-pushed the denik/artifacts-double-test branch from 585171b to 2e1ba04CompareMarch 20, 2025 21:42
@denik
deniktemporarily deployed to test-trigger-is March 20, 2025 21:42 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is March 20, 2025 21:59 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is March 21, 2025 08:12 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is March 21, 2025 08:24 — with GitHub Actions Inactive
@denik
deniktemporarily deployed to test-trigger-is March 21, 2025 09:29 — with GitHub Actions Inactive
@denikdenik changed the title Fix a bug where wheel is uploaded twiceFix a bug in artifacts where glob was applied on non-abs pathMar 21, 2025
@denik
denik marked this pull request as ready for review March 21, 2025 09:32
@denik
denik requested a review from andrewnesterMarch 21, 2025 09:32
@denik
deniktemporarily deployed to test-trigger-is March 21, 2025 09:32 — with GitHub Actions Inactive
Comment threadbundle/artifacts/expand_globs.go Outdated
}

func expandGlobs(ctx context.Context, b *bundle.Bundle, name string) diag.Diagnostics {
err := b.Config.MarkMutatorEntry(ctx)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just make it a mutator again and rely on bundle.Apply? What's the benefit of having function like this now?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, 240d442

@denik
deniktemporarily deployed to test-trigger-is March 21, 2025 09:43 — with GitHub Actions Inactive
@denik
denik requested a review from andrewnesterMarch 21, 2025 09:44
@denik
denik enabled auto-merge March 21, 2025 09:44
@denik
denik added this pull request to the merge queueMar 21, 2025
Merged via the queue into main with commit 4b685beMar 21, 2025
@denik
denik deleted the denik/artifacts-double-test branch March 21, 2025 10:14
my_prebuilt_whl:
type: whl
files:
- source: prebuilt/other_test_code-0.0.1-py3-none-any.whl

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: 4-space indent

@@ -0,0 +1,6 @@
Badness = 'Uploading the same wheel twice'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see this in the output.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, seems like I copy-pasted test.toml. #2559

denik added a commit that referenced this pull request May 20, 2026
## Changes
Bug introduced in #2526.
When calling expandGlobs as a function without mutator entry/exit copy,
we did not copy typed structure into dynamic structure. Thus expandGlobs
saw an old version of config, without absolute paths introduced by code
before it. Since it calls filepath.Glob which depends on CWD, it stopped
doing the matching.
## Tests
New regression test to catch this issues.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@denik@pietern@andrewnester