Skip to content

[JAX] Fixing CI failure due to incorrect use of static_argnums in jax.jit - #785

Merged
denera merged 4 commits into
NVIDIA:mainfrom
denera:te-jax-single-gpu-encoder-test-fix
Apr 18, 2024
Merged

[JAX] Fixing CI failure due to incorrect use of static_argnums in jax.jit#785
denera merged 4 commits into
NVIDIA:mainfrom
denera:te-jax-single-gpu-encoder-test-fix

Conversation

@denera

Copy link
Copy Markdown
Collaborator

This PR fixes the incorrect syntax for static_argnums in the single-GPU encoder example.

JAX has apparently been raising a warning about this for close to 2 years that went unnoticed on our end because we run TE/JAX CI with a global warning ignore: pytest -Wignore

This PR replaces -Wignore with a limited set of filtered warnings defined under tests/jax/pytest.ini. The list of warnings here are based on JAX's own pytest ignores. Any warnings in the CI not covered by this list and need to be filtered should be added to this file moving forward.

@deneradenera added bug Something isn't working jax labels Apr 16, 2024
@deneradenera self-assigned this Apr 16, 2024
…warning filtering for pytest
Signed-off-by: Alp Dener <adener@nvidia.com>
@denera
deneraforce-pushed the te-jax-single-gpu-encoder-test-fix branch from 960a1cc to 5a78a4dCompareApril 16, 2024 16:48
@denera

Copy link
Copy Markdown
CollaboratorAuthor

/te-ci jax

@nouiznouiz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if 2 PRs would have been better. No strong opinion here.

@mingxu1067

mingxu1067 commented Apr 16, 2024

Copy link
Copy Markdown
Collaborator

Should we fix test_single_gpu_mnist.py#L77 as well? It is the same error I found in other CI pipelines.

Signed-off-by: Alp Dener <adener@nvidia.com>
Signed-off-by: Alp Dener <adener@nvidia.com>
@denera

Copy link
Copy Markdown
CollaboratorAuthor

/te-ci jax

@mingxu1067mingxu1067 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: Alp Dener <adener@nvidia.com>
@denera

Copy link
Copy Markdown
CollaboratorAuthor

/te-ci jax

@denera
denera merged commit c63766d into NVIDIA:mainApr 18, 2024
timmoon10 pushed a commit to timmoon10/TransformerEngine that referenced this pull request Apr 22, 2024
…ax.jit (NVIDIA#785)
* fixed static argnums for jax.jit in single gpu encoder test, changed warning filtering for pytest
Signed-off-by: Alp Dener <adener@nvidia.com>
* propagating the fix to the JAX mnist example
Signed-off-by: Alp Dener <adener@nvidia.com>
* fixed missing space ibetween flags i QAA scripts
Signed-off-by: Alp Dener <adener@nvidia.com>
* added TE warnings into the ignore list
Signed-off-by: Alp Dener <adener@nvidia.com>
---------
Signed-off-by: Alp Dener <adener@nvidia.com>
Signed-off-by: Tim Moon <tmoon@nvidia.com>
ksivaman pushed a commit that referenced this pull request Apr 24, 2024
* [JAX] Fixing CI failure due to incorrect use of `static_argnums` in jax.jit (#785)
* fixed static argnums for jax.jit in single gpu encoder test, changed warning filtering for pytest
Signed-off-by: Alp Dener <adener@nvidia.com>
* propagating the fix to the JAX mnist example
Signed-off-by: Alp Dener <adener@nvidia.com>
* fixed missing space ibetween flags i QAA scripts
Signed-off-by: Alp Dener <adener@nvidia.com>
* added TE warnings into the ignore list
Signed-off-by: Alp Dener <adener@nvidia.com>
---------
Signed-off-by: Alp Dener <adener@nvidia.com>
Signed-off-by: Tim Moon <tmoon@nvidia.com>
* [JAX] Allow multi-dims for dgamma and dbeta in LN descriptor. (#780)
* Allow multi-dims for dgamma and dbeta in LN descriptor.
Signed-off-by: Ming Huang <mingh@nvidia.com>
* Fix the jit error in examples/jax
Signed-off-by: Ming Huang <mingh@nvidia.com>
---------
Signed-off-by: Ming Huang <mingh@nvidia.com>
Signed-off-by: Tim Moon <tmoon@nvidia.com>
---------
Signed-off-by: Alp Dener <adener@nvidia.com>
Signed-off-by: Tim Moon <tmoon@nvidia.com>
Signed-off-by: Ming Huang <mingh@nvidia.com>
Co-authored-by: Alp Dener <adener@nvidia.com>
Co-authored-by: Ming-Xu Huang <mingh@nvidia.com>
pggPL pushed a commit to pggPL/TransformerEngine that referenced this pull request May 23, 2024
…ax.jit (NVIDIA#785)
* fixed static argnums for jax.jit in single gpu encoder test, changed warning filtering for pytest
Signed-off-by: Alp Dener <adener@nvidia.com>
* propagating the fix to the JAX mnist example
Signed-off-by: Alp Dener <adener@nvidia.com>
* fixed missing space ibetween flags i QAA scripts
Signed-off-by: Alp Dener <adener@nvidia.com>
* added TE warnings into the ignore list
Signed-off-by: Alp Dener <adener@nvidia.com>
---------
Signed-off-by: Alp Dener <adener@nvidia.com>
Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugSomething isn't workingjax

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@denera@mingxu1067@nouiz@zlsh80826