Skip to content

Add support for CI testing - #124

Merged
leofang merged 58 commits into
NVIDIA:mainfrom
sandeepd-nv:add_ci_test
Dec 8, 2024
Merged

Add support for CI testing#124
leofang merged 58 commits into
NVIDIA:mainfrom
sandeepd-nv:add_ci_test

Conversation

@sandeepd-nv

@sandeepd-nvsandeepd-nv commented Sep 23, 2024

Copy link
Copy Markdown
Contributor

No description provided.

@sandeepd-nvsandeepd-nv added the CI/CD CI/CD infrastructure label Sep 23, 2024
@sandeepd-nvsandeepd-nv self-assigned this Sep 23, 2024
@sandeepd-nv

sandeepd-nv commented Sep 26, 2024

Copy link
Copy Markdown
ContributorAuthor

Blocked on #128.

@leofang

leofang commented Oct 9, 2024

Copy link
Copy Markdown
Member

Blocked on #128.

See #128 (comment), thx!

@leofangleofang mentioned this pull request Dec 3, 2024
Merged
leofang
leofang reviewed Dec 4, 2024
View reviewed changes
Comment thread.github/workflows/gh-build-and-test.yml Outdated
@leofangleofang added the P0 High priority - Must do! label Dec 4, 2024
@copy-pr-bot

copy-pr-botBot commented Dec 7, 2024

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@leofang

leofang commented Dec 7, 2024

Copy link
Copy Markdown
Member

/ok to test

@leofang

leofang commented Dec 8, 2024

Copy link
Copy Markdown
Member

/ok to test

@leofang

leofang commented Dec 8, 2024

Copy link
Copy Markdown
Member

/ok to test

@leofang

leofang commented Dec 8, 2024

Copy link
Copy Markdown
Member

/ok to test

@leofang

leofang commented Dec 8, 2024

Copy link
Copy Markdown
Member

/ok to test

1 similar comment
@leofang

leofang commented Dec 8, 2024

Copy link
Copy Markdown
Member

/ok to test

@leofangleofang closed this Dec 8, 2024
@leofangleofang reopened this Dec 8, 2024
@leofang

leofang commented Dec 8, 2024

Copy link
Copy Markdown
Member

/ok to test

@leofang
leofang marked this pull request as ready for review December 8, 2024 23:09
@leofangleofang added this to the cuda.core beta 2 milestone Dec 8, 2024
@leofangleofang added test Improvements or additions to tests cuda.bindings Everything related to the cuda.bindings module cuda.core Everything related to the cuda.core module labels Dec 8, 2024
@leofang

leofang commented Dec 8, 2024•
edited
Loading

Copy link
Copy Markdown
Member

Here are some updates since Friday:

  1. it makes the CI logs hard to browse (everything from the same workflow is collapsed inside the same title)

It turns out that this is a misunderstanding (of mine), sorry! It's the other way around: It's the composite actions that do this, not reusable workflows, see, e.g. https://docs.github.com/en/actions/sharing-automations/avoiding-duplication#comparison-of-reusable-workflows-and-composite-actions. So I refactored in the opposite (and wrong) direction.

Let us do this (change all composite actions to reusable workflows) in a follow-up PR since the CI is now working and there's no reason to delay.

  1. just discovered: environment variables cannot be passed across workflows (https://github.com/orgs/community/discussions/26671), which therefore requires both build and test workflows to do the same setup, and it requires complex logic to manage

This is another misunderstanding (of mine), sorry (again)! For both cases (distinct jobs vs distinct workflows), it require some handling of input/output. There's no way for sharing the env vars in either case.

We are overcomplicating the CI for a simple project and bending backward

As part of this I removed all CI scripts in commit 7b074f0. It is the best that we focus on testing pip-based workflows for now, and add conda next (which would be treated differently). Mixing-and-matching is not ideal.

leofang
leofang reviewed Dec 8, 2024
View reviewed changes
Comment threadcuda_core/tests/conftest.py
Comment on lines +24 to +26
@pytest.fixture(scope="session", autouse=True)
def always_init_cuda():
handle_return(driver.cuInit(0))

@leofangleofangDec 8, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, @ksimpson-work the CI was able to catch this issue: Depending on how the tests are run, it could be possible that a test ends without CUDA even initialized. So we must ensure it ourselves by the test start time.

leofang
leofang reviewed Dec 8, 2024
View reviewed changes
Comment threadcuda_core/tests/conftest.py
Comment on lines +39 to +42
ctx = handle_return(driver.cuCtxGetCurrent())
if int(ctx) == 0:
# no active context, do nothing
return

@leofangleofangDec 8, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, @ksimpson-work another issue caught by the CI (and also back in #261): A test could end early without a CUDA context set current, so we need to detect this at the test teardown time.

leofang
leofang reviewed Dec 8, 2024
View reviewed changes
Comment threadcuda_core/tests/example_tests/utils.py
import os
import sys

import cupy as cp

@leofangleofangDec 8, 2024•
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now I treat CuPy as an optional test dependency, so any reference to CuPy in this file should be removed. (We're not using too much memory during tests anyway.)

leofang
leofang reviewed Dec 8, 2024
View reviewed changes
Comment threadcuda_core/tests/test_program.py
Comment on lines +16 to +21
def can_load_generated_ptx():
_, driver_ver = cuda.cuDriverGetVersion()
_, nvrtc_major, nvrtc_minor = nvrtc.nvrtcVersion()
if nvrtc_major * 1000 + nvrtc_minor * 10 > driver_ver:
return False
return True

@leofangleofangDec 8, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @ksimpson-work this is akin to this snippet that I added to CuPy in the past (PTX might not be loadable/JIT'able if it's newer than the driver):
https://github.com/cupy/cupy/blob/8eb16ac910e85c119a20f68a69de9a2e6034069c/tests/cupy_tests/core_tests/test_raw.py#L557-L568

@leofang
leofang merged commit 0723d62 into NVIDIA:mainDec 8, 2024
@leofang

leofang commented Dec 8, 2024

Copy link
Copy Markdown
Member

Thanks for help, @sandeepd-nv!

This was referenced Dec 8, 2024
Closed
Merged
Closed
@leofangleofang linked an issue Dec 9, 2024 that may be closed by this pull request
Add tests to github actions #253
Closed
@leofangleofang mentioned this pull request Dec 9, 2024
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@leofangleofangleofang approved these changes

Assignees

@sandeepd-nvsandeepd-nv

Labels

CI/CDCI/CD infrastructurecuda.bindingsEverything related to the cuda.bindings modulecuda.coreEverything related to the cuda.core moduleP0High priority - Must do!testImprovements or additions to tests

Projects

None yet

Milestone

cuda.core beta 2

Development

Successfully merging this pull request may close these issues.

Add tests to github actions

2 participants

@sandeepd-nv@leofang