Skip to content

Add configurable GPU launch wrappers #2

Description

@akrivx

Summary

Introduce free-function GPU launch wrappers to centralise direct kernel launches and indirect GPU operations (library calls / composite ops), enabling optional instrumentation (timing, NVTX, error checking) configured statically at compile time.

Design doc: docs/design/gpu-launch-wrappers.md

Motivation

We want a single convention for submitting GPU work that:

  • keeps call sites simple,
  • avoids runtime overhead when instrumentation is disabled,
  • enables optional profiling/logging/validation,
  • supports both direct kernel launches and indirect "GPU ops".

Scope

In scope

  • Free functions: launch_kernel, launch_op
  • Compile-time policy selection
  • Optional instrumentation compiled out when disabled
  • Nested launch wrapper calls are not allowed:
    • Calling launch_kernel or launch_op from within an active wrapper region is invalid.
    • Must be detected (at least in debug builds).

Out of scope

  • Mandatory enforcement (raw <<<...>>> still allowed)
  • Intercepting internal library kernel launches
  • Runtime configuration
  • CUDA Graph capture integration (for now)

Acceptance Criteria

  • launch_kernel() wraps kernel launches
  • launch_op() wraps indirect GPU work (e.g. cuBLAS/CUB calls or composite ops)
  • Non-nested invariant enforced (debug assert / optional hard error)
  • Behaviour configurable at compile time via policy type
  • Zero runtime overhead when features disabled
  • No launcher objects required
  • Semantics match design doc

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions