Skip to content

Add Apache Kyuubi Provider - #61707

Closed
MonsterChenzhuo wants to merge 5 commits into
apache:mainfrom
MonsterChenzhuo:kyuubi
Closed

Add Apache Kyuubi Provider#61707
MonsterChenzhuo wants to merge 5 commits into
apache:mainfrom
MonsterChenzhuo:kyuubi

Conversation

@MonsterChenzhuo

@MonsterChenzhuoMonsterChenzhuo commented Feb 10, 2026

Copy link
Copy Markdown
Contributor

Description

This PR introduces a new provider for Apache Kyuubi.

Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on Lakehouses. It is compatible with the HiveServer2 protocol.

While the Hive provider can be used to connect to Kyuubi, Kyuubi often uses its own beeline executable or requires specific configurations. This provider adds a dedicated KyuubiHook and KyuubiOperator to streamline the integration and allow users to configure the specific path to the Kyuubi Beeline executable via the UI.

Changes

Added apache-airflow-providers-apache-kyuubi provider structure.
Implemented KyuubiHook (inherits from HiveCliHook):

  • Adds a custom connection form widget beeline_path.
  • Overrides command preparation to use the custom Beeline path if configured.
  • Implemented KyuubiOperator (inherits from HiveOperator).
  • Added Unit Tests for Hook and Operator.
  • Added Documentation (Connection, Operator, Index).

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@MonsterChenzhuo
MonsterChenzhuoforce-pushed the kyuubi branch 2 times, most recently from ff4b741 to 1889d83CompareFebruary 10, 2026 10:43
@tirkarthi

Copy link
Copy Markdown
Contributor

Please go through the guidelines for a new provider which requires a discussion and vote in the mailing list. Please link here if there are any existing discussions.

https://github.com/apache/airflow/blob/main/PROVIDERS.rst#accepting-new-community-providers

@jscheffl

Copy link
Copy Markdown
Contributor

Additionally besides topping a discussion to devlist also global CI parts are missing, e.g. Codeowners, labels and pipeline additions. See #57610 as an example

@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actionsgithub-actionsBot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Mar 28, 2026
@kaxilkaxil reopened this Apr 2, 2026
@kaxil

kaxil commented Apr 2, 2026

Copy link
Copy Markdown
Member

@MonsterChenzhuo Would love to help see this over the line if you are still interested

@github-actionsgithub-actionsBot removed the stale Stale PRs per the .github/workflows/stale.yml policy file label Apr 3, 2026
@potiuk

Copy link
Copy Markdown
Member

@MonsterChenzhuo A few things need addressing before review — see our Pull Request quality criteria.

  • Image build. See docs.

No rush.


Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actionsgithub-actionsBot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Jul 21, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providersstaleStale PRs per the .github/workflows/stale.yml policy file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@MonsterChenzhuo@tirkarthi@jscheffl@kaxil@potiuk