Skip to content

Add openlineage system tests for missing use cases - #68664

Merged
mobuchowski merged 1 commit into
apache:mainfrom
kacpermuda:ol-system-tests-extension
Jun 17, 2026
Merged

Add openlineage system tests for missing use cases#68664
mobuchowski merged 1 commit into
apache:mainfrom
kacpermuda:ol-system-tests-extension

Conversation

@kacpermuda

@kacpermudakacpermuda commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

PR improving only OL system tests and just few more nits around typing: whitelisting "ol" and "OL" in word spelling, and extending types in openlineage/api with Dataset as addition to InputDataset and OutputDataset to prevent mypy errors for users. I need to improve typing in OL client, will followup with PR removing these type: ignore lines.

Adding some missing test cases for OL system tests:

  • custom operator with all custom OL methods
  • custom operator with some OL methods
  • all facets (custom ol operator producing all OL facets so that we can validate if nothing is broken in python client where we import them from)
  • timetables serialization for timetables that are missing now
  • edge labels
  • hitl
  • manual lineage api functions from openlineage/api
  • fail task events
  • fail dag events

Also some json formatting, improved logging in test operator, added possibility for test expected templates to cover all AF versions after some, f.e. 3.3+ etc.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@kacpermuda
kacpermudaforce-pushed the ol-system-tests-extension branch 3 times, most recently from 3509e65 to 6646b0bCompareJune 17, 2026 11:05
@kacpermuda
kacpermudaforce-pushed the ol-system-tests-extension branch from 6646b0b to 2662ac6CompareJune 17, 2026 12:28
@mobuchowski
mobuchowski merged commit 674544c into apache:mainJun 17, 2026
86 checks passed
@kacpermuda
kacpermuda deleted the ol-system-tests-extension branch June 17, 2026 13:40
shahar1 added a commit to shahar1/airflow that referenced this pull request Jun 17, 2026
Several PRs landed on main after the initial 2026-06-16 preparation and belong
in this wave:
* openlineage: type-hint widening on the public emit_dataset_lineage /
emit_query_lineage helpers (apache#68664) was not in the original wave; include it
as a new 2.18.1 release.
* amazon, cncf.kubernetes, google: the task_instance mypy fix in provider
triggers (apache#68685) is recorded as a Misc entry in each provider's pending
section (no version escalation - all three were already bumped higher).
shahar1 added a commit to shahar1/airflow that referenced this pull request Jun 18, 2026
Several PRs landed on main after the initial 2026-06-16 preparation and belong
in this wave:
* openlineage: type-hint widening on the public emit_dataset_lineage /
emit_query_lineage helpers (apache#68664) was not in the original wave; include it
as a new 2.18.1 release.
* amazon, cncf.kubernetes, google: the task_instance mypy fix in provider
triggers (apache#68685) is recorded as a Misc entry in each provider's pending
section (no version escalation - all three were already bumped higher).
* common-ci
shahar1 added a commit to shahar1/airflow that referenced this pull request Jun 18, 2026
Several PRs landed on main after the initial 2026-06-16 preparation and belong
in this wave:
* openlineage: type-hint widening on the public emit_dataset_lineage /
emit_query_lineage helpers (apache#68664) was not in the original wave; include it
as a new 2.18.1 release.
* amazon, cncf.kubernetes, google: the task_instance mypy fix in provider
triggers (apache#68685) is recorded as a Misc entry in each provider's pending
section (no version escalation - all three were already bumped higher).
* common-ci
shahar1 added a commit to shahar1/airflow that referenced this pull request Jun 18, 2026
Several PRs landed on main after the initial 2026-06-16 preparation and belong
in this wave:
* openlineage: type-hint widening on the public emit_dataset_lineage /
emit_query_lineage helpers (apache#68664) was not in the original wave; include it
as a new 2.18.1 release.
* amazon, cncf.kubernetes, google: the task_instance mypy fix in provider
triggers (apache#68685) is recorded as a Misc entry in each provider's pending
section (no version escalation - all three were already bumped higher).
* common-ci
shahar1 added a commit to shahar1/airflow that referenced this pull request Jun 19, 2026
Several PRs landed on main after the initial 2026-06-16 preparation and belong
in this wave:
* openlineage: type-hint widening on the public emit_dataset_lineage /
emit_query_lineage helpers (apache#68664) was not in the original wave; include it
as a new 2.18.1 release.
* amazon, cncf.kubernetes, google: the task_instance mypy fix in provider
triggers (apache#68685) is recorded as a Misc entry in each provider's pending
section (no version escalation - all three were already bumped higher).
* common-ci
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@kacpermuda@mobuchowski