Skip to content

feat(providers/openai): support batch api in hook/operator/trigger - #41554

Merged
Lee-W merged 14 commits into
apache:mainfrom
josix:feat/openai/support-batch-api
Aug 22, 2024
Merged

feat(providers/openai): support batch api in hook/operator/trigger#41554
Lee-W merged 14 commits into
apache:mainfrom
josix:feat/openai/support-batch-api

Conversation

@josix

@josixjosix commented Aug 17, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

closes: #41336

  • Hook that supports batch api related operation
  • Operator/Trigger for Batch API
  • Unit tests for new added behaviors to hook
  • Unit tests for new added behaviors to operator
  • Unit tests for new added behaviors to trigger
  • Update how-to guide
  • Update provider.yaml

image

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@josix
josixforce-pushed the feat/openai/support-batch-api branch from bfd7cfc to 2641b4aCompareAugust 17, 2024 16:39
ginger0207
ginger0207 reviewed Aug 19, 2024
View reviewed changes
Comment threadairflow/providers/openai/hooks/openai.py Outdated
@josix
josixforce-pushed the feat/openai/support-batch-api branch 2 times, most recently from a0b3069 to 95a8b0bCompareAugust 19, 2024 19:06
@josix
josix marked this pull request as ready for review August 19, 2024 19:07
@josix
josixforce-pushed the feat/openai/support-batch-api branch from 95a8b0b to 6410863CompareAugust 19, 2024 19:39
@josix
josixforce-pushed the feat/openai/support-batch-api branch from 6410863 to 2bba21aCompareAugust 19, 2024 19:56
vincbeck
vincbeck reviewed Aug 19, 2024
View reviewed changes
Comment threadairflow/providers/openai/operators/openai.py Outdated
Lee-W
Lee-W reviewed Aug 20, 2024
View reviewed changes
Comment threadairflow/providers/openai/hooks/openai.py
Comment threadairflow/providers/openai/hooks/openai.py
Comment threadairflow/providers/openai/hooks/openai.py Outdated
Comment threadairflow/providers/openai/hooks/openai.py Outdated
Comment threadairflow/providers/openai/hooks/openai.py Outdated
Comment threadairflow/providers/openai/operators/openai.py Outdated
Comment threadairflow/providers/openai/operators/openai.py Outdated
Comment threadairflow/providers/openai/triggers/openai.py Outdated
Comment threadairflow/providers/openai/triggers/openai.py Outdated
Comment threadairflow/providers/openai/triggers/openai.py Outdated
josixand others added 7 commits August 20, 2024 18:33
@josix@Lee-W
Update airflow/providers/openai/hooks/openai.py
2a73cd3
Co-authored-by: Wei Lee <weilee.rx@gmail.com>
@josix@Lee-W
Update airflow/providers/openai/hooks/openai.py
52f1d26
Co-authored-by: Wei Lee <weilee.rx@gmail.com>
@josix@Lee-W
Update airflow/providers/openai/hooks/openai.py
59cdd5f
Co-authored-by: Wei Lee <weilee.rx@gmail.com>
@josix@Lee-W
Update airflow/providers/openai/triggers/openai.py
e18bbd3
Co-authored-by: Wei Lee <weilee.rx@gmail.com>
@josix@Lee-W
Update airflow/providers/openai/operators/openai.py
4db4c01
Co-authored-by: Wei Lee <weilee.rx@gmail.com>
@josix@Lee-W
Update airflow/providers/openai/triggers/openai.py
71bc168
Co-authored-by: Wei Lee <weilee.rx@gmail.com>
@josix@Lee-W
Update airflow/providers/openai/hooks/openai.py
9ce0172
Co-authored-by: Wei Lee <weilee.rx@gmail.com>
@josix
josixforce-pushed the feat/openai/support-batch-api branch from 92718bd to d2a5415CompareAugust 20, 2024 12:43
vincbeck
vincbeck reviewed Aug 20, 2024
View reviewed changes
Comment threadtests/providers/openai/operators/test_openai.py Outdated
@josix
josixforce-pushed the feat/openai/support-batch-api branch from d2a5415 to 2bc3c4bCompareAugust 20, 2024 16:25
@josix
josix requested review from Lee-W and vincbeckAugust 20, 2024 16:47
josix
josix commented Aug 20, 2024
View reviewed changes
Comment threadtests/providers/openai/operators/test_openai.py
josix
josix commented Aug 20, 2024
View reviewed changes
Comment threadtests/providers/openai/operators/test_openai.py
Lee-W
Lee-W approved these changes Aug 21, 2024
View reviewed changes

@Lee-WLee-W left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left one nitpick. we should be able to merge this one once addressed

Comment threadairflow/providers/openai/exceptions.py Outdated
Comment threadairflow/providers/openai/hooks/openai.py Outdated
Comment threadtests/system/providers/openai/example_trigger_batch_operator.py
@josix
josix requested a review from ginger0207August 21, 2024 09:22
@Lee-W
Lee-W merged commit 00e73e6 into apache:mainAug 22, 2024
potiuk added a commit to potiuk/airflow that referenced this pull request Aug 22, 2024
@potiuk
Fix failing no-pydantic tests for OpenAI.
af8b82d
The apache#41554 broke exclusion of openai tests when no pydantic is
installed.
@potiukpotiuk mentioned this pull request Aug 22, 2024
Merged
potiuk added a commit that referenced this pull request Aug 22, 2024
@potiuk
Fix failing no-pydantic tests for OpenAI. (#41677)
c19986c
The #41554 broke exclusion of openai tests when no pydantic is
installed.
@eladkaleladkal mentioned this pull request Sep 21, 2024
Closed
@takayoshi-makabetakayoshi-makabe mentioned this pull request Jul 7, 2026
Merged
1 task
This was referenced Aug 3, 2026
Closed
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Lee-WLee-WLee-W approved these changes
@vincbeckvincbeckvincbeck approved these changes
+1 more reviewer
@ginger0207ginger0207ginger0207 approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

area:providersprovider:openai

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[OpenAI Provider] Support Batch API in OpenAI Hook

4 participants

@josix@Lee-W@ginger0207@vincbeck