Skip to content

Simplify TaskSDK's CommsDecoder interface - #45927

Closed
ashb wants to merge 1 commit into
mainfrom
simplify-tasksdk-comms-code
Closed

Simplify TaskSDK's CommsDecoder interface#45927
ashb wants to merge 1 commit into
mainfrom
simplify-tasksdk-comms-code

Conversation

@ashb

@ashbashb commented Jan 22, 2025

Copy link
Copy Markdown
Member

Most of the uses cases of send_request are also followed by get_message --
so lets do that for users -- this makes it closer to an HTTP request too,
where you send a request and get a request.

Some message types don't have a response, so in order to know if we should
call get_message or not we define a "marker" class of NoResponseMessage,
and if we subclass that then we don't expect, and don't read anything back.

The other option I considered was to have the supervisor always send a message
back (i.e. send an empty line when there is otherwise no response to send) and
have get_message() typed to be ToTask | None. It felt marginally better to
have get_message() typed and behave to always expect a messsage.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

Most of the uses cases of `send_request` are also followed by `get_message` --
so lets do that for users -- this makes it closer to an HTTP request too,
where you send a request and get a request.
Some message types don't have a response, so in order to know if we should
call `get_message` or not we define a "marker" class of `NoResponseMessage`,
and if we subclass that then we don't expect, and don't read anything back.
The other option I considered was to have the supervisor always send a message
back (i.e. send an empty line when there is otherwise no response to send) and
have `get_message()` typed to be `ToTask | None`. It felt marginally better to
have `get_message()` typed and behave to always expect a messsage.
@amoghrajesh
amoghrajesh requested review from amoghrajesh and kaxil and removed request for amoghrajeshJanuary 22, 2025 12:17

@amoghrajeshamoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this approach better than having to change the get_message's typing. One comment, otherwise looks good

@@ -265,7 +269,7 @@ class PutVariable(BaseModel):
type: Literal["PutVariable"] = "PutVariable"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add for this one too?

@ashb

ashb commented Jan 22, 2025

Copy link
Copy Markdown
MemberAuthor

I've changed my mind on this -- we should change everything to always have a resposne, even if that is {"ok": true} so that we can catch and propagate exceptions to the Task code.

@ashbashb closed this Jan 22, 2025
@ashb

ashb commented Jan 22, 2025

Copy link
Copy Markdown
MemberAuthor

I'm parking this change for now and will revisit it for fuller exception handling/propagation.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@ashb@amoghrajesh