Skip to content

Add support union type - #754

Closed
pingsutw wants to merge 6 commits into
masterfrom
union_type
Closed

pingsutw wants to merge 6 commits into
masterfrom
union_type

Conversation

@pingsutw

Copy link
Copy Markdown
Member

Signed-off-by: Kevin Su pingsutw@apache.org

TL;DR

Add union type transformer, and it can handle both union and optional type

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

How did you fix the bug, make the feature etc. Link to any design docs etc

Tracking Issue

flyteorg/flyte#1349

Follow-up issue

No

Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Kevin Su <pingsutw@apache.org>
@codecov

codecov Bot commented Nov 17, 2021

Copy link
Copy Markdown

Codecov Report

Merging #754 (4a1429c) into master (7890201) will increase coverage by 0.03%.
The diff coverage is 89.74%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #754      +/-   ##
==========================================
+ Coverage   85.61%   85.65%   +0.03%     
==========================================
  Files         342      342              
  Lines       28914    29033     +119     
  Branches     2377     2388      +11     
==========================================
+ Hits        24755    24868     +113     
- Misses       3527     3533       +6     
  Partials      632      632              
Impacted Files Coverage Δ
flytekit/core/type_engine.py 88.16% <78.04%> (-0.32%) ⬇️
flytekit/models/types.py 97.16% <87.50%> (-1.24%) ⬇️
tests/flytekit/unit/core/test_type_hints.py 96.08% <97.14%> (+0.04%) ⬆️
tests/flytekit/unit/core/test_type_engine.py 99.73% <100.00%> (+0.01%) ⬆️
flytekit/remote/remote.py 70.40% <0.00%> (+0.26%) ⬆️
flytekit/core/interface.py 82.11% <0.00%> (+0.45%) ⬆️
flytekit/types/schema/types.py 78.20% <0.00%> (+0.85%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7890201...4a1429c. Read the comment docs.

Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Kevin Su <pingsutw@apache.org>
@pingsutw pingsutw changed the title [WIP] Add support union type Add support union type Nov 18, 2021
Signed-off-by: Kevin Su <pingsutw@apache.org>
super().__init__("Typed Union", typing.Union)

@staticmethod
def get_sub_type(t: Type[T]) -> Type[T]:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this method does not really make sense since a Union type only ever has multiple "sub types"

i.e. Union[x] is by definition equivalent to x

I think the way this method is used in get_literal_type implies that the type signature should actually be (t: Type[T]) -> List[Type] instead

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will update it, thanks

if hasattr(t, "__origin__") and t.__origin__ is typing.Union: # type: ignore
if hasattr(t, "__args__"):
return t.__args__ # type: ignore
raise ValueError("Only generic univariate typing.Union[T] type is supported.")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see above: only polyvariate Union types are meaningful

for v in st:
try:
val = TypeEngine.to_python_value(ctx, lv, v)
if val:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

val could be None if this is an Optional. None should not indicate failure

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

val will be None only when Literal is None. if Literal is None, we will return None in line 634.
So we should return val if and only if val has value here, otherwise, we should raise an error.

@EngHabu

EngHabu commented Dec 30, 2021

Copy link
Copy Markdown
Collaborator

Closing in favor of #763

@EngHabu EngHabu closed this Dec 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants