feat: Add allOf support for model definitions BNCH-18722 by packyg · Pull Request #12 · benchling/openapi-python-client

packyg · 2020-12-05T00:27:18Z

Collapses the child elements into one, without class heirarchy, mixins, etc

This is a replaying of 2670d11 (first implementation) and 9f5b95a (a bugfix) onto the new main-v.0.7.0, modified for the refactored upstream.

This should bring main-v.0.7.0 up to par with main for the features we implemented in our fork (dropping our Unset implementation for theirs)

We cannot really cut over to the upstream until a couple of issues are addressed:

Add additional property support by subscripting model openapi-generators/openapi-python-client#252 - fixes enums in QS params. The current upstream would break listing endpoints as it would request e.g.
/api/v2/custom-entities?pageSize=2&sort=ListCustomEntitiesSort.MODIFIEDATDESC
Bugfix: Implement __str__ for enum props to fix query string openapi-generators/openapi-python-client#259 - adds additionalProperties support. The current upstream switched from dicts to empty models for unmodeled properties so we can't read or write entity schema fields with the current upstream.

See also: openapi-generators#98 (upstream issue by @dtkav) that this implements - we should try upstreaming this if all goes well

Combines the child elements into one, without class heirarchy, mixins, etc

packyg · 2020-12-05T00:32:22Z

@adamrp is a non-blocking reviewer - just spreading context on the codegen/SDK

bowenwr

What a journey. Thanks for all your efforts on this @packyg.

I generated a client with this and tried running a subset of our integration tests and they fail. In general, this PR looks OK to me but in 296ffb6 we introduce something like if not isinstance(some_prop, Unset): which is causing an error.

For example in list_boxes.py generated from this:

 json_sort: Union[Unset, ListBoxesSort] = UNSET
if not isinstance(sort, Unset):
json_sort = sort

Causes TypeError: isinstance() arg 2 must be a type or tuple of types. I think we need to be using sort == UNSET.

Something to visit next week as we close in on syncing everything up.

dtkav

Nice one!

dtkav · 2020-12-07T05:44:13Z

+ new_optional_props = [op for op in self.optional_properties if op.name != prop.name]
+ self.optional_properties.clear()
+ self.optional_properties.extend(new_optional_props)


out of curiosity, what's the difference between this and
self.optional_properties = [op for op in self.optional_properties if op.name != prop.name]
?

@dtkav There isn't, except this class is marked with @attr.s(auto_attribs=True, frozen=True) so errors if I attempt to set the property

Ah, TIL - thanks!

packyg · 2020-12-07T18:50:19Z

Causes TypeError: isinstance() arg 2 must be a type or tuple of types. I think we need to be using sort == UNSET.
Something to visit next week as we close in on syncing everything up.

Hmm I'll need to look at this more. The upstream is using the isinstance pattern and it seems to be working for them

adamrp · 2020-12-07T21:11:28Z

Causes TypeError: isinstance() arg 2 must be a type or tuple of types. I think we need to be using sort == UNSET.
Something to visit next week as we close in on syncing everything up.
Hmm I'll need to look at this more. The upstream is using the isinstance pattern and it seems to be working for them

Don't quote me on this, but I think isinstance has to take Python types (not Typing types) -- e.g., a class name or a tuple of class names

packyg · 2020-12-07T21:12:59Z

Don't quote me on this, but I think isinstance has to take Python types (not Typing types) -- e.g., a class name or a tuple of class names

That's what I don't understand - Unset is a class, which UNSET is an instance of

adamrp

Approving, although I wish I knew why isinstance is failing... 🤔

bowenwr · 2020-12-07T22:37:58Z

Approving, although I wish I knew why isinstance is failing... 🤔

In my example it's because sort is passed as None which causes {NameError}name 'Unset' is not defined.

The generated client is looking for Union[Unset, ListBoxesSort]. We could change all the defaults in the SDK to use UNSET but I hate that it then exposes a pretty specific implementation detail.

Maybe we could persuade the maintainers to also check is not None but more likely I guess we could map None -> Unset for our cases where needed.

packyg · 2020-12-07T22:49:21Z

Maybe we could persuade the maintainers to also check is not None but more likely I guess we could map None -> Unset for our cases where needed.

@bowenwr We don't want them to do that, at least for JSON serialization, do we? If we did that it would be treating Unset the same as None and we're back to not having a distinction between unset/semantically meaningful None

As for URL and QS parameters -that's a different story, I could see the use in treating UNSET and None the same.

Still don't know why checking isinstance(None, Unset) causes a NameError - did Unset not get imported?

As far as for now - I think it'd be OK for us to use this implementation detail in the SDK since it's so tightly coupled to the generated client.

bowenwr · 2020-12-07T23:00:14Z

Maybe we could persuade the maintainers to also check is not None but more likely I guess we could map None -> Unset for our cases where needed.
@bowenwr We don't want them to do that, at least for JSON serialization, do we? If we did that it would be treating Unset the same as None and we're back to not having a distinction between unset/semantically meaningful None
As for URL and QS parameters -that's a different story, I could see the use in treating UNSET and None the same.
Still don't know why checking isinstance(None, Unset) causes a NameError - did Unset not get imported?
As far as for now - I think it'd be OK for us to use this implementation detail in the SDK since it's so tightly coupled to the generated client.

@packyg

As for URL and QS parameters -that's a different story, I could see the use in treating UNSET and None the same.

Yes, this is the case to which I'm referring. Specifically a query param. For JSON serialization, you're correct.

did Unset not get imported?

It's imported.

Generated endpoint:

fromtypingimportAny, Dict, Optional, Unionimporthttpxfrom ...clientimportClientfrom ...models.bad_request_errorimportBadRequestErrorfrom ...models.box_listimportBoxListfrom ...models.list_boxes_sortimportListBoxesSortfrom ...typesimportUNSET, Response, Unsetdef_get_kwargs(
*,
client: Client,
page_size: Union[Unset, int] =50,
next_token: Union[Unset, str] =UNSET,
sort: Union[Unset, ListBoxesSort] =ListBoxesSort.MODIFIEDATDESC,
schema_id: Union[Unset, str] =UNSET,
modified_at: Union[Unset, str] =UNSET,
name: Union[Unset, str] =UNSET,
name_includes: Union[Unset, str] =UNSET,
empty_positions: Union[Unset, int] =UNSET,
empty_positionsgte: Union[Unset, int] =UNSET,
empty_positionsgt: Union[Unset, int] =UNSET,
empty_positionslte: Union[Unset, int] =UNSET,
empty_positionslt: Union[Unset, int] =UNSET,
empty_containers: Union[Unset, int] =UNSET,
empty_containersgte: Union[Unset, int] =UNSET,
empty_containersgt: Union[Unset, int] =UNSET,
empty_containerslte: Union[Unset, int] =UNSET,
empty_containerslt: Union[Unset, int] =UNSET,
ancestor_storage_id: Union[Unset, str] =UNSET,
storage_contents_id: Union[Unset, str] =UNSET,
storage_contents_ids: Union[Unset, str] =UNSET,
archive_reason: Union[Unset, str] =UNSET,
ids: Union[Unset, str] =UNSET,
barcodes: Union[Unset, str] =UNSET,
) ->Dict[str, Any]:
url="{}/boxes".format(client.base_url)
headers: Dict[str, Any] =client.get_headers()
json_sort: Union[Unset, ListBoxesSort] =UNSETifnotisinstance(sort, Unset):
json_sort=sortparams: Dict[str, Any] = {}
ifpage_sizeisnotUNSET:
params["pageSize"] =page_sizeifnext_tokenisnotUNSET:
params["nextToken"] =next_tokenifsortisnotUNSET:
params["sort"] =json_sortifschema_idisnotUNSET:
params["schemaId"] =schema_idifmodified_atisnotUNSET:
params["modifiedAt"] =modified_atifnameisnotUNSET:
params["name"] =nameifname_includesisnotUNSET:
params["nameIncludes"] =name_includesifempty_positionsisnotUNSET:
params["emptyPositions"] =empty_positionsifempty_positionsgteisnotUNSET:
params["emptyPositions.gte"] =empty_positionsgteifempty_positionsgtisnotUNSET:
params["emptyPositions.gt"] =empty_positionsgtifempty_positionslteisnotUNSET:
params["emptyPositions.lte"] =empty_positionslteifempty_positionsltisnotUNSET:
params["emptyPositions.lt"] =empty_positionsltifempty_containersisnotUNSET:
params["emptyContainers"] =empty_containersifempty_containersgteisnotUNSET:
params["emptyContainers.gte"] =empty_containersgteifempty_containersgtisnotUNSET:
params["emptyContainers.gt"] =empty_containersgtifempty_containerslteisnotUNSET:
params["emptyContainers.lte"] =empty_containerslteifempty_containersltisnotUNSET:
params["emptyContainers.lt"] =empty_containersltifancestor_storage_idisnotUNSET:
params["ancestorStorageId"] =ancestor_storage_idifstorage_contents_idisnotUNSET:
params["storageContentsId"] =storage_contents_idifstorage_contents_idsisnotUNSET:
params["storageContentsIds"] =storage_contents_idsifarchive_reasonisnotUNSET:
params["archiveReason"] =archive_reasonifidsisnotUNSET:
params["ids"] =idsifbarcodesisnotUNSET:
params["barcodes"] =barcodesreturn {
"url": url,
"headers": headers,
"cookies": client.get_cookies(),
"timeout": client.get_timeout(),
"params": params,
}
def_parse_response(*, response: httpx.Response) ->Optional[Union[BoxList, BadRequestError]]:
ifresponse.status_code==200:
response_200=BoxList.from_dict(response.json())
returnresponse_200ifresponse.status_code==400:
response_400=BadRequestError.from_dict(response.json())
returnresponse_400returnNonedef_build_response(*, response: httpx.Response) ->Response[Union[BoxList, BadRequestError]]:
returnResponse(
status_code=response.status_code,
content=response.content,
headers=response.headers,
parsed=_parse_response(response=response),
)
defsync_detailed(
*,
client: Client,
page_size: Union[Unset, int] =50,
next_token: Union[Unset, str] =UNSET,
sort: Union[Unset, ListBoxesSort] =ListBoxesSort.MODIFIEDATDESC,
schema_id: Union[Unset, str] =UNSET,
modified_at: Union[Unset, str] =UNSET,
name: Union[Unset, str] =UNSET,
name_includes: Union[Unset, str] =UNSET,
empty_positions: Union[Unset, int] =UNSET,
empty_positionsgte: Union[Unset, int] =UNSET,
empty_positionsgt: Union[Unset, int] =UNSET,
empty_positionslte: Union[Unset, int] =UNSET,
empty_positionslt: Union[Unset, int] =UNSET,
empty_containers: Union[Unset, int] =UNSET,
empty_containersgte: Union[Unset, int] =UNSET,
empty_containersgt: Union[Unset, int] =UNSET,
empty_containerslte: Union[Unset, int] =UNSET,
empty_containerslt: Union[Unset, int] =UNSET,
ancestor_storage_id: Union[Unset, str] =UNSET,
storage_contents_id: Union[Unset, str] =UNSET,
storage_contents_ids: Union[Unset, str] =UNSET,
archive_reason: Union[Unset, str] =UNSET,
ids: Union[Unset, str] =UNSET,
barcodes: Union[Unset, str] =UNSET,
) ->Response[Union[BoxList, BadRequestError]]:
kwargs=_get_kwargs(
client=client,
page_size=page_size,
next_token=next_token,
sort=sort,
schema_id=schema_id,
modified_at=modified_at,
name=name,
name_includes=name_includes,
empty_positions=empty_positions,
empty_positionsgte=empty_positionsgte,
empty_positionsgt=empty_positionsgt,
empty_positionslte=empty_positionslte,
empty_positionslt=empty_positionslt,
empty_containers=empty_containers,
empty_containersgte=empty_containersgte,
empty_containersgt=empty_containersgt,
empty_containerslte=empty_containerslte,
empty_containerslt=empty_containerslt,
ancestor_storage_id=ancestor_storage_id,
storage_contents_id=storage_contents_id,
storage_contents_ids=storage_contents_ids,
archive_reason=archive_reason,
ids=ids,
barcodes=barcodes,
)
response=httpx.get(
**kwargs,
)
return_build_response(response=response)
defsync(
*,
client: Client,
page_size: Union[Unset, int] =50,
next_token: Union[Unset, str] =UNSET,
sort: Union[Unset, ListBoxesSort] =ListBoxesSort.MODIFIEDATDESC,
schema_id: Union[Unset, str] =UNSET,
modified_at: Union[Unset, str] =UNSET,
name: Union[Unset, str] =UNSET,
name_includes: Union[Unset, str] =UNSET,
empty_positions: Union[Unset, int] =UNSET,
empty_positionsgte: Union[Unset, int] =UNSET,
empty_positionsgt: Union[Unset, int] =UNSET,
empty_positionslte: Union[Unset, int] =UNSET,
empty_positionslt: Union[Unset, int] =UNSET,
empty_containers: Union[Unset, int] =UNSET,
empty_containersgte: Union[Unset, int] =UNSET,
empty_containersgt: Union[Unset, int] =UNSET,
empty_containerslte: Union[Unset, int] =UNSET,
empty_containerslt: Union[Unset, int] =UNSET,
ancestor_storage_id: Union[Unset, str] =UNSET,
storage_contents_id: Union[Unset, str] =UNSET,
storage_contents_ids: Union[Unset, str] =UNSET,
archive_reason: Union[Unset, str] =UNSET,
ids: Union[Unset, str] =UNSET,
barcodes: Union[Unset, str] =UNSET,
) ->Optional[Union[BoxList, BadRequestError]]:
""" List boxes """returnsync_detailed(
client=client,
page_size=page_size,
next_token=next_token,
sort=sort,
schema_id=schema_id,
modified_at=modified_at,
name=name,
name_includes=name_includes,
empty_positions=empty_positions,
empty_positionsgte=empty_positionsgte,
empty_positionsgt=empty_positionsgt,
empty_positionslte=empty_positionslte,
empty_positionslt=empty_positionslt,
empty_containers=empty_containers,
empty_containersgte=empty_containersgte,
empty_containersgt=empty_containersgt,
empty_containerslte=empty_containerslte,
empty_containerslt=empty_containerslt,
ancestor_storage_id=ancestor_storage_id,
storage_contents_id=storage_contents_id,
storage_contents_ids=storage_contents_ids,
archive_reason=archive_reason,
ids=ids,
barcodes=barcodes,
).parsedasyncdefasyncio_detailed(
*,
client: Client,
page_size: Union[Unset, int] =50,
next_token: Union[Unset, str] =UNSET,
sort: Union[Unset, ListBoxesSort] =ListBoxesSort.MODIFIEDATDESC,
schema_id: Union[Unset, str] =UNSET,
modified_at: Union[Unset, str] =UNSET,
name: Union[Unset, str] =UNSET,
name_includes: Union[Unset, str] =UNSET,
empty_positions: Union[Unset, int] =UNSET,
empty_positionsgte: Union[Unset, int] =UNSET,
empty_positionsgt: Union[Unset, int] =UNSET,
empty_positionslte: Union[Unset, int] =UNSET,
empty_positionslt: Union[Unset, int] =UNSET,
empty_containers: Union[Unset, int] =UNSET,
empty_containersgte: Union[Unset, int] =UNSET,
empty_containersgt: Union[Unset, int] =UNSET,
empty_containerslte: Union[Unset, int] =UNSET,
empty_containerslt: Union[Unset, int] =UNSET,
ancestor_storage_id: Union[Unset, str] =UNSET,
storage_contents_id: Union[Unset, str] =UNSET,
storage_contents_ids: Union[Unset, str] =UNSET,
archive_reason: Union[Unset, str] =UNSET,
ids: Union[Unset, str] =UNSET,
barcodes: Union[Unset, str] =UNSET,
) ->Response[Union[BoxList, BadRequestError]]:
kwargs=_get_kwargs(
client=client,
page_size=page_size,
next_token=next_token,
sort=sort,
schema_id=schema_id,
modified_at=modified_at,
name=name,
name_includes=name_includes,
empty_positions=empty_positions,
empty_positionsgte=empty_positionsgte,
empty_positionsgt=empty_positionsgt,
empty_positionslte=empty_positionslte,
empty_positionslt=empty_positionslt,
empty_containers=empty_containers,
empty_containersgte=empty_containersgte,
empty_containersgt=empty_containersgt,
empty_containerslte=empty_containerslte,
empty_containerslt=empty_containerslt,
ancestor_storage_id=ancestor_storage_id,
storage_contents_id=storage_contents_id,
storage_contents_ids=storage_contents_ids,
archive_reason=archive_reason,
ids=ids,
barcodes=barcodes,
)
asyncwithhttpx.AsyncClient() as_client:
response=await_client.get(**kwargs)
return_build_response(response=response)
asyncdefasyncio(
*,
client: Client,
page_size: Union[Unset, int] =50,
next_token: Union[Unset, str] =UNSET,
sort: Union[Unset, ListBoxesSort] =ListBoxesSort.MODIFIEDATDESC,
schema_id: Union[Unset, str] =UNSET,
modified_at: Union[Unset, str] =UNSET,
name: Union[Unset, str] =UNSET,
name_includes: Union[Unset, str] =UNSET,
empty_positions: Union[Unset, int] =UNSET,
empty_positionsgte: Union[Unset, int] =UNSET,
empty_positionsgt: Union[Unset, int] =UNSET,
empty_positionslte: Union[Unset, int] =UNSET,
empty_positionslt: Union[Unset, int] =UNSET,
empty_containers: Union[Unset, int] =UNSET,
empty_containersgte: Union[Unset, int] =UNSET,
empty_containersgt: Union[Unset, int] =UNSET,
empty_containerslte: Union[Unset, int] =UNSET,
empty_containerslt: Union[Unset, int] =UNSET,
ancestor_storage_id: Union[Unset, str] =UNSET,
storage_contents_id: Union[Unset, str] =UNSET,
storage_contents_ids: Union[Unset, str] =UNSET,
archive_reason: Union[Unset, str] =UNSET,
ids: Union[Unset, str] =UNSET,
barcodes: Union[Unset, str] =UNSET,
) ->Optional[Union[BoxList, BadRequestError]]:
""" List boxes """return (
awaitasyncio_detailed(
client=client,
page_size=page_size,
next_token=next_token,
sort=sort,
schema_id=schema_id,
modified_at=modified_at,
name=name,
name_includes=name_includes,
empty_positions=empty_positions,
empty_positionsgte=empty_positionsgte,
empty_positionsgt=empty_positionsgt,
empty_positionslte=empty_positionslte,
empty_positionslt=empty_positionslt,
empty_containers=empty_containers,
empty_containersgte=empty_containersgte,
empty_containersgt=empty_containersgt,
empty_containerslte=empty_containerslte,
empty_containerslt=empty_containerslt,
ancestor_storage_id=ancestor_storage_id,
storage_contents_id=storage_contents_id,
storage_contents_ids=storage_contents_ids,
archive_reason=archive_reason,
ids=ids,
barcodes=barcodes,
)
).parsed

bowenwr · 2020-12-07T23:22:31Z

To be clear, the comments about Unset shouldn't hold up this PR. I just wanna capture it as part of our known "closing the gap" series.

Collapses the child elements into one, without class heirarchy, mixins, etc This is a replaying of 2670d11 (first implementation) and 9f5b95a (a bugfix) onto the new `main-v.0.7.0`, modified for the refactored upstream. This should bring `main-v.0.7.1` up to par with `main` for the features we implemented in our fork (dropping our `Unset` implementation for theirs)

packyg added 5 commits December 4, 2020 14:04

Add allOf support for model definitions
307431c
Combines the child elements into one, without class heirarchy, mixins, etc

Fix schemas object building
e7c9a2e

Update existing tests
e0c734c

Add test for resolve_references
abbcee7

Update test_build_schemas for resolve_references on models
729851c

packyg requested review from adamrp, bowenwr and dtkav December 5, 2020 00:27

bowenwr approved these changes Dec 5, 2020
View reviewed changes

dtkav approved these changes Dec 7, 2020
View reviewed changes

adamrp approved these changes Dec 7, 2020
View reviewed changes

packyg changed the base branch from main-v.0.7.0 to main-v.0.7.1December 8, 2020 23:41

packyg changed the base branch from main-v.0.7.1 to main-v.0.7.0December 8, 2020 23:41

packyg merged commit 782bbd0 into main-v.0.7.0Dec 8, 2020

packyg deleted the update-allOf-support branch December 8, 2020 23:47

Uh oh!

Conversation

packyg commented Dec 5, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

packyg commented Dec 5, 2020

Uh oh!

bowenwr left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dtkav left a comment

Choose a reason for hiding this comment

Uh oh!

dtkavDec 7, 2020

Choose a reason for hiding this comment

Uh oh!

packygDec 7, 2020

Choose a reason for hiding this comment

Uh oh!

dtkavDec 7, 2020

Choose a reason for hiding this comment

Uh oh!

packyg commented Dec 7, 2020

Uh oh!

adamrp commented Dec 7, 2020

Uh oh!

packyg commented Dec 7, 2020

Uh oh!

adamrp left a comment

Choose a reason for hiding this comment

Uh oh!

bowenwr commented Dec 7, 2020

Uh oh!

packyg commented Dec 7, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bowenwr commented Dec 7, 2020

Uh oh!

bowenwr commented Dec 7, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

packyg commented Dec 5, 2020•
edited
Loading

bowenwr left a comment •
edited
Loading

packyg commented Dec 7, 2020•
edited
Loading