Skip to content

[AOT] Initial implementation of --unpacked-api - #8023

Merged
areusch merged 9 commits into
apache:mainfrom
Mousius:untyped-operator-pass
Jun 4, 2021
Merged

[AOT] Initial implementation of --unpacked-api#8023
areusch merged 9 commits into
apache:mainfrom
Mousius:untyped-operator-pass

Conversation

@Mousius

@MousiusMousius commented May 12, 2021

Copy link
Copy Markdown
Member

Based on the discussions in the AOT embedded improvements RFC, this adds a flag to the target which changes the internal operators to an unpacked API. The unpacked API spreads the input buffers across the operator function, for example:

int32_t operator(void* arg0, void* arg1);

As opposed to the traditional packed API:

int32_t operator(void** args);

Unaffected is the entrypoint function, which retains a packed API for compatibility with other parts of TVM. The entrypoint function is generated as part of the metadata as suggested by @tqchen so we can easily swap it for implementing --micro-entrypoint.

cc: @giuseros@mbaret@manupa-arm@areusch@tqchen

giuseros
giuseros requested changes May 18, 2021
View reviewed changes

@giuserosgiuseros left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job @Mousius ! Did a first pass, but didn't spot anything major.

Comment threadsrc/relay/backend/aot_executor_codegen.cc Outdated
Comment threadsrc/relay/backend/aot_executor_codegen.cc
Comment threadsrc/relay/backend/aot_executor_codegen.cc Outdated
Comment threadtests/python/relay/aot/test_crt_aot.py Outdated
@MousiusMousius changed the title [AOT] Initial implementation of --no-typed-operators[AOT] Initial implementation of --typed-operatorsMay 21, 2021
@Mousius
Mousiusforce-pushed the untyped-operator-pass branch from 814ed20 to 40b964dCompareMay 24, 2021 08:53
@Mousius

Mousius commented May 24, 2021

Copy link
Copy Markdown
MemberAuthor

@giuseros everythings back to green, could you take another look? 😸

giuseros
giuseros approved these changes May 24, 2021
View reviewed changes

@giuserosgiuseros left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, only a very minor comment left for me

Comment threadsrc/target/source/source_module.cc Outdated
areusch
areusch requested changes May 24, 2021
View reviewed changes

@areuschareusch left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Mousius did an initial pass

Comment threadsrc/relay/backend/aot_executor_codegen.cc Outdated
Comment threadsrc/driver/driver_api.cc Outdated
Comment threadsrc/tir/transforms/make_unpacked_api.cc
@MousiusMousius changed the title [AOT] Initial implementation of --typed-operators[AOT] Initial implementation of --unpacked-apiMay 26, 2021
@areuschareusch mentioned this pull request May 27, 2021
Closed
17 tasks
Mousius added 8 commits June 1, 2021 10:27
@Mousius
[AOT] Initial implementation of --no-typed-operators
d0842df
Based on the discussions in the AOT embedded improvements RFC, this adds a flag to the target which changes the internal operators to an unpacked API. The unpacked API spreads the input buffers across the operator function, for example:
int32_t operator(void* arg0, void* arg1);
As opposed to the traditional packed API:
int32_t operator(void** args);
Uneffected is the entrypoint function, which retains a packed API for
compatibility with other parts of TVM. This is done by changing the
passes taken by none entrypoint (CallingConv::kEntryPoint) functions.
@Mousius
Move entrypoint generation outside of main passes
1f03e8f
This removes the logic for deciding the entrypoint from the compiler
passes and instead moves it into the metadata code generation. By moving
the generation, we can generate a variety of entrypoints on top of the
compiler output (such as the micro entrypoint discussed in the RFC).
@Mousius
Use buffers in make_unpacked_api tests
00000ba
@Mousius
Enable --no-typed-operators for llvm
af6e766
@Mousius
Change --no-typed-operators to --typed-operators=0 to match other opt…
c468c46
…ions
@Mousius
Refactor typed-operators lookup into use_typed_operators_
95b0204
(Also contains minor clean up of output variables)
@Mousius
Rename --typed-operators to --unpacked-api
7344269
(Also moves the entrypoint name to a constant)
@Mousius
Move all properties into init list to avoid double init
d1e0ab2
@Mousius
Mousiusforce-pushed the untyped-operator-pass branch from 9dcad5b to d1e0ab2CompareJune 1, 2021 09:27
@Mousius

Mousius commented Jun 1, 2021

Copy link
Copy Markdown
MemberAuthor

@areusch I think this is good to go, could you take another look?

areusch
areusch requested changes Jun 1, 2021
View reviewed changes

@areuschareusch left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @Mousius , apologies for the delay. just a couple small comments on this one.

Comment threadsrc/target/target_kind.cc Outdated
Comment threadsrc/relay/backend/aot_executor_codegen.cc Outdated
Comment threadtests/python/relay/aot/test_crt_aot.py Outdated
Comment threadinclude/tvm/runtime/module.h Outdated
@Mousius

Mousius commented Jun 3, 2021

Copy link
Copy Markdown
MemberAuthor

@areusch I think I've incorporated all the changes and CI is still green 🎉

@areusch
areusch merged commit a769ece into apache:mainJun 4, 2021
@areusch

areusch commented Jun 4, 2021

Copy link
Copy Markdown
Contributor

thanks @Mousius, the PR is now merged!

trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Jun 17, 2021
@MousiusTrevor Morris
[AOT] Initial implementation of --unpacked-api (apache#8023)
9e0d8aa
* [AOT] Initial implementation of --no-typed-operators
Based on the discussions in the AOT embedded improvements RFC, this adds a flag to the target which changes the internal operators to an unpacked API. The unpacked API spreads the input buffers across the operator function, for example:
int32_t operator(void* arg0, void* arg1);
As opposed to the traditional packed API:
int32_t operator(void** args);
Uneffected is the entrypoint function, which retains a packed API for
compatibility with other parts of TVM. This is done by changing the
passes taken by none entrypoint (CallingConv::kEntryPoint) functions.
* Move entrypoint generation outside of main passes
This removes the logic for deciding the entrypoint from the compiler
passes and instead moves it into the metadata code generation. By moving
the generation, we can generate a variety of entrypoints on top of the
compiler output (such as the micro entrypoint discussed in the RFC).
* Use buffers in make_unpacked_api tests
* Enable --no-typed-operators for llvm
* Change --no-typed-operators to --typed-operators=0 to match other options
* Refactor typed-operators lookup into use_typed_operators_
(Also contains minor clean up of output variables)
* Rename --typed-operators to --unpacked-api
(Also moves the entrypoint name to a constant)
* Move all properties into init list to avoid double init
* Remove AutoTVM breaking default and improve clarity
trevor-m pushed a commit to neo-ai/tvm that referenced this pull request Jun 17, 2021
@Mousius@trevor-m
[AOT] Initial implementation of --unpacked-api (apache#8023)
1502032
* [AOT] Initial implementation of --no-typed-operators
Based on the discussions in the AOT embedded improvements RFC, this adds a flag to the target which changes the internal operators to an unpacked API. The unpacked API spreads the input buffers across the operator function, for example:
int32_t operator(void* arg0, void* arg1);
As opposed to the traditional packed API:
int32_t operator(void** args);
Uneffected is the entrypoint function, which retains a packed API for
compatibility with other parts of TVM. This is done by changing the
passes taken by none entrypoint (CallingConv::kEntryPoint) functions.
* Move entrypoint generation outside of main passes
This removes the logic for deciding the entrypoint from the compiler
passes and instead moves it into the metadata code generation. By moving
the generation, we can generate a variety of entrypoints on top of the
compiler output (such as the micro entrypoint discussed in the RFC).
* Use buffers in make_unpacked_api tests
* Enable --no-typed-operators for llvm
* Change --no-typed-operators to --typed-operators=0 to match other options
* Refactor typed-operators lookup into use_typed_operators_
(Also contains minor clean up of output variables)
* Rename --typed-operators to --unpacked-api
(Also moves the entrypoint name to a constant)
* Move all properties into init list to avoid double init
* Remove AutoTVM breaking default and improve clarity
@junrushaojunrushao mentioned this pull request Nov 1, 2021
Closed
@LunderbergLunderberg mentioned this pull request Sep 26, 2022
Merged
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@areuschareuschareusch approved these changes
@giuserosgiuserosgiuseros approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Mousius@areusch@giuseros