Uh oh!
There was an error while loading. Please reload this page.
Rename runtime-config to executor-config and add documentation for Model Library Format - #8270
Conversation
leandron
left a comment
There was a problem hiding this comment.
It looks very clear documentation. I think users would benefit also from a short code snippet giving an example of how to generate a MLF package.
Also a few nits and suggestions.
| -------------------------- | ||
| TVM traditionally exports generated libraries as Dynamic Shared Objects | ||
| (e.g. DLLs (Windows) or .so (linux)). Inference can be performed on those libraries by loading them |
There was a problem hiding this comment.
| (e.g. DLLs (Windows) or .so (linux)). Inference can be performed on those libraries by loading them | |
| (e.g. DLLs (Windows) or .so (linux)). Inferences can be performed on those libraries by loading them |
| into an executable using ``libtvm_runtime.so``. This process is very dependent on services provided | ||
| by traditional OS. | ||
| For deployment to unconventional platforms (e.g. those lacking traditional OS), the microTVM project |
There was a problem hiding this comment.
Maybe use embedded instead?
| For deployment to unconventional platforms (e.g. those lacking traditional OS), the microTVM project | |
| For deployment to embedded platforms (e.g. those lacking traditional OS), the microTVM project |
There was a problem hiding this comment.
i feel like it's not strictly limited to embedded though
There was a problem hiding this comment.
I agree.
to that end, I think we should word this as a general output format that produces what strictly TVM (code)generates. The creation of .so/.dll requires external toolchains called after tvm compilation -- such as LLVM and C compilers.
WDYT ?
| Model Library Format | ||
| ==================== |
There was a problem hiding this comment.
I think it would be beneficial to introduce MLF as an acronym for Model Lbrary Format early in the document, so that we can get people used to it?
| Model Library Format | |
| ==================== | |
| Model Library Format (MLF) | |
| ========================== |
There was a problem hiding this comment.
i haven't used the acronym yet in this doc though. but i agree it's an easy shorthand for the format. maybe it would make sense more in tvmc docs, where it's a command-line param? wdyt?
Mousius
commented
Jun 17, 2021
Would it be worth adding a JSON Schema for |
| * ``<target>`` - Identifies the TVM target on which the code should run. Currently, only ``host`` | ||
| is supported. | ||
| * ``<unique_name>`` - A unique slug identifying this file. Currently ``lib<n>``, with ``<n>>` an | ||
| autoincrementing integer. |
There was a problem hiding this comment.
maybe change to auto-incrementing?
| ``parameters`` | ||
| ^^^^^^^^^^^^^^ | ||
| Contains machine-parseable parameters. A variety of formats may be provided, but at present, only |
manupak
left a comment
There was a problem hiding this comment.
Looks good, just one more nit -- (bar other's comments)
| `Memory Usage Summary`_. | ||
| - ``model_name``: The name of this model (e.g. the ``name`` parameter supplied to | ||
| ``tvm.relay.build``). | ||
| - ``runtimes``: A list of runtimes supported by this model. Currently, this list is always |
areusch
commented
Jul 9, 2021
@leandron@manupa-arm please take another look! |
manupak
left a comment
There was a problem hiding this comment.
Did some (final) nit picking! hope you dont mind.
Mostly text changes.
| into an executable using ``libtvm_runtime.so``. This process is very dependent on services provided | ||
| by traditional OS. | ||
| For deployment to unconventional platforms (e.g. those lacking traditional OS), the microTVM project |
There was a problem hiding this comment.
I agree.
to that end, I think we should word this as a general output format that produces what strictly TVM (code)generates. The creation of .so/.dll requires external toolchains called after tvm compilation -- such as LLVM and C compilers.
WDYT ?
| For deployment to unconventional platforms (e.g. those lacking traditional OS), the microTVM project | ||
| can be used to export a generated library in pieces. In this case, microTVM provides another output | ||
| format, Model Library Format. Model Library Format is a tarball containing a file for each part of |
There was a problem hiding this comment.
I think we should not use "microTVM provides another output format", though we could use microTVM as an example this output format becomes useful.
| format, Model Library Format. Model Library Format is a tarball containing a file for each part of | ||
| the TVM compiler output. | ||
| What can be Exported |
There was a problem hiding this comment.
| What can be Exported | |
| What can be exported ? |
| Directory Layout | ||
| ---------------- | ||
| Model Library Format is traditionally contained within a tarball. All paths are relative to the root |
There was a problem hiding this comment.
| Model Library Format is traditionally contained within a tarball. All paths are relative to the root | |
| Model Library Format is contained within a tarball. All paths are relative to the root |
There was a problem hiding this comment.
nit: we dont have any other way. Do we?
| sub-target which describes that relay backend used for that ``device_type``. | ||
| - ``version``: A numeric version number that identifies the format used in this Model Library | ||
| Format. This number is incremented when the metadata structure or on-disk structure changes. | ||
| This document reflects version ``3``. |
There was a problem hiding this comment.
| This document reflects version ``3``. | |
| This document reflects version ``5``. |
areusch
commented
Jul 15, 2021
@manupa-arm it won't let me continue your earlier thread, but basically i'd agree with that idea in the medium term however we cannot yet export BYOC Module in MLF. once that is possible, i think we should promote it to a general TVM output format. |
areusch
commented
Jul 15, 2021
@manupa-arm please take another look |
leandron
commented
Jul 20, 2021
Thanks @areusch@mehrdadh@Mousius@manupa-arm, this is merged now! |
…del Library Format (apache#8270) * Rename runtime-config to executor-config. * Add documentation. * address comments, make tests pass * fix unit test * fix sphinx doc errors * address manupa comments
…del Library Format (apache#8270) * Rename runtime-config to executor-config. * Add documentation. * address comments, make tests pass * fix unit test * fix sphinx doc errors * address manupa comments
This PR:
runtime-configtoexecutor-configto match Rename GraphRuntime to GraphExecutor #7653.@mehrdadh@leandron@gromero@manupa-arm@jwfromm@guberti@Mousius@giuseros