[tvmc] command line driver 'compile' (part 2/4) by leandron · Pull Request #6302 · apache/tvm

leandron · 2020-08-19T08:30:16Z

This is a follow-up PR on top of #6112, introducing compile subcommand on tvmc or python -m tvm.driver.tvmc

Add 'compile' subcommand into tvmc (tvm.driver.tvmc)
Add frontends: Keras, ONNX, TensorFlow, tflite, PyTorch
Add tests for the 'compile' subcommand
Add frontend dependencies on setup.py to make TVM python package ready-to-use on install

Known limitations:

It covers a subset of TVM supported frontends: Keras, ONNX, tflite, TensorFlow and PyTorch
It assumes graph runtime only

There are still 2 patches to be submitted: tvmc tune and tvmc run, both depending on this.

In case you want to have a look and test compile, you can use the --dump-codegen ll or --dump-codegen asm or --dump-codegen relay to check the output module as source.

A sample usage would look like (assumes TVM is built and working):

wget https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz
tar xvvzf mobilenet_v1_1.0_224_quant.tgz
python -m tvm.driver.tvmc compile --target=llvm --output model.tar --dump-code relay mobilenet_v1_1.0_224_quant.tflite
cat model.relay
# should output mobilenet as relay

@tqchen @comaniac @jroesch, can you have a look?

leandron · 2020-08-19T08:34:05Z

Also cc @tom-gall who sent a patch on top of my previous PR adding supporting AArch32. Can you send that one again, once this one is merged?

tom-gall · 2020-08-19T13:44:13Z

Also cc @tom-gall who sent a patch on top of my previous PR adding supporting AArch32. Can you send that one again, once this one is merged?

Sure happy to.

comaniac

Reviewed the argument interface only and have some comments. I'd prefer to have consents on the interface before diving into the implementations.

tqchen · 2020-08-20T18:22:37Z

One thing to note is that given we are updating the target convention to use the json as an input, it might be helpful to think about what is the target that can be accepted from the command line. cc @comaniac @junrushao1994

junrushao · 2020-08-20T18:24:46Z

@tqchen I am thinking about unifying target creation from legacy string, JSON file and tags.

comaniac · 2020-08-20T19:03:08Z

One thing to note is that given we are updating the target convention to use the json as an input, it might be helpful to think about what is the target that can be accepted from the command line. cc @comaniac @junrushao1994

Zhi and I had a discussion with ARM folks today and here is a rough proposal:

--target could accept either a target or a JSON file path.
In the case of target string, it should match either a predefined tag (e.g., rasp4b), or it has to be a valid target string that can be parsed by target.create().
In the case of JSON file, it must be valid to be parsed by target JSON parser.

As a result, the users that know nothing about the target can simply provide a tag. On ther other hand, advance users can still specify targets in any supported formats.

leandron · 2020-09-09T16:03:13Z

I updated it removing a skipped test due to #6410, which is now fixed.

leandron · 2020-09-10T15:45:03Z

There was an issue we discovered with ConvertLayout when running the new tests introduced here, once #6442 is merged, all the tests should pass.

@comaniac would you mind having another look into this PR?

comaniac

Thanks for the changes. It looks much better now.

comaniac · 2020-09-10T18:04:29Z

cc @masahi @junrushao1994 @tqchen for a final review.

leandron · 2020-09-11T10:13:23Z

Thanks @comaniac - I submitted a new version incorporating pretty much all the feedback (apart from the Target one, details in the comment).

One test is still expected to fail due to #6442, which is blocked by the Python linting massive refactoring #6448 to come.

leandron · 2020-09-16T15:17:58Z

I recently rebased with the latest changes and un-skipped the test fixed by #6442

@tqchen @comaniac @junrushao1994 would you mind having another look into this PR?

comaniac

LGTM. We can merge it after passing the CI.
Thanks for the efforts :)

* Add 'compile' subcommand into tvmc (tvm.driver.tvmc) * Add frontends: Keras, ONNX, TensorFlow, tflite, PyTorch * Add tests for the 'compile' subcommand * Enable command line driver tests as part of integration tests * Skip tests if the cross-compilation toolchain is not installed Co-authored-by: Marcus Shawcroft <marcus.shawcroft@arm.com> Co-authored-by: Matthew Barrett <Matthew.Barrett@arm.com> Co-authored-by: Dmitriy Smirnov <dmitriy.smirnov@arm.com> Co-authored-by: Luke Hutton <luke.hutton@arm.com> Co-authored-by: Giuseppe Rossini <giuseppe.rossini@arm.com> Co-authored-by: Matthew Barrett <matthew.barrett@arm.com> Co-authored-by: Elen Kalda <elen.kalda@arm.com> Co-authored-by: Ramana Radhakrishnan <ramana.radhakrishnan@arm.com> Co-authored-by: Jeremy Johnson <jeremy.johnson@arm.com> Co-authored-by: Ina Dobreva <Ina.Dobreva@arm.com>

…ages

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

tqchen · 2020-09-18T18:37:41Z

Thanks @leandron @comaniac @masahi !

* [tvmc] command line driver 'compile' (part 2/4) * Add 'compile' subcommand into tvmc (tvm.driver.tvmc) * Add frontends: Keras, ONNX, TensorFlow, tflite, PyTorch * Add tests for the 'compile' subcommand * Enable command line driver tests as part of integration tests * Skip tests if the cross-compilation toolchain is not installed Co-authored-by: Marcus Shawcroft <marcus.shawcroft@arm.com> Co-authored-by: Matthew Barrett <Matthew.Barrett@arm.com> Co-authored-by: Dmitriy Smirnov <dmitriy.smirnov@arm.com> Co-authored-by: Luke Hutton <luke.hutton@arm.com> Co-authored-by: Giuseppe Rossini <giuseppe.rossini@arm.com> Co-authored-by: Matthew Barrett <matthew.barrett@arm.com> Co-authored-by: Elen Kalda <elen.kalda@arm.com> Co-authored-by: Ramana Radhakrishnan <ramana.radhakrishnan@arm.com> Co-authored-by: Jeremy Johnson <jeremy.johnson@arm.com> Co-authored-by: Ina Dobreva <Ina.Dobreva@arm.com> * tvmc: adjust TODOs * tvmc: fix linting errors * Address code-review comments * Adjust pytest fixture to not break when there is no tensorflow * Fix frontend tests, to cope with different frameworks in different images * Apply suggestions from code review Co-authored-by: Cody Yu <comaniac0422@gmail.com> * Fix lint and code-review issues * Re-format with black. * tvmc: Move dependencies to extras_requires Co-authored-by: Marcus Shawcroft <marcus.shawcroft@arm.com> Co-authored-by: Matthew Barrett <Matthew.Barrett@arm.com> Co-authored-by: Dmitriy Smirnov <dmitriy.smirnov@arm.com> Co-authored-by: Luke Hutton <luke.hutton@arm.com> Co-authored-by: Giuseppe Rossini <giuseppe.rossini@arm.com> Co-authored-by: Elen Kalda <elen.kalda@arm.com> Co-authored-by: Ramana Radhakrishnan <ramana.radhakrishnan@arm.com> Co-authored-by: Jeremy Johnson <jeremy.johnson@arm.com> Co-authored-by: Ina Dobreva <Ina.Dobreva@arm.com> Co-authored-by: Cody Yu <comaniac0422@gmail.com>

OneIsAll-w · 2021-03-15T08:34:09Z

tvmc first-run with 'compile' problem
I’m attempting to compile the resnet-50-v2.7.onnx model using the command :
python -m tvm.driver.tvmc compile --target "llvm" --output compiled_module.tar resnet50-v2-7.onnx
and I’m running into an error:
WARNING:root:Attribute momentum is ignored in relay.sym.batch_norm
WARNING:root:Attribute spatial is ignored in relay.sym.batch_norm
Traceback (most recent call last):
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 24, in
tvmc.main.main()
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 92, in main
sys.exit(_main(sys.argv[1:]))
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 85, in _main
return args.func(args)
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 114, in drive_compile
args.desired_layout,
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 182, in compile_model
with open(target) as target_file:
IsADirectoryError: [Errno 21] Is a directory: 'llvm'
Therefore,I want to ask you this problem,Where might I be going wrong?

leandron · 2021-03-15T10:04:13Z

tvmc first-run with 'compile' problem
I’m attempting to compile the resnet-50-v2.7.onnx model using the command :
python -m tvm.driver.tvmc compile --target "llvm" --output compiled_module.tar resnet50-v2-7.onnx
and I’m running into an error:
WARNING:root:Attribute momentum is ignored in relay.sym.batch_norm
WARNING:root:Attribute spatial is ignored in relay.sym.batch_norm
Traceback (most recent call last):
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 24, in
tvmc.main.main()
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 92, in main
sys.exit(_main(sys.argv[1:]))
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 85, in _main
return args.func(args)
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 114, in drive_compile
args.desired_layout,
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 182, in compile_model
with open(target) as target_file:
IsADirectoryError: [Errno 21] Is a directory: 'llvm'
Therefore,I want to ask you this problem,Where might I be going wrong?

This is a small issue, that only reproduce because you probably have an llvm directory in your current directory. I just submitted a fix to it in #7663. Thanks for reporting.

OneIsAll-w · 2021-03-15T11:41:01Z

Thanks a lot @leandron
but I still have some problems in compiling the resnet-50-v2.7.onnx model using the command :
python -m tvm.driver.tvmc compile --target "llvm" --output compiled_module.tar resnet50-v2-7.onnx
and I’m running into an error:
WARNING:root:Attribute momentum is ignored in relay.sym.batch_norm
WARNING:root:Attribute spatial is ignored in relay.sym.batch_norm
Traceback (most recent call last):
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 24, in
tvmc.main.main()
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 92, in main
sys.exit(_main(sys.argv[1:]))
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 85, in _main
return args.func(args)
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 114, in drive_compile
args.desired_layout,
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 189, in compile_model
tvm_target = tvm.target.Target(target)
File "/home/tw/tvm/python/tvm/target/target.py", line 92, in init
self.init_handle_by_constructor(_ffi_api.Target, tag_or_str_or_dict)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/object.py", line 131, in init_handle_by_constructor
handle = init_by_constructor(fconstructor, args)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 246, in init_handle_by_constructor
values, tcodes, num_args = _make_tvm_args(args, temp_args)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 163, in _make_tvm_args
values[i].v_str = c_str(arg)
File "/home/tw/tvm/python/tvm/_ffi/base.py", line 88, in c_str
return ctypes.c_char_p(string.encode("utf-8"))
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 0-1: surrogates not allowed
Sorry，I still cannot solve this problem

leandron · 2021-03-15T12:13:42Z

Thanks a lot @leandron
but I still have some problems in compiling the resnet-50-v2.7.onnx model using the command :
python -m tvm.driver.tvmc compile --target "llvm" --output compiled_module.tar resnet50-v2-7.onnx
and I’m running into an error:
WARNING:root:Attribute momentum is ignored in relay.sym.batch_norm
WARNING:root:Attribute spatial is ignored in relay.sym.batch_norm
Traceback (most recent call last):
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 24, in
tvmc.main.main()
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 92, in main
sys.exit(_main(sys.argv[1:]))
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 85, in _main
return args.func(args)
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 114, in drive_compile
args.desired_layout,
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 189, in compile_model
tvm_target = tvm.target.Target(target)
File "/home/tw/tvm/python/tvm/target/target.py", line 92, in init
self.init_handle_by_constructor(_ffi_api.Target, tag_or_str_or_dict)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/object.py", line 131, in init_handle_by_constructor
handle = init_by_constructor(fconstructor, args)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 246, in init_handle_by_constructor
values, tcodes, num_args = _make_tvm_args(args, temp_args)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 163, in _make_tvm_args
values[i].v_str = c_str(arg)
File "/home/tw/tvm/python/tvm/_ffi/base.py", line 88, in c_str
return ctypes.c_char_p(string.encode("utf-8"))
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 0-1: surrogates not allowed
Sorry，I still cannot solve this problem

It seems it has something to do with the actual string you're sending on the command line. It is not strictly a bug in tvmc at this point. Can you please file an issue so that we can investigate that?

OneIsAll-w · 2021-03-15T12:20:49Z

Thanks a lot @leandron
but I still have some problems in compiling the resnet-50-v2.7.onnx model using the command :
python -m tvm.driver.tvmc compile --target "llvm" --output compiled_module.tar resnet50-v2-7.onnx
and I’m running into an error:
WARNING:root:Attribute momentum is ignored in relay.sym.batch_norm
WARNING:root:Attribute spatial is ignored in relay.sym.batch_norm
Traceback (most recent call last):
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/tw/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 24, in
tvmc.main.main()
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 92, in main
sys.exit(_main(sys.argv[1:]))
File "/home/tw/tvm/python/tvm/driver/tvmc/main.py", line 85, in _main
return args.func(args)
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 114, in drive_compile
args.desired_layout,
File "/home/tw/tvm/python/tvm/driver/tvmc/compiler.py", line 189, in compile_model
tvm_target = tvm.target.Target(target)
File "/home/tw/tvm/python/tvm/target/target.py", line 92, in init
self.init_handle_by_constructor(_ffi_api.Target, tag_or_str_or_dict)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/object.py", line 131, in init_handle_by_constructor
handle = init_by_constructor(fconstructor, args)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 246, in init_handle_by_constructor
values, tcodes, num_args = _make_tvm_args(args, temp_args)
File "/home/tw/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 163, in _make_tvm_args
values[i].v_str = c_str(arg)
File "/home/tw/tvm/python/tvm/_ffi/base.py", line 88, in c_str
return ctypes.c_char_p(string.encode("utf-8"))
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 0-1: surrogates not allowed
Sorry，I still cannot solve this problem
It seems it has something to do with the actual string you're sending on the command line. It is not strictly a bug in tvmc at this point. Can you please file an issue so that we can investigate that?

I didn't change anything, I still used the command:
python -m tvm.driver.tvmc compile --target "llvm" --output compiled_module.tar resnet50-v2-7.onnx

comaniac · 2021-03-15T17:08:04Z

@OneIsAll-w please do not use the closed PR for bug reporting and discussion. It would be better to create a post in https://discuss.tvm.apache.org/ and point to this PR.

leandron mentioned this pull request Aug 19, 2020
[TEST][FLAKY] test_auto_scheduler_cost_model.py::test_xgb_model #6305
Closed

comaniac requested changes Aug 19, 2020
View reviewed changes

ZihengJiang assigned comaniac Aug 21, 2020

ZihengJiang added status: need review status: review in progress and removed status: need review labels Aug 21, 2020

leandron mentioned this pull request Aug 21, 2020
[Target] Add python binding to new JSON target construction. #6315
Merged

comaniac requested changes Aug 28, 2020
View reviewed changes

This was referenced Sep 7, 2020
AttributeError during ConvertLayout to NHWC #6410
Closed
[FLAKY] relay/test_op_level1.py::test_binary_op failing intermittently #6417
Closed

comaniac requested changes Sep 10, 2020
View reviewed changes

comaniac approved these changes Sep 16, 2020
View reviewed changes

tqchen approved these changes Sep 16, 2020
View reviewed changes

tqchen requested changes Sep 16, 2020
View reviewed changes

Comment threadpython/setup.py Outdated

leandronand others added 6 commits September 17, 2020 11:25

tvmc: adjust TODOs
143f7e0

tvmc: fix linting errors
5ec6551

Address code-review comments
1e6ef0c

Adjust pytest fixture to not break when there is no tensorflow
9db5680

Fix frontend tests, to cope with different frameworks in different im…
c217622
…ages

leandronand others added 3 commits September 17, 2020 11:25

Apply suggestions from code review
9887550
Co-authored-by: Cody Yu <comaniac0422@gmail.com>

Fix lint and code-review issues
7fd5afa

Re-format with black.
dd2af42

tqchen requested changes Sep 17, 2020
View reviewed changes

Comment threadpython/setup.py Outdated

tvmc: Move dependencies to extras_requires
c55f8f2

leandron requested a review from tqchen September 18, 2020 08:48

tqchen approved these changes Sep 18, 2020
View reviewed changes

tqchen merged commit 292b640 into apache:masterSep 18, 2020

tqchen added status: accepted and removed status: review in progress labels Sep 18, 2020

ZihengJiang mentioned this pull request Sep 25, 2020
TVM v0.7 Release Note Candidate #6486
Closed

leandron deleted the tvmc_compile_feature branch October 1, 2020 13:55

leandron mentioned this pull request Mar 15, 2021
[TVMC] Fix to check whether a path passed to --target is strictly a file #7663
Merged

Uh oh!

Conversation

leandron commented Aug 19, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

leandron commented Aug 19, 2020

Uh oh!

tom-gall commented Aug 19, 2020

Uh oh!

comaniac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tqchen commented Aug 20, 2020

Uh oh!

junrushao commented Aug 20, 2020

Uh oh!

comaniac commented Aug 20, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

leandron commented Sep 9, 2020

Uh oh!

leandron commented Sep 10, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

comaniac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

comaniac commented Sep 10, 2020

Uh oh!

leandron commented Sep 11, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

leandron commented Sep 16, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

comaniac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tqchen commented Sep 18, 2020

Uh oh!

OneIsAll-w commented Mar 15, 2021

Uh oh!

leandron commented Mar 15, 2021

Uh oh!

OneIsAll-w commented Mar 15, 2021

Uh oh!

leandron commented Mar 15, 2021

Uh oh!

OneIsAll-w commented Mar 15, 2021

leandron commented Aug 19, 2020•
edited
Loading

leandron commented Sep 10, 2020•
edited
Loading

leandron commented Sep 11, 2020•
edited
Loading

leandron commented Sep 16, 2020•
edited
Loading