[RFC] Pipeline Executor

[huajsj]

Split relay graph into Subgraph then doing Subgraph Pipeline :RFC

In this RFC, we present a new framework for subgraph pipelining that first split relay graph into multiple subgraph/group operator s then doing subgraph/group operators scheduling

Discussion: https://discuss.tvm.apache.org/t/rfc-compute-graph-pipeline-with-new-subgraph-executor/9839

[comaniac]

Thanks for the RFC. My general opinions:

• The whole picture of this RFC is not clear. If it only targets to a runtime executor, it is not a complete end-to-end solution. At high-level, you should present the ultimate goal, including graph partitioning and execution. Based on that, you could further break them down to several milestones so that we can track the progress while having a clear final picture.
• Please do proof-reading in advance. There are too many spelling and grammar errors to read the proposal smoothly.

cc @tmoreau89@areusch@jtuyls

[huajsj]

@comaniac , @jtuyls , all review comments addressed, please review again.

[comaniac]

Overall LGTM. It would be great if other folks can also take a look; otherwise I plan to merge this RFC after 3 days.

cc @areusch@jroesch@zhiics@junrushao1994@jtuyls

[comaniac]

btw, please change the RFC file name to align the PR number (0014).

[huajsj]

btw, please change the RFC file name to align the PR number (0014).

@comaniac, thanks for the follow up, the file name change already done.

[comaniac]

Due to no objection, this RFC is now merged. Thanks @huajsj

[HungYangChang]

Hello @huajsj I am super interested in your work!
As I have asked in discussion form, I am using Hikey 970 with 4 big and 4 little CPUs to do inference now. One of my interest is to split my BERT model into the different layers (sub-graph) and then assign to different cores.

I will definitely try to replicate your work in these two days and see what can I get. Thanks for your great work!