Add support for setting shape inference function on fused nodes - #7007
Conversation
|
Hi Ashwini Khade (@askhade), could you please help? |
…o shape_inf_fuse
40e781d to
f211653
Compare
| return *model_; | ||
| } | ||
| }; | ||
|
|
There was a problem hiding this comment.
Please use InferenceSessionWrapper
There was a problem hiding this comment.
Done, thanks for the pointer (didn't realize this existed). We need access to the mutable graph though (in order to call .resolve() which clears out the inferred shapes) so I added an additional GetMutableGraph method to InferenceSessionWrapper.
bdeedbd to
b02ec0c
Compare
b02ec0c to
623cdba
Compare
|
/azp run Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux Nuphar CI Pipeline,Linux OpenVINO CI Pipeline |
|
/azp run MacOS CI Pipeline,MacOS NoContribops CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline |
|
/azp run orttraining-amd-gpu-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule,orttraining-ortmodule-distributed |
|
Azure Pipelines successfully started running 5 pipeline(s). |
|
Azure Pipelines successfully started running 7 pipeline(s). |
|
Azure Pipelines successfully started running 5 pipeline(s). |
|
/azp run Windows WebAssembly CI Pipeline |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
Scott McKay (@skottmckay) The CI for minimal ORT seems to fail (despite the regular CPU pipeline succeeding). Should I need to surround this by an |
Yes please. Put it inside We have an extremely limited dependency on ONNX in the minimal build in order to minimize the binary size. The type/shape info is saved in the ORT format model used by the minimal build and not recalculated at runtime. |
|
Fixed, thanks for the info! Could you please re-run CI? |
|
/azp run MacOS CI Pipeline,MacOS NoContribops CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Windows WebAssembly CI Pipeline |
|
/azp run Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux Nuphar CI Pipeline,Linux OpenVINO CI Pipeline |
|
Azure Pipelines successfully started running 6 pipeline(s). |
|
/azp run orttraining-amd-gpu-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule,orttraining-ortmodule-distributed |
|
Azure Pipelines successfully started running 5 pipeline(s). |
|
Azure Pipelines successfully started running 7 pipeline(s). |
| op_schema->SetDoc(meta_def->doc_string); | ||
| op_schema->SinceVersion(meta_def->since_version); | ||
|
|
||
| if (meta_def->type_and_shape_inference_function) { |
There was a problem hiding this comment.
Probably need an ifdef around all usages (both here and in unit tests).
There was a problem hiding this comment.
Seems to pass the minimal CI without it; I guess function.cc and the unit test are never included in the minimal build?
82a1424 to
87cfbe4
Compare
|
/azp run orttraining-amd-gpu-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule,orttraining-ortmodule-distributed |
|
/azp run MacOS CI Pipeline,MacOS NoContribops CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Windows WebAssembly CI Pipeline |
|
/azp run Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux Nuphar CI Pipeline,Linux OpenVINO CI Pipeline |
|
Azure Pipelines successfully started running 5 pipeline(s). |
|
Azure Pipelines successfully started running 6 pipeline(s). |
|
Azure Pipelines successfully started running 7 pipeline(s). |
Description: See #7006.
The ability to manually set a shape inference function for function-based nodes is useful for execution providers who perform node-fusion during graph partitioning. While normally shape info is computed before partitioning and hence is preserved by node fusion, in the case of graph transforms that operate after partitioning (level 3 and above, e.g. NHWC graph transformations) shape info can be reset or new nodeArgs created. In such cases, rather than manually setting shape info in every graph transform that deals with fused nodes, it can be cleaner to just define the shape inference function upfront.