How to add a new backend?

[JingliangGao]

🚀 The feature, motivation and pitch

Hi, I've already seen that some backends already supported from here. Is there a convenient way to add a new backend, like CANN or OpenCL, into executorch? BTW, if executorch will support OpenCL backend in the future?

cc @cccclai

[psiddh]

Requesting @GregoryComer to chime in

[GregoryComer]

Sure. We don't currently have plans to support OpenCL - @SS-JIA did a detailed analysis a while back and we decided to pursue Vulkan for targeting mobile GPUs. Do you have any specific use cases or hardware targets in mind that would benefit from OpenCL? I don't think we've looked at CANN before.

Regarding new backends, we do accept contributions. If this is something you're interested in, I'd recommend starting by posting an RFC under https://github.com/pytorch/executorch/discussions with a high-level overview of the backend goals, hardware and software dependencies, and design.

We have some documentation around backend design under https://docs.pytorch.org/executorch/stable/backend-delegate-advanced.html, though it's sparse. In general, each backend needs to define a set of lowering-time processing steps, create a serialization blob format, and implement the runtime support to run the delegated block. The XNNPACK backend might be a good place to start looking at these pieces: https://github.com/pytorch/executorch/tree/main/backends/xnnpack. I'm happy to help answer any more questions you have.

[cccclai]

On a side note, adding a new backend is fairly easy. In-tree will be stricter because we need to make sure the backends under executorch repro meets the quality. But if out of tree also works for you, you can fork the repro and add one

[leograssroot]

Thanks for the guidance here. I opened a related RFC discussion for an Ascend CANN backend/delegate:

#20732