opdiff: cross-backend PyTorch operator testing + results dashboard

[0xShug0]

I’ve been experimenting with opdiff, a framework for systematically testing PyTorch operators and modules across multiple backends (Torch, ONNX Runtime, CoreML, ExecuTorch, etc.). Tests are defined with YAML and can sweep hundreds of operators and module variants across different backend configs, then compare results against a chosen baseline for correctness and numerical parity.

Repo: https://github.com/0xShug0/opdiff/

While running large operator sweeps, I realized summary stats hide a lot of useful signal (export succeeds but runtime fails, backend-specific numerical drift, config-specific failures, etc.). To make the results easier to explore, I built a small dashboard:

https://opdiff.com/

You can search operators/modules, compare backend configs, and see export vs runtime failures instead of just aggregated numbers.

Some of these runs already surfaced real issues (a couple ExecuTorch bugs were filed). If you spot something interesting, feel free to dig in or report upstream.

[cccclai]

This seems super interesting, cc: @GregoryComer@mergennachin

[0xShug0]

@cccclai@GregoryComer@mergennachin Let me know what other ops, modules, models, or experiments you'd like to see. Also feel free to point out any results that look "off" (e.g., framework or experiment artifacts rather than real backend issues), and I'll use that feedback to improve the framework and the testing setup.