[attempt 2] Compute contiguity symbolically to avoid dde, and introduce c++ sym_is_contiguous by laithsakka · Pull Request #157472 · pytorch/pytorch

laithsakka · 2025-07-02T17:04:58Z

Summary:
When we compute contiguity for a tensor with dynamic shapes we first:

Try to compute it without guarding.
If all shapes hinted, compute it with potentially adding guards.
if any input is not hinted, compute it symbolically.

sym_is_contiguous return a SymBool that is then either evaluated or guard_or_false can be called
on it to avoid data dependent errors.

ex:
bool is_contiguous = input.sym_is_contiguous().guard_or_false(FILE, LINE);
is_contiguous_or_false is a helper function that does that.

In this PR I only handle default contiguity, will follow up with changes for other formats like channel_last .
We use this patter in this PR for several locations to avoid DDEs.

Test Plan:
contbuild & OSS CI,

Rollback Plan:

Reviewed By: malfet

Differential Revision: D77639021

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @jerryzh168

pytorch-bot · 2025-07-02T17:05:02Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/157472

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 6dccadf with merge base d5d14ee ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-07-02T17:05:19Z

This pull request was exported from Phabricator. Differential Revision: D77639021

facebook-github-bot · 2025-07-02T17:12:16Z

This pull request was exported from Phabricator. Differential Revision: D77639021

…ce c++ sym_is_contiguous (pytorch#157472) Summary: Pull Request resolved: pytorch#157472 When we compute contiguity for a tensor with dynamic shapes we first: 1) Try to compute it without guarding. 2) If all shapes hinted, compute it with potentially adding guards. 3) if any input is not hinted, compute it symbolically. sym_is_contiguous return a SymBool that is then either evaluated or guard_or_false can be called on it to avoid data dependent errors. ex: bool is_contiguous = input.sym_is_contiguous().guard_or_false(__FILE__, __LINE__); is_contiguous_or_false is a helper function that does that. In this PR I only handle default contiguity, will follow up with changes for other formats like channel_last . We use this patter in this PR for several locations to avoid DDEs. Test Plan: contbuild & OSS CI, Rollback Plan: Reviewed By: huydhn, malfet Differential Revision: D77639021

facebook-github-bot · 2025-07-02T17:31:43Z

This pull request was exported from Phabricator. Differential Revision: D77639021

jeanschmidt · 2025-07-02T19:03:01Z

Seems that the imported diff have multiple red signals internally...

facebook-github-bot · 2025-07-02T23:05:04Z

@pytorchbot merge

(Initiating merge automatically since Phabricator Diff has merged)

pytorchmergebot · 2025-07-02T23:06:54Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

… add aten.sym_is_contiguous. (#159197) This might cause some new DDEs on call sites that do not use is_contiguous_or_false() or sym_is_contiguous() but want to find those call sites to handle this properly by calling is_contiguous_or_false() and not is_contiguous() explitly when appropriate. I had to fix one issue after removing the implicit size oblivious reasoning. here is context we defined in this #157472 sym_is_contiguous to be the function computing contiguity for dynamic shapes in c++. It returns a symbolic expression that represents contiguity and guaranteed not to throw a DDE. when people call is_contiguous we do sym_is_contiguous().guard_bool() when people call is_contiguous_or_false we do sym_is_contiguous().guard_or_false() one issue not handled well was this path ``` c10::SymBool TensorImpl::sym_is_contiguous_custom( at::MemoryFormat memory_format) const { if (C10_UNLIKELY(matches_python_custom(SizesStridesPolicy::CustomStrides))) { return pyobj_slot_.load_pyobj_interpreter()->is_contiguous( this, memory_format); } return sym_is_contiguous_default(memory_format); } ``` namely if we call sym_is_contiguous_custom but we have matches_python_custom(SizesStridesPolicy::CustomStrides) return true , then we used to call is_contiguous(this, memory_format); This used to go through the load_pyobj_interpreter and end up calling the python is_contiguous call which used implicit size oblivious reasoning. once we removed that implicit size oblivious reasoning, the right thing we want is to call return pyobj_slot_.load_pyobj_interpreter()->sym_is_contiguous(this, memory_format); otherwise we would get DDE even if the caller is doing sym_is_contiguous. so I had to define it for pyinterpreter, and then I had to override it for nested tensors. Pull Request resolved: #159197 Approved by: https://github.com/ezyang

… add aten.sym_is_contiguous. (#159197) (#159197) Summary: This might cause some new DDEs on call sites that do not use is_contiguous_or_false() or sym_is_contiguous() but want to find those call sites to handle this properly by calling is_contiguous_or_false() and not is_contiguous() explitly when appropriate. I had to fix one issue after removing the implicit size oblivious reasoning. here is context we defined in this #157472 sym_is_contiguous to be the function computing contiguity for dynamic shapes in c++. It returns a symbolic expression that represents contiguity and guaranteed not to throw a DDE. when people call is_contiguous we do sym_is_contiguous().guard_bool() when people call is_contiguous_or_false we do sym_is_contiguous().guard_or_false() one issue not handled well was this path ``` c10::SymBool TensorImpl::sym_is_contiguous_custom( at::MemoryFormat memory_format) const { if (C10_UNLIKELY(matches_python_custom(SizesStridesPolicy::CustomStrides))) { return pyobj_slot_.load_pyobj_interpreter()->is_contiguous( this, memory_format); } return sym_is_contiguous_default(memory_format); } ``` namely if we call sym_is_contiguous_custom but we have matches_python_custom(SizesStridesPolicy::CustomStrides) return true , then we used to call is_contiguous(this, memory_format); This used to go through the load_pyobj_interpreter and end up calling the python is_contiguous call which used implicit size oblivious reasoning. once we removed that implicit size oblivious reasoning, the right thing we want is to call return pyobj_slot_.load_pyobj_interpreter()->sym_is_contiguous(this, memory_format); otherwise we would get DDE even if the caller is doing sym_is_contiguous. so I had to define it for pyinterpreter, and then I had to override it for nested tensors. Pull Request resolved: #159197 Approved by: https://github.com/ezyang Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/e444cd24d48b3a46f067974f2cc157f5ed27709f Rollback Plan: Differential Revision: D80435179

… add aten.sym_is_contiguous. (pytorch#159197) This might cause some new DDEs on call sites that do not use is_contiguous_or_false() or sym_is_contiguous() but want to find those call sites to handle this properly by calling is_contiguous_or_false() and not is_contiguous() explitly when appropriate. I had to fix one issue after removing the implicit size oblivious reasoning. here is context we defined in this pytorch#157472 sym_is_contiguous to be the function computing contiguity for dynamic shapes in c++. It returns a symbolic expression that represents contiguity and guaranteed not to throw a DDE. when people call is_contiguous we do sym_is_contiguous().guard_bool() when people call is_contiguous_or_false we do sym_is_contiguous().guard_or_false() one issue not handled well was this path ``` c10::SymBool TensorImpl::sym_is_contiguous_custom( at::MemoryFormat memory_format) const { if (C10_UNLIKELY(matches_python_custom(SizesStridesPolicy::CustomStrides))) { return pyobj_slot_.load_pyobj_interpreter()->is_contiguous( this, memory_format); } return sym_is_contiguous_default(memory_format); } ``` namely if we call sym_is_contiguous_custom but we have matches_python_custom(SizesStridesPolicy::CustomStrides) return true , then we used to call is_contiguous(this, memory_format); This used to go through the load_pyobj_interpreter and end up calling the python is_contiguous call which used implicit size oblivious reasoning. once we removed that implicit size oblivious reasoning, the right thing we want is to call return pyobj_slot_.load_pyobj_interpreter()->sym_is_contiguous(this, memory_format); otherwise we would get DDE even if the caller is doing sym_is_contiguous. so I had to define it for pyinterpreter, and then I had to override it for nested tensors. Pull Request resolved: pytorch#159197 Approved by: https://github.com/ezyang

… add aten.sym_is_contiguous. (#159197) (#160869) Summary: This might cause some new DDEs on call sites that do not use is_contiguous_or_false() or sym_is_contiguous() but want to find those call sites to handle this properly by calling is_contiguous_or_false() and not is_contiguous() explitly when appropriate. I had to fix one issue after removing the implicit size oblivious reasoning. here is context we defined in this #157472 sym_is_contiguous to be the function computing contiguity for dynamic shapes in c++. It returns a symbolic expression that represents contiguity and guaranteed not to throw a DDE. when people call is_contiguous we do sym_is_contiguous().guard_bool() when people call is_contiguous_or_false we do sym_is_contiguous().guard_or_false() one issue not handled well was this path ``` c10::SymBool TensorImpl::sym_is_contiguous_custom( at::MemoryFormat memory_format) const { if (C10_UNLIKELY(matches_python_custom(SizesStridesPolicy::CustomStrides))) { return pyobj_slot_.load_pyobj_interpreter()->is_contiguous( this, memory_format); } return sym_is_contiguous_default(memory_format); } ``` namely if we call sym_is_contiguous_custom but we have matches_python_custom(SizesStridesPolicy::CustomStrides) return true , then we used to call is_contiguous(this, memory_format); This used to go through the load_pyobj_interpreter and end up calling the python is_contiguous call which used implicit size oblivious reasoning. once we removed that implicit size oblivious reasoning, the right thing we want is to call return pyobj_slot_.load_pyobj_interpreter()->sym_is_contiguous(this, memory_format); otherwise we would get DDE even if the caller is doing sym_is_contiguous. so I had to define it for pyinterpreter, and then I had to override it for nested tensors. Approved by: https://github.com/ezyang Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/e444cd24d48b3a46f067974f2cc157f5ed27709f Rollback Plan: Reviewed By: ezyang Differential Revision: D80435179

… add aten.sym_is_contiguous. [attempt2] (#160869) [relanding again after fixing internal build] Summary: This might cause some new DDEs on call sites that do not use is_contiguous_or_false() or sym_is_contiguous() but want to find those call sites to handle this properly by calling is_contiguous_or_false() and not is_contiguous() explitly when appropriate. I had to fix one issue after removing the implicit size oblivious reasoning. here is context we defined in this #157472 sym_is_contiguous to be the function computing contiguity for dynamic shapes in c++. It returns a symbolic expression that represents contiguity and guaranteed not to throw a DDE. when people call is_contiguous we do sym_is_contiguous().guard_bool() when people call is_contiguous_or_false we do sym_is_contiguous().guard_or_false() one issue not handled well was this path ``` c10::SymBool TensorImpl::sym_is_contiguous_custom( at::MemoryFormat memory_format) const { if (C10_UNLIKELY(matches_python_custom(SizesStridesPolicy::CustomStrides))) { return pyobj_slot_.load_pyobj_interpreter()->is_contiguous( this, memory_format); } return sym_is_contiguous_default(memory_format); } ``` namely if we call sym_is_contiguous_custom but we have matches_python_custom(SizesStridesPolicy::CustomStrides) return true , then we used to call is_contiguous(this, memory_format); This used to go through the load_pyobj_interpreter and end up calling the python is_contiguous call which used implicit size oblivious reasoning. once we removed that implicit size oblivious reasoning, the right thing we want is to call return pyobj_slot_.load_pyobj_interpreter()->sym_is_contiguous(this, memory_format); otherwise we would get DDE even if the caller is doing sym_is_contiguous. so I had to define it for pyinterpreter, and then I had to override it for nested tensors. Approved by: https://github.com/ezyang Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/e444cd24d48b3a46f067974f2cc157f5ed27709f Rollback Plan: Differential Revision: D80435179 Pull Request resolved: #160869 Approved by: https://github.com/ezyang

… add aten.sym_is_contiguous. (pytorch#159197) This might cause some new DDEs on call sites that do not use is_contiguous_or_false() or sym_is_contiguous() but want to find those call sites to handle this properly by calling is_contiguous_or_false() and not is_contiguous() explitly when appropriate. I had to fix one issue after removing the implicit size oblivious reasoning. here is context we defined in this pytorch#157472 sym_is_contiguous to be the function computing contiguity for dynamic shapes in c++. It returns a symbolic expression that represents contiguity and guaranteed not to throw a DDE. when people call is_contiguous we do sym_is_contiguous().guard_bool() when people call is_contiguous_or_false we do sym_is_contiguous().guard_or_false() one issue not handled well was this path ``` c10::SymBool TensorImpl::sym_is_contiguous_custom( at::MemoryFormat memory_format) const { if (C10_UNLIKELY(matches_python_custom(SizesStridesPolicy::CustomStrides))) { return pyobj_slot_.load_pyobj_interpreter()->is_contiguous( this, memory_format); } return sym_is_contiguous_default(memory_format); } ``` namely if we call sym_is_contiguous_custom but we have matches_python_custom(SizesStridesPolicy::CustomStrides) return true , then we used to call is_contiguous(this, memory_format); This used to go through the load_pyobj_interpreter and end up calling the python is_contiguous call which used implicit size oblivious reasoning. once we removed that implicit size oblivious reasoning, the right thing we want is to call return pyobj_slot_.load_pyobj_interpreter()->sym_is_contiguous(this, memory_format); otherwise we would get DDE even if the caller is doing sym_is_contiguous. so I had to define it for pyinterpreter, and then I had to override it for nested tensors. Pull Request resolved: pytorch#159197 Approved by: https://github.com/ezyang

… add aten.sym_is_contiguous. [attempt2] (pytorch#160869) [relanding again after fixing internal build] Summary: This might cause some new DDEs on call sites that do not use is_contiguous_or_false() or sym_is_contiguous() but want to find those call sites to handle this properly by calling is_contiguous_or_false() and not is_contiguous() explitly when appropriate. I had to fix one issue after removing the implicit size oblivious reasoning. here is context we defined in this pytorch#157472 sym_is_contiguous to be the function computing contiguity for dynamic shapes in c++. It returns a symbolic expression that represents contiguity and guaranteed not to throw a DDE. when people call is_contiguous we do sym_is_contiguous().guard_bool() when people call is_contiguous_or_false we do sym_is_contiguous().guard_or_false() one issue not handled well was this path ``` c10::SymBool TensorImpl::sym_is_contiguous_custom( at::MemoryFormat memory_format) const { if (C10_UNLIKELY(matches_python_custom(SizesStridesPolicy::CustomStrides))) { return pyobj_slot_.load_pyobj_interpreter()->is_contiguous( this, memory_format); } return sym_is_contiguous_default(memory_format); } ``` namely if we call sym_is_contiguous_custom but we have matches_python_custom(SizesStridesPolicy::CustomStrides) return true , then we used to call is_contiguous(this, memory_format); This used to go through the load_pyobj_interpreter and end up calling the python is_contiguous call which used implicit size oblivious reasoning. once we removed that implicit size oblivious reasoning, the right thing we want is to call return pyobj_slot_.load_pyobj_interpreter()->sym_is_contiguous(this, memory_format); otherwise we would get DDE even if the caller is doing sym_is_contiguous. so I had to define it for pyinterpreter, and then I had to override it for nested tensors. Approved by: https://github.com/ezyang Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/e444cd24d48b3a46f067974f2cc157f5ed27709f Rollback Plan: Differential Revision: D80435179 Pull Request resolved: pytorch#160869 Approved by: https://github.com/ezyang

…d concretely compute_contiguous asks three questions in order (see #157472): can contiguity be proved without guarding, is every shape hinted so it can be computed concretely, or must a symbolic predicate be handed back. But it ran the first and third eagerly by calling _compute_contiguous_sym, which does both, and only then checked whether the shapes were hinted. For a hinted tensor whose contiguity is not provable guard-free, the predicate was built out of per-dimension SymInt operations and then thrown away, because the concrete computation below it is the answer. Split the two phases apart in Contiguity.h - _contiguous_or_false_sym proves it without guarding, _contiguous_expr_sym builds the predicate - and ask the questions in cost order. _compute_contiguous_sym stays as a wrapper that does both, so its behaviour is unchanged for any other caller. This matters because each SymInt operation in those loops crosses into Python, builds a sympy expression and runs it through ShapeEnv.replace, at a few microseconds a time. With counters in SymbolicShapeMeta, one model's cold compile with dynamic=True does 7126 contiguity derivations, and 3354 of them - 47% - take exactly the path where the predicate is discarded. Time spent deriving contiguity, three runs each: before 753ms 1059ms 747ms (mean 853ms) after 554ms 548ms 545ms (mean 549ms) -304ms, -36%, and with much less variance. That is below the ~1s run-to-run noise of end-to-end compile time on this machine, so this is measured on the derivation itself rather than on the wall clock; the counts of which path each derivation takes are identical before and after (3772 concrete, 3354 hinted, 0 symbolic), which is what says the reordering did not move any answers. Test plan: ``` python test/test_dynamic_shapes.py -k TestSymbolicContiguity python test/test_dynamic_shapes.py python test/test_proxy_tensor.py ``` TestSymbolicContiguity is new: 14 layouts (contiguous, permuted, channels-last, size-1 dims, zero-size, mixed symbolic/concrete, 1d through 5d) crossed with checks on the contiguity answer, the numel expression, and non-overlapping-and- dense, plus one case asserting the guard-free path specializes nothing. It pins the answers rather than the path, so it passes against both the old and the new ordering - which is how it was used to develop this change. The other two suites produce failure sets identical to a run with the change reverted (this host has pre-existing aarch64 CPU codegen failures). This PR was authored with the assistance of an AI coding agent. ghstack-source-id: 3a73082 Pull-Request: #193220

laithsakka requested review from a team, albanD and soulitzer as code owners July 2, 2025 17:04

pytorch-bot Bot added ciflow/inductor module: cpu CPU specific problem (e.g., perf, algorithm) release notes: releng release notes category labels Jul 2, 2025

facebook-github-bot added the fb-exported label Jul 2, 2025

laithsakka force-pushed the export-D77639021 branch from d59962a to d3cd6e3 Compare July 2, 2025 17:12

laithsakka requested a review from ezyang July 2, 2025 17:19

laithsakka force-pushed the export-D77639021 branch from d3cd6e3 to 6dccadf Compare July 2, 2025 17:31

aorenste approved these changes Jul 2, 2025

View reviewed changes

pytorch-bot Bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jul 2, 2025

albanD removed their request for review July 2, 2025 19:38

pytorchmergebot added the merging label Jul 2, 2025

pytorchmergebot closed this in 7cfd054 Jul 2, 2025

pytorchmergebot added Merged and removed merging labels Jul 2, 2025

soulitzer removed their request for review July 3, 2025 13:46

laithsakka mentioned this pull request Jul 31, 2025

Remove guard_size_oblivious from default contiguity python check, and add aten.sym_is_contiguous. #159197

Closed

laithsakka mentioned this pull request Aug 18, 2025

Remove guard_size_oblivious from default contiguity python check, and add aten.sym_is_contiguous. [attempt2] #160869

Closed

anijain2305 mentioned this pull request Aug 12, 2026

[c10] Don't build the contiguity predicate when the answer is computed concretely #193220

Open

Navigation Menu

[attempt 2] Compute contiguity symbolically to avoid dde, and introduce c++ sym_is_contiguous - #157472

[attempt 2] Compute contiguity symbolically to avoid dde, and introduce c++ sym_is_contiguous#157472
laithsakka wants to merge 1 commit into
pytorch:mainfrom
laithsakka:export-D77639021

laithsakka commented Jul 2, 2025 •

edited by pytorch-bot Bot

Loading

Uh oh!

pytorch-bot Bot commented Jul 2, 2025 •

edited

Loading

Uh oh!

facebook-github-bot commented Jul 2, 2025

Uh oh!

facebook-github-bot commented Jul 2, 2025

Uh oh!

facebook-github-bot commented Jul 2, 2025

Uh oh!

jeanschmidt commented Jul 2, 2025

Uh oh!

facebook-github-bot commented Jul 2, 2025

Uh oh!

pytorchmergebot commented Jul 2, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

laithsakka commented Jul 2, 2025 • edited by pytorch-bot Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/157472

✅ No Failures

Uh oh!

facebook-github-bot commented Jul 2, 2025

Uh oh!

facebook-github-bot commented Jul 2, 2025

Uh oh!

facebook-github-bot commented Jul 2, 2025

Uh oh!

jeanschmidt commented Jul 2, 2025

Uh oh!

facebook-github-bot commented Jul 2, 2025

Uh oh!

pytorchmergebot commented Jul 2, 2025

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

laithsakka commented Jul 2, 2025 •

edited by pytorch-bot Bot

Loading

pytorch-bot Bot commented Jul 2, 2025 •

edited

Loading