Skip to content

Do not sort operands as part of simplification [blocks: #3486] - #1997

Merged
tautschnig merged 3 commits into
diffblue:developfrom
tautschnig:no-sort-operands
Jan 28, 2021
Merged

Do not sort operands as part of simplification [blocks: #3486]#1997
tautschnig merged 3 commits into
diffblue:developfrom
tautschnig:no-sort-operands

Conversation

@tautschnig

@tautschnigtautschnig commented Apr 3, 2018

Copy link
Copy Markdown
Collaborator

Do-not-merge as above: this needs proper profiling to confirm the effect with data.

@kroening

Copy link
Copy Markdown
Collaborator

Does it make a difference whether or not the number of operands is large?
Also, is this perhaps about wrecking sharing? Or is the cost of the comparison too high for deep expressions?

@tautschnig

Copy link
Copy Markdown
CollaboratorAuthor

Does it make a difference whether or not the number of operands is large?
Also, is this perhaps about wrecking sharing? Or is the cost of the comparison too high for deep expressions?

It's just the plain number of compare operations. 163M sort_operands calls, contributing to 1B irept::operator< calls, resulting in 4.5B irept::compare calls. I think close to zero of which are necessary. (Data for ReachSafety-BitVectors, the smallest of SV-COMP's categories, running for at most 600s per benchmark.)

@tautschnigtautschnig added Needs data This PR claims improvements that require further data to substantiate the claims. and removed do not merge labels Mar 29, 2019

@allredjallredj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✔️
Passed Diffblue compatibility checks (cbmc commit: 51fcd8c).
Build URL: https://travis-ci.com/diffblue/test-gen/builds/106401199

@tautschnigtautschnig changed the title [SV-COMP'18 8/19] Needs more profiling: Do not sort operands as part of simplificationDo not sort operands as part of simplification [blocks: #3486]Mar 30, 2019
@tautschnig

Copy link
Copy Markdown
CollaboratorAuthor

Combined with #1998 this increases the number of symex_step calls per second (on ReachSafety-ECA) from 2752.28 to 3749.87. Profiling them independently confuses gprof, because sort_and_join is invoked from different contexts.

@tautschnigtautschnig removed the Needs data This PR claims improvements that require further data to substantiate the claims. label Mar 30, 2019
return saj_table[i];
}

bool sort_and_join(exprt &expr)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

! This needs a new name

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it no longer necessarily sorts! A person reading sort_and_join(x, false); might reasonably suppose that regardless of what that false means, x is certainly now sorted.

@tautschnigtautschnig self-assigned this Apr 1, 2019

@smowtonsmowton left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This roughly halved a long-running case that simplifies a very large expression

@smowton

Copy link
Copy Markdown
Contributor

(but do still please rename sort_and_join)

@codecov

codecovBot commented Nov 7, 2020

Copy link
Copy Markdown

Codecov Report

Merging #1997 (375f208) into develop (f2fafac) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@ Coverage Diff @@## develop #1997 +/- ##
========================================
Coverage 69.64% 69.65% ========================================
Files 1248 1248 Lines 100911 100921 +10 ========================================
+ Hits 70284 70292 +8 - Misses 30627 30629 +2 
FlagCoverage Δ
cproversmt243.37% <100.00%> (+0.01%)⬆️
regression66.62% <100.00%> (+<0.01%)⬆️
unit32.18% <55.55%> (-0.01%)⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted FilesCoverage Δ
src/util/simplify_expr.cpp85.97% <100.00%> (ø)
src/util/simplify_expr_floatbv.cpp95.23% <100.00%> (+0.18%)⬆️
src/util/simplify_utils.cpp98.67% <100.00%> (+0.05%)⬆️
src/util/simplify_expr_array.cpp88.40% <0.00%> (-2.90%)⬇️
src/solvers/flattening/boolbv_quantifier.cpp96.05% <0.00%> (-1.32%)⬇️
src/util/simplify_expr_pointer.cpp88.07% <0.00%> (-0.39%)⬇️
src/goto-symex/symex_builtin_functions.cpp95.16% <0.00%> (+0.96%)⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f2fafac...375f208. Read the comment docs.

@martin-csmartin-cs left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable. Thanks @smowton for the performance comments.

Comment threadregression/goto-analyzer/constant_propagation_07/test.desc
Comment threadsrc/util/expr.h Outdated

@martin-csmartin-cs left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes but handling 'commutative but not associative' would be good.

\[main.NaN.15\] line \d+ NaN on - in mynan - \(float\)n: SUCCESS
\[main.NaN.16\] line \d+ NaN on \* in mynan \* \(float\)n: SUCCESS
\[main.NaN.17\] line \d+ NaN on / in mynan / \(float\)n: SUCCESS
\[main.NaN.6\] line \d+ NaN on \+ in myinf \+ -myinf: FAILURE

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually; this raises an interesting question. Floating-point addition and multiplication are commutative but not associative. If seems like normalising / sorting these would be worth while.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I am able to follow: why would sorting (which this PR largely disables) be particularly beneficial for those operations? We surely can selectively re-enable sorting, but I don't think I understand the rationale just yet.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

float a;
float b;
if (!isnan(a) && !isnan(b)) {
assert(a + b === b + a);
}

Normalising the order of commutative-but-not-associative operations means we can resolve this at simplification, rather than an expensive solver call. It is not a massively common case but it is very cheap to implement and IIRC is a huge speed up in this case. Note that this can be purely local as it is not associative.

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems we didn't actually simplify such expressions before as commutative-but-not-associative operations were exempt. I have now added 375f208 to take care of this, thank you very much for raising this!

Comment threadregression/goto-analyzer/constant_propagation_07/test.desc
String refinement may or may not be successful (resulting in "ERROR"
results rather than "FAILURE") when working on an unconstrained string
(which the null dereference yields), depending on the models returned by
the SAT solver.
Data for ReachSafety-BitVectors, the smallest of SV-COMP's categories, running
for at most 600s per benchmark: 163M sort_operands calls, contributing to 1B
irept::operator< calls, resulting in 4.5B irept::compare calls. None of these
calls are actually necessary.
As part of this work, also clean up naming of "sort_and_join" functions
(as most of them don't actually sort or join, but instead just prepare
for doing so).
Testing floating-point equality is expensive, and thus warrants handling
special cases (even if they might be rare) in the simplifier. This
(simplistic) test would take 5 seconds with CaDiCaL as SAT back-end when
not performing simplification.
@tautschnig
tautschnig merged commit cf40d22 into diffblue:developJan 28, 2021
@tautschnig
tautschnig deleted the no-sort-operands branch January 28, 2021 09:42
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@tautschnig@kroening@smowton@martin-cs@allredj@peterschrammel