Feature/issue 125 cholesky derivative by rtrangucci · Pull Request #135 · stan-dev/math

rtrangucci · 2015-07-30T22:58:04Z

Summary:

Introduces var specialization of cholesky_decompose

Intended Effect:

Wall-time boost of about about 30% to 50% over currently implemented naive autodiff-through-Eigen version, memory use O(D^2) instead of O(D^3).

How to Verify:

./runTests.py test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp

runs two gradient tests comparing results to finite diff.

Side Effects:

None.

Documentation:

Included in code.

Reviewer Suggestions:

Anyone

memory usage for large matrices

jgabry · 2015-07-30T23:10:17Z

drops the mic, quits

On Thursday, July 30, 2015, Rob Trangucci notifications@github.com wrote:

Summary:
Introduces var specialization of cholesky_decompose
Intended Effect:
Wall-time boost of about about 30% to 50% over currently implemented naive
autodiff-through-Eigen version, memory use O(D^2) instead of O(D^3).
How to Verify:
./runTests.py test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp
runs two gradient tests comparing results to finite diff.
Side Effects:
None.
Documentation:
Included in code.
Reviewer Suggestions:
Anyone
You can view, comment on, or merge this pull request online at:
#135
Commit Summary
b f/i-125 first cut at cholesky derivative structure
b f/i-125 analytic cholesky derivative works
b f/i-125 wall-clock about as fast as Eigen, with 1/10th
b f/i-125 sparse Eigen version, much slower than scalars
b f/i-125 fastest implementation currently
b f/i-125 cholesky-derivative nearly complete for _test.hpp
b f/i-125 indexing fixed
b f/i-125 cleaned up file
b f/i-125 separate gradient rev tests from higher-order tests
b f/i-125 documentation and speedup, tests
b f/i-125 more doc
Merge branch 'develop' into feature/issue-125-cholesky-derivative
File Changes
A stan/math/rev/mat/fun/cholesky_decompose.hpp
https://github.com/stan-dev/math/pull/135/files#diff-0 (170)
M test/unit/math/mix/mat/fun/cholesky_decompose_test.cpp
https://github.com/stan-dev/math/pull/135/files#diff-1 (45)
M test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp
https://github.com/stan-dev/math/pull/135/files#diff-2 (133)
Patch Links:
https://github.com/stan-dev/math/pull/135.patch
https://github.com/stan-dev/math/pull/135.diff
—
Reply to this email directly or view it on GitHub
#135.

rtrangucci · 2015-07-31T00:11:13Z

Ha, just wait, it'll fail unit tests and/or cpplint soon.

Rob

On Jul 30, 2015, at 7:10 PM, Jonah Gabry notifications@github.com wrote:
drops the mic, quits
On Thursday, July 30, 2015, Rob Trangucci notifications@github.com wrote:
Summary:
Introduces var specialization of cholesky_decompose
Intended Effect:
Wall-time boost of about about 30% to 50% over currently implemented naive
autodiff-through-Eigen version, memory use O(D^2) instead of O(D^3).
How to Verify:
./runTests.py test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp
runs two gradient tests comparing results to finite diff.
Side Effects:
None.
Documentation:
Included in code.
Reviewer Suggestions:
Anyone
You can view, comment on, or merge this pull request online at:
#135
Commit Summary
b f/i-125 first cut at cholesky derivative structure
b f/i-125 analytic cholesky derivative works
b f/i-125 wall-clock about as fast as Eigen, with 1/10th
b f/i-125 sparse Eigen version, much slower than scalars
b f/i-125 fastest implementation currently
b f/i-125 cholesky-derivative nearly complete for _test.hpp
b f/i-125 indexing fixed
b f/i-125 cleaned up file
b f/i-125 separate gradient rev tests from higher-order tests
b f/i-125 documentation and speedup, tests
b f/i-125 more doc
Merge branch 'develop' into feature/issue-125-cholesky-derivative
File Changes
A stan/math/rev/mat/fun/cholesky_decompose.hpp
https://github.com/stan-dev/math/pull/135/files#diff-0 (170)
M test/unit/math/mix/mat/fun/cholesky_decompose_test.cpp
https://github.com/stan-dev/math/pull/135/files#diff-1 (45)
M test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp
https://github.com/stan-dev/math/pull/135/files#diff-2 (133)
Patch Links:
https://github.com/stan-dev/math/pull/135.patch
https://github.com/stan-dev/math/pull/135.diff
—
Reply to this email directly or view it on GitHub
#135.
—
Reply to this email directly or view it on GitHub.

bob-carpenter · 2015-07-31T03:24:35Z

Rob's not quitting Stan, he's just changing day jobs :-)

I do love those memory numbers.

I'd also say Ben or Marcus should be reviewing this one,
but I think they already weighed in on the linear algebra side.

Bob

On Jul 30, 2015, at 7:10 PM, Jonah Gabry notifications@github.com wrote:
drops the mic, quits
On Thursday, July 30, 2015, Rob Trangucci notifications@github.com wrote:
Summary:
Introduces var specialization of cholesky_decompose
Intended Effect:
Wall-time boost of about about 30% to 50% over currently implemented naive
autodiff-through-Eigen version, memory use O(D^2) instead of O(D^3).
How to Verify:
./runTests.py test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp
runs two gradient tests comparing results to finite diff.
Side Effects:
None.
Documentation:
Included in code.
Reviewer Suggestions:
Anyone
You can view, comment on, or merge this pull request online at:
#135
Commit Summary
b f/i-125 first cut at cholesky derivative structure
b f/i-125 analytic cholesky derivative works
b f/i-125 wall-clock about as fast as Eigen, with 1/10th
b f/i-125 sparse Eigen version, much slower than scalars
b f/i-125 fastest implementation currently
b f/i-125 cholesky-derivative nearly complete for _test.hpp
b f/i-125 indexing fixed
b f/i-125 cleaned up file
b f/i-125 separate gradient rev tests from higher-order tests
b f/i-125 documentation and speedup, tests
b f/i-125 more doc
Merge branch 'develop' into feature/issue-125-cholesky-derivative
File Changes
A stan/math/rev/mat/fun/cholesky_decompose.hpp
https://github.com/stan-dev/math/pull/135/files#diff-0 (170)
M test/unit/math/mix/mat/fun/cholesky_decompose_test.cpp
https://github.com/stan-dev/math/pull/135/files#diff-1 (45)
M test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp
https://github.com/stan-dev/math/pull/135/files#diff-2 (133)
Patch Links:
https://github.com/stan-dev/math/pull/135.patch
https://github.com/stan-dev/math/pull/135.diff
—
Reply to this email directly or view it on GitHub
#135.
—
Reply to this email directly or view it on GitHub.

No storage of double* L_ in ctor Eigen access using coeffRef instead of raw arrays Use info about row-major algo to eliminate an O(N^2) loop after O(N^3) loop

rtrangucci · 2015-08-02T22:09:34Z

@syclik ? The error on travis says

test/prob/generate_tests.cpp:9:10: fatal error: 'boost/algorithm/string.hpp'
file not found
#include <boost/algorithm/string.hpp>

syclik · 2015-08-02T23:42:19Z

I'll take a look tonight.

On Sunday, August 2, 2015, Rob Trangucci notifications@github.com wrote:

@syclik https://github.com/syclik ? The error on travis says
test/prob/generate_tests.cpp:9:10: fatal error: 'boost/algorithm/string.hpp'
file not found
#include <boost/algorithm/string.hpp>
—
Reply to this email directly or view it on GitHub
#135 (comment).

rtrangucci · 2015-08-02T23:46:54Z

Thanks Daniel

Rob

On Sun, Aug 2, 2015 at 7:42 PM, Daniel Lee notifications@github.com wrote:

I'll take a look tonight.
On Sunday, August 2, 2015, Rob Trangucci notifications@github.com wrote:
@syclik https://github.com/syclik ? The error on travis says
test/prob/generate_tests.cpp:9:10: fatal error:
'boost/algorithm/string.hpp'
file not found
#include <boost/algorithm/string.hpp>
—
Reply to this email directly or view it on GitHub
#135 (comment).
—
Reply to this email directly or view it on GitHub
#135 (comment).

rtrangucci · 2015-08-04T02:05:25Z

@bob-carpenter @syclik I think this is probably ready for code review whenever you're ready. Thanks guys.

bob-carpenter · 2015-08-04T04:52:36Z

Can you get Marcus or Ben to review it? My matrix algebra's
been described as "texbook" by those in the know, so I'd rather
have them review it than do something stupid.

On Aug 3, 2015, at 10:05 PM, Rob Trangucci notifications@github.com wrote:
@bob-carpenter @syclik I think this is probably ready for code review whenever you're ready. Thanks guys.
—
Reply to this email directly or view it on GitHub.

rtrangucci · 2015-08-04T11:38:31Z

@bgoodri Would you be able to take a look at this? @bob-carpenter I had @mbrubake look at my code, it sounded like he thought the code was up to snuff from a C++-optimization standpoint. That's actually why I wanted you to look at it too, because I thought you'd have a good perspective on memory allocation, etc.

mbrubake · 2015-08-04T15:21:47Z

I had separately given Rob my feedback and suggestions on the vari implementation so I don't really have much to add there.

One thing though which is missing is a gradient test which doesn't use the constrained transformation. The tricky thing about getting the (autodiff) gradients of something like Cholesky correct is how to properly handle the symmetry when propagating the derivatives. For instance, it's common for the gradient of the upper or lower triangular portion to be zero when it shouldn't be or to be off by a factor of 2. @rtrangucci can you add tests which don't use cov_matrix_constrain and instead look at the FD and AD gradients wrt the full matrix?

bob-carpenter · 2015-08-04T15:35:37Z

The variables should be private if you're being picky.

bgoodri · 2015-08-21T20:05:56Z

with -fsanitize=address -g -O0

test/unit/math/rev/mat/fun/cholesky_decompose_test --gtest_output="xml:test/unit/math/rev/mat/fun/cholesky_decompose_test.xml"
Running main() from gtest_main.cc
[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 3 tests from AgradRevMatrix
[ RUN ] AgradRevMatrix.mat_cholesky
=================================================================
==7918==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60400000da30 at pc 0x00000040aad7 bp 0x7fff37bc7450 sp 0x7fff37bc7448
WRITE of size 8 at 0x60400000da30 thread T0
#0 0x40aad6 in stan::math::cholesky_decompose(Eigen::Matrix<stan::math::var, -1, -1, 0, -1, -1> const&) stan/math/rev/mat/fun/cholesky_decompose.hpp:155
#1 0x40d42f in AgradRevMatrix_mat_cholesky_Test::TestBody() test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp:211
#2 0x4ca577 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) lib/gtest_1.7.0/src/gtest.cc:2078
#3 0x4c34f5 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) lib/gtest_1.7.0/src/gtest.cc:2114
#4 0x49ea59 in testing::Test::Run() lib/gtest_1.7.0/src/gtest.cc:2151
#5 0x49f64d in testing::TestInfo::Run() lib/gtest_1.7.0/src/gtest.cc:2326
#6 0x49ff51 in testing::TestCase::Run() lib/gtest_1.7.0/src/gtest.cc:2444
#7 0x4a9ed0 in testing::internal::UnitTestImpl::RunAllTests() lib/gtest_1.7.0/src/gtest.cc:4315
#8 0x4cc868 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) lib/gtest_1.7.0/src/gtest.cc:2078
#9 0x4c4c45 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) lib/gtest_1.7.0/src/gtest.cc:2114
#10 0x4a7d22 in testing::UnitTest::Run() lib/gtest_1.7.0/src/gtest.cc:3926
#11 0x4060fa in RUN_ALL_TESTS() lib/gtest_1.7.0/include/gtest/gtest.h:2288
#12 0x406056 in main lib/gtest_1.7.0/src/gtest_main.cc:37
#13 0x7f1e8141bb44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b44)
#14 0x405f58 (/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test+0x405f58)
0x60400000da30 is located 0 bytes to the right of 32-byte region [0x60400000da10,0x60400000da30)
allocated by thread T0 here:
#0 0x7f1e822e6cbe in posix_memalign (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x94cbe)
#1 0x410b5f in Eigen::internal::aligned_malloc(unsigned long) (/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test+0x410b5f)
#2 0x440dcd in void* Eigen::internal::conditional_aligned_malloc<true>(unsigned long) (/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test+0x440dcd)
#3 0x438714 in stan::math::var* Eigen::internal::conditional_aligned_new_auto<stan::math::var, true>(unsigned long) (/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test+0x438714)
#4 0x42c23a in Eigen::DenseStorage<stan::math::var, -1, -1, -1, 0>::resize(long, long, long) (/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test+0x42c23a)
#5 0x421d15 in Eigen::PlainObjectBase<Eigen::Matrix<stan::math::var, -1, -1, 0, -1, -1> >::resize(long, long) (/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test+0x421d15)
#6 0x425648 in void Eigen::PlainObjectBase<Eigen::Matrix<stan::math::var, -1, -1, 0, -1, -1> >::_init2<long, long>(long, long, Eigen::internal::enable_if<true, long>::type*) (/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test+0x425648)
#7 0x41b15c in Eigen::Matrix<stan::math::var, -1, -1, 0, -1, -1>::Matrix<long, long>(long const&, long const&) (/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test+0x41b15c)
#8 0x40a993 in stan::math::cholesky_decompose(Eigen::Matrix<stan::math::var, -1, -1, 0, -1, -1> const&) stan/math/rev/mat/fun/cholesky_decompose.hpp:148
#9 0x40d42f in AgradRevMatrix_mat_cholesky_Test::TestBody() test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp:211
#10 0x4ca577 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) lib/gtest_1.7.0/src/gtest.cc:2078
#11 0x4c34f5 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) lib/gtest_1.7.0/src/gtest.cc:2114
#12 0x49ea59 in testing::Test::Run() lib/gtest_1.7.0/src/gtest.cc:2151
#13 0x49f64d in testing::TestInfo::Run() lib/gtest_1.7.0/src/gtest.cc:2326
#14 0x49ff51 in testing::TestCase::Run() lib/gtest_1.7.0/src/gtest.cc:2444
#15 0x4a9ed0 in testing::internal::UnitTestImpl::RunAllTests() lib/gtest_1.7.0/src/gtest.cc:4315
#16 0x4cc868 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) lib/gtest_1.7.0/src/gtest.cc:2078
#17 0x4c4c45 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) lib/gtest_1.7.0/src/gtest.cc:2114
#18 0x4a7d22 in testing::UnitTest::Run() lib/gtest_1.7.0/src/gtest.cc:3926
#19 0x4060fa in RUN_ALL_TESTS() lib/gtest_1.7.0/include/gtest/gtest.h:2288
#20 0x406056 in main lib/gtest_1.7.0/src/gtest_main.cc:37
#21 0x7f1e8141bb44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b44)
SUMMARY: AddressSanitizer: heap-buffer-overflow stan/math/rev/mat/fun/cholesky_decompose.hpp:155 stan::math::cholesky_decompose(Eigen::Matrix<stan::math::var, -1, -1, 0, -1, -1> const&)
Shadow bytes around the buggy address:
0x0c087fff9af0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c087fff9b00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c087fff9b10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c087fff9b20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c087fff9b30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c087fff9b40: fa fa 00 00 00 00[fa]fa fa fa 00 00 00 00 fa fa
0x0c087fff9b50: fa fa 00 00 00 00 fa fa fa fa 00 00 00 00 fa fa
0x0c087fff9b60: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa
0x0c087fff9b70: fa fa 00 00 00 00 00 fa fa fa fd fd fd fd fd fd
0x0c087fff9b80: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fd
0x0c087fff9b90: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
==7918==ABORTING
test/unit/math/rev/mat/fun/cholesky_decompose_test --gtest_output="xml:test/unit/math/rev/mat/fun/cholesky_decompose_test.xml" failed

bgoodri · 2015-08-21T20:21:08Z

But the test actually survives the previous problem and aborts later on a double free. In gdb,

(gdb) run
Starting program: /opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test [Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Running main() from gtest_main.cc
[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 3 tests from AgradRevMatrix
[ RUN ] AgradRevMatrix.mat_cholesky
[ OK ] AgradRevMatrix.mat_cholesky (0 ms)
[ RUN ] AgradRevMatrix.exception_mat_cholesky
[ OK ] AgradRevMatrix.exception_mat_cholesky (0 ms)
[ RUN ] AgradRevMatrix.mat_cholesky_1st_deriv
*** Error in `/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test': double free or corruption (out): 0x000000000072f6c0 ***
Program received signal SIGABRT, Aborted.
0x00007ffff6fb9107 in raise () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0 0x00007ffff6fb9107 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007ffff6fba4e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007ffff6ff7214 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x00007ffff6ffc9ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x00007ffff6ffd6f6 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#5 0x000000000040bcf5 in Eigen::internal::aligned_free (ptr=0x72f6c0) at lib/eigen_3.2.4/Eigen/src/Core/util/Memory.h:243
#6 0x000000000041d75a in Eigen::internal::conditional_aligned_free<true> (ptr=0x72f6c0) at lib/eigen_3.2.4/Eigen/src/Core/util/Memory.h:317
#7 0x0000000000417628 in Eigen::internal::conditional_aligned_delete_auto<stan::math::var, true> (ptr=0x72f6c0, size=81) at lib/eigen_3.2.4/Eigen/src/Core/util/Memory.h:445
#8 0x0000000000410dc9 in Eigen::DenseStorage<stan::math::var, -1, -1, -1, 0>::~DenseStorage (this=0x7fffffffd860, __in_chrg=<optimized out>)
at lib/eigen_3.2.4/Eigen/src/Core/DenseStorage.h:236
#9 0x000000000040e7f8 in Eigen::PlainObjectBase<Eigen::Matrix<stan::math::var, -1, -1, 0, -1, -1> >::~PlainObjectBase (this=0x7fffffffd860, __in_chrg=<optimized out>)
at lib/eigen_3.2.4/Eigen/src/Core/PlainObjectBase.h:88
#10 0x000000000040e814 in Eigen::Matrix<stan::math::var, -1, -1, 0, -1, -1>::~Matrix (this=0x7fffffffd860, __in_chrg=<optimized out>) at lib/eigen_3.2.4/Eigen/src/Core/Matrix.h:127
#11 0x0000000000419bd8 in chol_functor::operator()<stan::math::var> (this=0x728af0, x=...) at test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp:46
#12 0x0000000000412b01 in stan::math::gradient<chol_functor> (f=..., x=..., fx=@0x727f70: 3.7989516472387148e+131, grad_fx=...) at ./stan/math/rev/mat/functor/gradient.hpp:54
#13 0x0000000000408527 in test_gradients (size=9) at test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp:115
#14 0x000000000040a78c in AgradRevMatrix_mat_cholesky_1st_deriv_Test::TestBody (this=0x7272e0) at test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp:247
#15 0x000000000048c8a0 in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void> (object=0x7272e0, method=&virtual testing::Test::TestBody(), location=0x4b205b "the test body") at lib/gtest_1.7.0/src/gtest.cc:2078
#16 0x00000000004880c2 in testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void> (object=0x7272e0, method=&virtual testing::Test::TestBody(), location=0x4b205b "the test body") at lib/gtest_1.7.0/src/gtest.cc:2114
#17 0x000000000046f1de in testing::Test::Run (this=0x7272e0) at lib/gtest_1.7.0/src/gtest.cc:2151
#18 0x000000000046f9ee in testing::TestInfo::Run (this=0x727300) at lib/gtest_1.7.0/src/gtest.cc:2326
#19 0x0000000000470095 in testing::TestCase::Run (this=0x727070) at lib/gtest_1.7.0/src/gtest.cc:2444
#20 0x0000000000476999 in testing::internal::UnitTestImpl::RunAllTests (this=0x726d50) at lib/gtest_1.7.0/src/gtest.cc:4315
#21 0x000000000048dcaa in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x726d50, method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0x4766f2 <testing::internal::UnitTestImpl::RunAllTests()>, location=0x4b28f0 "auxiliary test code (environments or event listeners)") at lib/gtest_1.7.0/src/gtest.cc:2078
#22 0x0000000000488ea2 in testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x726d50, method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0x4766f2 <testing::internal::UnitTestImpl::RunAllTests()>, location=0x4b28f0 "auxiliary test code (environments or event listeners)") at lib/gtest_1.7.0/src/gtest.cc:2114
#23 0x00000000004756eb in testing::UnitTest::Run (this=0x704020 <testing::UnitTest::GetInstance()::instance>) at lib/gtest_1.7.0/src/gtest.cc:3926
#24 0x000000000040561d in RUN_ALL_TESTS () at lib/gtest_1.7.0/include/gtest/gtest.h:2288
#25 0x00000000004055b7 in main (argc=1, argv=0x7fffffffe018) at lib/gtest_1.7.0/src/gtest_main.cc:37

bob-carpenter · 2015-08-22T02:45:09Z

Was there a version of this working that we can revert to for the
next release? It doesn't need to be optimally efficient, but it does
need to pass tests before we can put it in.

Bob

On Aug 21, 2015, at 4:21 PM, bgoodri notifications@github.com wrote:
But the test actually survives the previous problem and aborts later on a double free. In gdb,
(gdb) run
Starting program: /opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Running main() from gtest_main.cc
[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 3 tests from AgradRevMatrix
[ RUN ] AgradRevMatrix.mat_cholesky
[ OK ] AgradRevMatrix.mat_cholesky (0 ms)
[ RUN ] AgradRevMatrix.exception_mat_cholesky
[ OK ] AgradRevMatrix.exception_mat_cholesky (0 ms)
[ RUN ] AgradRevMatrix.mat_cholesky_1st_deriv
*** Error in `/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test': double free or corruption (out): 0x000000000072f6c0 ***
Program received signal SIGABRT, Aborted.
0x00007ffff6fb9107 in raise () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0 0x00007ffff6fb9107 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007ffff6fba4e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007ffff6ff7214 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x00007ffff6ffc9ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x00007ffff6ffd6f6 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#5 0x000000000040bcf5 in Eigen::internal::aligned_free (ptr=0x72f6c0) at lib/eigen_3.2.4/Eigen/src/Core/util/Memory.h:243
#6 0x000000000041d75a in Eigen::internal::conditional_aligned_free (ptr=0x72f6c0) at lib/eigen_3.2.4/Eigen/src/Core/util/Memory.h:317
#7 0x0000000000417628 in Eigen::internal::conditional_aligned_delete_auto<stan::math::var, true> (ptr=0x72f6c0, size=81) at lib/eigen_3.2.4/Eigen/src/Core/util/Memory.h:445
#8 0x0000000000410dc9 in Eigen::DenseStorage<stan::math::var, -1, -1, -1, 0>::~DenseStorage (this=0x7fffffffd860, __in_chrg=)
at lib/eigen_3.2.4/Eigen/src/Core/DenseStorage.h:236
#9 0x000000000040e7f8 in Eigen::PlainObjectBase<Eigen::Matrix<stan::math::var, -1, -1, 0, -1, -1> >::~PlainObjectBase (this=0x7fffffffd860, __in_chrg=)
at lib/eigen_3.2.4/Eigen/src/Core/PlainObjectBase.h:88
#10 0x000000000040e814 in Eigen::Matrix<stan::math::var, -1, -1, 0, -1, -1>::~Matrix (this=0x7fffffffd860, __in_chrg=) at lib/eigen_3.2.4/Eigen/src/Core/Matrix.h:127
#11 0x0000000000419bd8 in chol_functor::operator()stan::math::var (this=0x728af0, x=...) at test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp:46
#12 0x0000000000412b01 in stan::math::gradient<chol_functor> (f=..., x=..., fx=@0x727f70: 3.7989516472387148e+131, grad_fx=...) at ./stan/math/rev/mat/functor/gradient.hpp:54
#13 0x0000000000408527 in test_gradients (size=9) at test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp:115
#14 0x000000000040a78c in AgradRevMatrix_mat_cholesky_1st_deriv_Test::TestBody (this=0x7272e0) at test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp:247
#15 0x000000000048c8a0 in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void> (object=0x7272e0, method=&virtual testing::Test::TestBody(),
location=0x4b205b "the test body") at lib/gtest_1.7.0/src/gtest.cc:2078
#16 0x00000000004880c2 in testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void> (object=0x7272e0, method=&virtual testing::Test::TestBody(),
location=0x4b205b "the test body") at lib/gtest_1.7.0/src/gtest.cc:2114
#17 0x000000000046f1de in testing::Test::Run (this=0x7272e0) at lib/gtest_1.7.0/src/gtest.cc:2151
#18 0x000000000046f9ee in testing::TestInfo::Run (this=0x727300) at lib/gtest_1.7.0/src/gtest.cc:2326
#19 0x0000000000470095 in testing::TestCase::Run (this=0x727070) at lib/gtest_1.7.0/src/gtest.cc:2444
#20 0x0000000000476999 in testing::internal::UnitTestImpl::RunAllTests (this=0x726d50) at lib/gtest_1.7.0/src/gtest.cc:4315
#21 0x000000000048dcaa in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x726d50,
method=(bool (testing::internal::UnitTestImpl::)(testing::internal::UnitTestImpl * const)) 0x4766f2 testing::internal::UnitTestImpl::RunAllTests(),
location=0x4b28f0 "auxiliary test code (environments or event listeners)") at lib/gtest_1.7.0/src/gtest.cc:2078
#22 0x0000000000488ea2 in testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x726d50,
method=(bool (testing::internal::UnitTestImpl::)(testing::internal::UnitTestImpl * const)) 0x4766f2 testing::internal::UnitTestImpl::RunAllTests(),
location=0x4b28f0 "auxiliary test code (environments or event listeners)") at lib/gtest_1.7.0/src/gtest.cc:2114
#23 0x00000000004756eb in testing::UnitTest::Run (this=0x704020 testing::UnitTest::GetInstance()::instance) at lib/gtest_1.7.0/src/gtest.cc:3926
#24 0x000000000040561d in RUN_ALL_TESTS () at lib/gtest_1.7.0/include/gtest/gtest.h:2288
#25 0x00000000004055b7 in main (argc=1, argv=0x7fffffffe018) at lib/gtest_1.7.0/src/gtest_main.cc:37
—
Reply to this email directly or view it on GitHub.

bgoodri · 2015-08-22T02:50:29Z

I tried to git bisect it and as usual, didn't make much headway due to
intermediate commits being broken (some of which being commits previously
in develop that got merged into this branch).

On Fri, Aug 21, 2015 at 10:45 PM, Bob Carpenter notifications@github.com
wrote:

Was there a version of this working that we can revert to for the
next release? It doesn't need to be optimally efficient, but it does
need to pass tests before we can put it in.
Bob
On Aug 21, 2015, at 4:21 PM, bgoodri notifications@github.com wrote:
But the test actually survives the previous problem and aborts later on
a double free. In gdb,
(gdb) run
Starting program:
/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test
[Thread debugging using libthread_db enabled]
Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".
Running main() from gtest_main.cc
[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 3 tests from AgradRevMatrix
[ RUN ] AgradRevMatrix.mat_cholesky
[ OK ] AgradRevMatrix.mat_cholesky (0 ms)
[ RUN ] AgradRevMatrix.exception_mat_cholesky
[ OK ] AgradRevMatrix.exception_mat_cholesky (0 ms)
[ RUN ] AgradRevMatrix.mat_cholesky_1st_deriv
*** Error in
`/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test':
double free or corruption (out): 0x000000000072f6c0 ***
Program received signal SIGABRT, Aborted.
0x00007ffff6fb9107 in raise () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0 0x00007ffff6fb9107 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007ffff6fba4e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007ffff6ff7214 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x00007ffff6ffc9ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x00007ffff6ffd6f6 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#5 0x000000000040bcf5 in Eigen::internal::aligned_free (ptr=0x72f6c0) at
lib/eigen_3.2.4/Eigen/src/Core/util/Memory.h:243
#6 0x000000000041d75a in Eigen::internal::conditional_aligned_free
(ptr=0x72f6c0) at lib/eigen_3.2.4/Eigen/src/Core/util/Memory.h:317
#7 0x0000000000417628 in
Eigen::internal::conditional_aligned_delete_auto<stan::math::var, true>
(ptr=0x72f6c0, size=81) at lib/eigen_3.2.4/Eigen/src/Core/util/Memory.h:445
#8 0x0000000000410dc9 in Eigen::DenseStorage<stan::math::var, -1, -1,
-1, 0>::~DenseStorage (this=0x7fffffffd860, __in_chrg=)
at lib/eigen_3.2.4/Eigen/src/Core/DenseStorage.h:236
#9 0x000000000040e7f8 in
Eigen::PlainObjectBase<Eigen::Matrix<stan::math::var, -1, -1, 0, -1, -1>
::~PlainObjectBase (this=0x7fffffffd860, __in_chrg=)
at lib/eigen_3.2.4/Eigen/src/Core/PlainObjectBase.h:88
#10 0x000000000040e814 in Eigen::Matrix<stan::math::var, -1, -1, 0, -1,
-1>::~Matrix (this=0x7fffffffd860, __in_chrg=) at
lib/eigen_3.2.4/Eigen/src/Core/Matrix.h:127
#11 0x0000000000419bd8 in chol_functor::operator()stan::math::var
(this=0x728af0, x=...) at
test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp:46
#12 0x0000000000412b01 in stan::math::gradient<chol_functor> (f=...,
x=..., fx=@0x727f70: 3.7989516472387148e+131, grad_fx=...) at
./stan/math/rev/mat/functor/gradient.hpp:54
#13 0x0000000000408527 in test_gradients (size=9) at
test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp:115
#14 0x000000000040a78c in
AgradRevMatrix_mat_cholesky_1st_deriv_Test::TestBody (this=0x7272e0) at
test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp:247
#15 0x000000000048c8a0 in
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,
void> (object=0x7272e0, method=&virtual testing::Test::TestBody(),
location=0x4b205b "the test body") at lib/gtest_1.7.0/src/gtest.cc:2078
#16 0x00000000004880c2 in
testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>
(object=0x7272e0, method=&virtual testing::Test::TestBody(),
location=0x4b205b "the test body") at lib/gtest_1.7.0/src/gtest.cc:2114
#17 0x000000000046f1de in testing::Test::Run (this=0x7272e0) at
lib/gtest_1.7.0/src/gtest.cc:2151
#18 0x000000000046f9ee in testing::TestInfo::Run (this=0x727300) at
lib/gtest_1.7.0/src/gtest.cc:2326
#19 0x0000000000470095 in testing::TestCase::Run (this=0x727070) at
lib/gtest_1.7.0/src/gtest.cc:2444
#20 0x0000000000476999 in testing::internal::UnitTestImpl::RunAllTests
(this=0x726d50) at lib/gtest_1.7.0/src/gtest.cc:4315
#21 0x000000000048dcaa in
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
bool> (object=0x726d50,
method=(bool
(testing::internal::UnitTestImpl::)(testing::internal::UnitTestImpl *
const)) 0x4766f2 testing::internal::UnitTestImpl::RunAllTests(),
location=0x4b28f0 "auxiliary test code (environments or event
listeners)") at lib/gtest_1.7.0/src/gtest.cc:2078
#22 0x0000000000488ea2 in
testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
bool> (object=0x726d50,
method=(bool
(testing::internal::UnitTestImpl::)(testing::internal::UnitTestImpl *
const)) 0x4766f2 testing::internal::UnitTestImpl::RunAllTests(),
location=0x4b28f0 "auxiliary test code (environments or event
listeners)") at lib/gtest_1.7.0/src/gtest.cc:2114
#23 0x00000000004756eb in testing::UnitTest::Run (this=0x704020
testing::UnitTest::GetInstance()::instance) at
lib/gtest_1.7.0/src/gtest.cc:3926
#24 0x000000000040561d in RUN_ALL_TESTS () at
lib/gtest_1.7.0/include/gtest/gtest.h:2288
#25 0x00000000004055b7 in main (argc=1, argv=0x7fffffffe018) at
lib/gtest_1.7.0/src/gtest_main.cc:37
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub
#135 (comment).

bgoodri · 2015-08-22T04:23:20Z

I think we're just going to all have to stare at this until we figure out the memory problem. The -fsanitize=address thing points to a heap-buffer-overflow at line 155 of cholesky_decompose.hpp, which is the last line of the inner loop here

 for (size_type j = 0; j < L.cols(); ++j) {
for (size_type i = j; i <= L.cols(); ++i) { # should this be i < L.cols(); ?
accum_i += i;
size_t pos = j + accum_i;
L.coeffRef(i, j).vi_ = baseVari->variRefL_[pos];
}

which seems to suggest that pos gets too big. I put in print statement one line prior to that and got just before it blows up

pos = 0 should be at most 2
pos = 1 should be at most 2
pos = 3 should be at most 2
pos = 2 should be at most 2
pos = 4 should be at most 2

which seems like an odd way of counting up to something that is too big. Then I changed the inner loop to continue only until i < L.cols(); and then I got

pos = 0 should be at most 2
pos = 1 should be at most 2
pos = 2 should be at most 2

which causes the test to pass so I am going to push that.
But if I leave -fsanitize=address in then I hit a different error (now a stack-buffer-overflow)

==11240==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffeddbc2cb8 at pc 0x0000004170ce bp 0x7ffeddbc2a40 sp 0x7ffeddbc2a38
READ of size 8 at 0x7ffeddbc2cb8 thread T0
#0 0x4170cd in stan::math::value_of(stan::math::var const&) (/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test+0x4170cd)
#1 0x40e9ea in var_dot<Eigen::Block<const Eigen::Matrix<stan::math::var, -1, -1>, 1, -1, false>, Eigen::Block<const Eigen::Matrix<stan::math::var, -1, -1>, -1, 1, true> > stan/math/rev/mat/fun/dot_product.hpp:68
#2 0x40e5b1 in dot_product_vari<Eigen::Block<const Eigen::Matrix<stan::math::var, -1, -1>, 1, -1, false>, Eigen::Block<const Eigen::Matrix<stan::math::var, -1, -1>, -1, 1, true> > stan/math/rev/mat/fun/dot_product.hpp:165
#3 0x42149a in boost::enable_if_c<boost::is_same<stan::math::var, stan::math::var>::value||boost::is_same<stan::math::var, stan::math::var>::value, Eigen::Matrix<stan::math::var, -1, -1, ((Eigen::._123)0)|((((-1)==(1))&&((-1)!=(1)))?((Eigen::._123)1) : ((((-1)==(1))&&((-1)!=(1)))?((Eigen::._123)0) : ((Eigen::._123)0))), -1, -1> >::type stan::math::multiply<stan::math::var, -1, -1, stan::math::var, -1, -1>(Eigen::Matrix<stan::math::var, -1, -1, ((Eigen::._123)0)|((((-1)==(1))&&((-1)!=(1)))?((Eigen::._123)1) : ((((-1)==(1))&&((-1)!=(1)))?((Eigen::._123)0) : ((Eigen::._123)0))), -1, -1> const&, Eigen::Matrix<stan::math::var, -1, -1, ((Eigen::._123)0)|((((-1)==(1))&&((-1)!=(1)))?((Eigen::._123)1) : ((((-1)==(1))&&((-1)!=(1)))?((Eigen::._123)0) : ((Eigen::._123)0))), -1, -1> const&) (/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test+0x42149a)
#4 0x40d52b in AgradRevMatrix_mat_cholesky_Test::TestBody() test/unit/math/rev/mat/fun/cholesky_decompose_test.cpp:213
#5 0x4ca643 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) lib/gtest_1.7.0/src/gtest.cc:2078
#6 0x4c35c1 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) lib/gtest_1.7.0/src/gtest.cc:2114
#7 0x49eb25 in testing::Test::Run() lib/gtest_1.7.0/src/gtest.cc:2151
#8 0x49f719 in testing::TestInfo::Run() lib/gtest_1.7.0/src/gtest.cc:2326
#9 0x4a001d in testing::TestCase::Run() lib/gtest_1.7.0/src/gtest.cc:2444
#10 0x4a9f9c in testing::internal::UnitTestImpl::RunAllTests() lib/gtest_1.7.0/src/gtest.cc:4315
#11 0x4cc934 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) lib/gtest_1.7.0/src/gtest.cc:2078
#12 0x4c4d11 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) lib/gtest_1.7.0/src/gtest.cc:2114
#13 0x4a7dee in testing::UnitTest::Run() lib/gtest_1.7.0/src/gtest.cc:3926
#14 0x40614a in RUN_ALL_TESTS() lib/gtest_1.7.0/include/gtest/gtest.h:2288
#15 0x4060a6 in main lib/gtest_1.7.0/src/gtest_main.cc:37
#16 0x7ff30690ab44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b44)
#17 0x405fa8 (/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test+0x405fa8)
Address 0x7ffeddbc2cb8 is located in stack of thread T0 at offset 104 in frame
#0 0x421277 in boost::enable_if_c<boost::is_same<stan::math::var, stan::math::var>::value||boost::is_same<stan::math::var, stan::math::var>::value, Eigen::Matrix<stan::math::var, -1, -1, ((Eigen::._123)0)|((((-1)==(1))&&((-1)!=(1)))?((Eigen::._123)1) : ((((-1)==(1))&&((-1)!=(1)))?((Eigen::._123)0) : ((Eigen::._123)0))), -1, -1> >::type stan::math::multiply<stan::math::var, -1, -1, stan::math::var, -1, -1>(Eigen::Matrix<stan::math::var, -1, -1, ((Eigen::._123)0)|((((-1)==(1))&&((-1)!=(1)))?((Eigen::._123)1) : ((((-1)==(1))&&((-1)!=(1)))?((Eigen::._123)0) : ((Eigen::._123)0))), -1, -1> const&, Eigen::Matrix<stan::math::var, -1, -1, ((Eigen::._123)0)|((((-1)==(1))&&((-1)!=(1)))?((Eigen::._123)1) : ((((-1)==(1))&&((-1)!=(1)))?((Eigen::._123)0) : ((Eigen::._123)0))), -1, -1> const&) (/opt/stan/lib/stan_math_2.7.0/test/unit/math/rev/mat/fun/cholesky_decompose_test+0x421277)
This frame has 2 object(s):
[32, 72) 'crow'
[128, 168) 'ccol' <== Memory access at offset 104 underflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
(longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow ??:0 stan::math::value_of(stan::math::var const&)
Shadow bytes around the buggy address:
0x10005bb70540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10005bb70550: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 00 f4 f4
0x10005bb70560: f2 f2 f2 f2 00 00 f4 f4 f3 f3 f3 f3 00 00 00 00
0x10005bb70570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10005bb70580: 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 00
=>0x10005bb70590: 00 00 00 f4 f4 f4 f2[f2]f2 f2 00 00 00 00 00 f4
0x10005bb705a0: f4 f4 f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00
0x10005bb705b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10005bb705c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1
0x10005bb705d0: f1 f1 00 f4 f4 f4 f2 f2 f2 f2 00 f4 f4 f4 f2 f2
0x10005bb705e0: f2 f2 00 f4 f4 f4 f2 f2 f2 f2 00 f4 f4 f4 f2 f2
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
==11240==ABORTING
test/unit/math/rev/mat/fun/cholesky_decompose_test --gtest_output="xml:test/unit/math/rev/mat/fun/cholesky_decompose_test.xml" failed

bgoodri · 2015-08-22T05:58:45Z

This failing test in test/unit/math/mix/mat/fun/mdivide_right_spd_test.cpp is crappy and I don't think it really has anything to do with the PR. The test is expecting an uninitialized matrix to fail the positive definiteness test, but it comes up as

6.92261e-310 0 1.11469e-316
6.92261e-310 1.10892e-316 1.11469e-316
0 1.10892e-316 1.1147e-316

which is perhaps technically positive definite if it is considered symmetric. Anyway, we should set its elements to zero explicitly but on a different PR.

rtrangucci · 2015-08-22T15:17:31Z

@bgoodri Let me try committing a version of the code that doesn't respect the memory locality of the input and output matrices. I don't have gdb though, Ben; would you be able to run the -fsanitize=address flag?

bgoodri · 2015-08-22T16:12:19Z

It looks fixed by 3f4f966 . Why are we going backwards?

rtrangucci · 2015-08-22T16:27:15Z

I misinterpreted; I thought there was still a problem after the commit. You can nuke my latest commit

Rob

On Aug 22, 2015, at 12:12 PM, bgoodri notifications@github.com wrote:
It looks fixed by 3f4f966 . Why are we going backwards?
—
Reply to this email directly or view it on GitHub.

rtrangucci · 2015-08-23T06:49:12Z

@bgoodri Isn't it a bad sign that test/unit/math/mix/mat/fun/mdivide_right_spd_test.cpp fails under 3f4f966 but doesn't fail under 14815b3 ?

bgoodri · 2015-08-23T13:48:12Z

A test based on undefined behavior has no grounds to EXPECT_THROW or expect anything else. Moreover, the function it ostensibly tests is neither exposed nor used internally. Furthermore, the unused function should never be used because it is yet another embarrassing case where we do two Cholesky factorizations in order to check whether it is safe to do a Cholesky factorization. The checks arguably yield the wrong result because Ax = b has an infinite number of solutions when A is a zero matrix and b is a zero vector. The ultimate Cholesky factorization of A doesn't utilize Rob's cholesky_decompose and thus is going to be twice as slow and consume an order of magnitude more RAM. And if we cared about the 15th decimal place, then it would be better to do a LDLT Cholesky factorization of A than an LLT.

It isn't worth holding up this PR over, but we should eliminate or fix the test.

bob-carpenter · 2015-08-23T22:55:15Z

Agreed about the test being wrong. And thanks for fixing.
I just merged the fix into math.

Bob

On Aug 23, 2015, at 9:48 AM, bgoodri notifications@github.com wrote:
A test based on undefined behavior has no grounds to EXPECT_THROW or expect anything else. Moreover, the function it ostensibly tests is neither exposed nor used internally. Furthermore, the unused function should never be used because it is yet another embarrassing case where we do two Cholesky factorizations in order to check whether it is safe to do a Cholesky factorization. The checks arguably yield the wrong result because Ax = b has an infinite number of solutions when A is a zero matrix and b is a zero vector. The ultimate Cholesky factorization of A doesn't utilize Rob's cholesky_decompose and thus is going to be twice as slow and consume an order of magnitude more RAM. And if we cared about the 15th decimal place, then it would be better to do a LDLT Cholesky factorization of A than an LLT.
It isn't worth holding up this PR over, but we should eliminate or fix the test.
—
Reply to this email directly or view it on GitHub.

rtrangucci · 2015-08-25T04:22:39Z

@syclik @bgoodri What's the best path forward with this branch? I can git revert my last commit, or just reset --hard. Either is fine by me, but I'm not sure what the protocol is.

bgoodri · 2015-08-25T07:03:44Z

I would say to git pull develop into the cholesky branch and then

git revert 14815b3

Ben

On Tue, Aug 25, 2015 at 12:22 AM, Rob Trangucci notifications@github.com
wrote:

@syclik https://github.com/syclik @bgoodri https://github.com/bgoodri
What's the best path forward with this branch? I can git revert my last
commit, or just reset --hard. Either is fine by me, but I'm not sure what
the protocol is.
—
Reply to this email directly or view it on GitHub
#135 (comment).

rtrangucci · 2015-08-25T12:43:34Z

Cool, will do this afternoon so we can this merged.

Rob

On Aug 25, 2015, at 3:03 AM, bgoodri notifications@github.com wrote:
I would say to git pull develop into the cholesky branch and then
git revert 14815b3
Ben
On Tue, Aug 25, 2015 at 12:22 AM, Rob Trangucci notifications@github.com
wrote:
@syclik https://github.com/syclik @bgoodri https://github.com/bgoodri
What's the best path forward with this branch? I can git revert my last
commit, or just reset --hard. Either is fine by me, but I'm not sure what
the protocol is.
—
Reply to this email directly or view it on GitHub
#135 (comment).
—
Reply to this email directly or view it on GitHub.

syclik · 2015-08-25T15:20:23Z

@bgoodri ok'ed this at the meeting today.

…vative Fixes#125. Feature/issue 125 cholesky derivative

rtrangucci · 2015-08-25T15:33:50Z

Is that ok that it's not the memory locality preserving commit?? I thought we wanted to revert to Ben's last commit and then merge...

Rob

On Aug 25, 2015, at 11:20 AM, Daniel Lee notifications@github.com wrote:
Merged #135.
—
Reply to this email directly or view it on GitHub.

syclik · 2015-08-25T15:53:46Z

should I revert this pull request?

On Tue, Aug 25, 2015 at 11:33 AM, Rob Trangucci notifications@github.com
wrote:

Is that ok that it's not the memory locality preserving commit?? I thought
we wanted to revert to Ben's last commit and then merge...
Rob
On Aug 25, 2015, at 11:20 AM, Daniel Lee notifications@github.com
wrote:
Merged #135.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub
#135 (comment).

rtrangucci · 2015-08-25T16:00:28Z

Yes, you should revert the pull request, revert my last commit on the PR and then merge the branch into develop. Or I can do it later today. Either way.

Rob

On Aug 25, 2015, at 11:53 AM, Daniel Lee notifications@github.com wrote:
should I revert this pull request?
On Tue, Aug 25, 2015 at 11:33 AM, Rob Trangucci notifications@github.com
wrote:
Is that ok that it's not the memory locality preserving commit?? I thought
we wanted to revert to Ben's last commit and then merge...
Rob
On Aug 25, 2015, at 11:20 AM, Daniel Lee notifications@github.com
wrote:
Merged #135.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub
#135 (comment).
—
Reply to this email directly or view it on GitHub.

syclik · 2015-08-25T16:04:21Z

fuck. might have to revert the revert.

On Tue, Aug 25, 2015 at 12:00 PM, Rob Trangucci notifications@github.com
wrote:

Yes, you should revert the pull request, revert my last commit on the PR
and then merge the branch into develop. Or I can do it later today. Either
way.
Rob
On Aug 25, 2015, at 11:53 AM, Daniel Lee notifications@github.com
wrote:
should I revert this pull request?
On Tue, Aug 25, 2015 at 11:33 AM, Rob Trangucci <
notifications@github.com>
wrote:
Is that ok that it's not the memory locality preserving commit?? I
thought
we wanted to revert to Ben's last commit and then merge...
Rob
On Aug 25, 2015, at 11:20 AM, Daniel Lee notifications@github.com
wrote:
Merged #135.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub
#135 (comment).
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub
#135 (comment).

syclik · 2015-08-25T16:15:22Z

I got it.

On Tue, Aug 25, 2015 at 12:04 PM, Daniel Lee bearlee@alum.mit.edu wrote:

fuck. might have to revert the revert.
On Tue, Aug 25, 2015 at 12:00 PM, Rob Trangucci notifications@github.com
wrote:
Yes, you should revert the pull request, revert my last commit on the PR
and then merge the branch into develop. Or I can do it later today. Either
way.
Rob
On Aug 25, 2015, at 11:53 AM, Daniel Lee notifications@github.com
wrote:
should I revert this pull request?
On Tue, Aug 25, 2015 at 11:33 AM, Rob Trangucci <
notifications@github.com>
wrote:
Is that ok that it's not the memory locality preserving commit?? I
thought
we wanted to revert to Ben's last commit and then merge...
Rob
On Aug 25, 2015, at 11:20 AM, Daniel Lee notifications@github.com
wrote:
Merged #135.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub
#135 (comment).
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub
#135 (comment).

syclik · 2015-08-25T16:19:18Z

it's a new pull request: #156

rtrangucci added 12 commits July 22, 2015 18:19

b f/i-125 first cut at cholesky derivative structure
e72d17b

b f/i-125 analytic cholesky derivative works
5bb01ab

b f/i-125 wall-clock about as fast as Eigen, with 1/10th
019d64b
memory usage for large matrices

b f/i-125 sparse Eigen version, much slower than scalars
eb2cab4

b f/i-125 fastest implementation currently
22146e9

b f/i-125 cholesky-derivative nearly complete for _test.hpp
e59c4cc

b f/i-125 indexing fixed
3bcbdb6

b f/i-125 cleaned up file
7851547

b f/i-125 separate gradient rev tests from higher-order tests
0d9e659

b f/i-125 documentation and speedup, tests
fc5d674

b f/i-125 more doc
25b093c

Merge branch 'develop' into feature/issue-125-cholesky-derivative
38f5a82

b f/i-125 fixed cpplint errors
f173b77

rtrangucci added 2 commits August 2, 2015 17:55

b f/i-125 cholesky_decompose improvements
ef9c518
No storage of double* L_ in ctor Eigen access using coeffRef instead of raw arrays Use info about row-major algo to eliminate an O(N^2) loop after O(N^3) loop

Merge branch 'develop' into feature/issue-125-cholesky-derivative
c6e4822

syclik added 2 commits August 2, 2015 22:51

Merge branch 'develop' into feature/issue-125-cholesky-derivative
41c35ee

Merge branch 'develop' into feature/issue-125-cholesky-derivative
be37bb7

bob-carpenter reviewed Aug 4, 2015
View reviewed changes

Comment threadstan/math/rev/mat/fun/cholesky_decompose.hpp
bob-carpenterAug 4, 2015
Copy link
Copy Markdown
Member
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The variables should be private if you're being picky.

really end inner loop in the right place
3f4f966

back to non-memory locality preserving input/output
14815b3

bgoodri mentioned this pull request Aug 23, 2015
mdivide_right_spd_test.cpp is asinine #153
Closed

syclik added a commit that referenced this pull request Aug 25, 2015
Merge pull request#135from stan-dev/feature/issue-125-cholesky-deri…
1aaee22
…vative Fixes#125. Feature/issue 125 cholesky derivative

syclik merged commit 1aaee22 into developAug 25, 2015

syclik added this to the v2.7.1 milestone Aug 25, 2015

syclik added the code reviewed label Aug 25, 2015

syclik mentioned this pull request Aug 25, 2015
Revert "Feature/issue 125 cholesky derivative" #155
Merged

Uh oh!

Uh oh!

Conversation

rtrangucci commented Jul 30, 2015

Summary:

Intended Effect:

How to Verify:

Side Effects:

Documentation:

Reviewer Suggestions:

Uh oh!

jgabry commented Jul 30, 2015

Anyone

Uh oh!

rtrangucci commented Jul 31, 2015

Anyone

Uh oh!

bob-carpenter commented Jul 31, 2015

Anyone

Uh oh!

rtrangucci commented Aug 2, 2015

Uh oh!

syclik commented Aug 2, 2015

Uh oh!

rtrangucci commented Aug 2, 2015

Uh oh!

rtrangucci commented Aug 4, 2015

Uh oh!

bob-carpenter commented Aug 4, 2015

Uh oh!

rtrangucci commented Aug 4, 2015

Uh oh!

mbrubake commented Aug 4, 2015

Uh oh!

bob-carpenterAug 4, 2015

Choose a reason for hiding this comment

Uh oh!

bgoodri commented Aug 21, 2015

Uh oh!

bgoodri commented Aug 21, 2015

Uh oh!

bob-carpenter commented Aug 22, 2015

Uh oh!

bgoodri commented Aug 22, 2015

Uh oh!

bgoodri commented Aug 22, 2015

Uh oh!

bgoodri commented Aug 22, 2015

Uh oh!

rtrangucci commented Aug 22, 2015

Uh oh!

bgoodri commented Aug 22, 2015

Uh oh!

rtrangucci commented Aug 22, 2015

Uh oh!

rtrangucci commented Aug 23, 2015

Uh oh!

bgoodri commented Aug 23, 2015

Uh oh!

bob-carpenter commented Aug 23, 2015

Uh oh!

rtrangucci commented Aug 25, 2015

Uh oh!

bgoodri commented Aug 25, 2015

Uh oh!

rtrangucci commented Aug 25, 2015

Uh oh!

syclik commented Aug 25, 2015

Uh oh!

rtrangucci commented Aug 25, 2015

Uh oh!

syclik commented Aug 25, 2015

Uh oh!

rtrangucci commented Aug 25, 2015

Uh oh!

syclik commented Aug 25, 2015

Uh oh!

syclik commented Aug 25, 2015

Uh oh!

syclik commented Aug 25, 2015