Implemented a function for smarter debug formatting. by andrei-papou · Pull Request #606 · rust-ndarray/ndarray

andrei-papou · 2019-03-24T19:01:18Z

PR for #398

How it works:

let arr = Array3::from_elem((15,2,15),1.0);println!("{:?}", arr);

[[[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
[[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
[[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
[[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
[[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
...,
[[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
[[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
[[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
[[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]],
[[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0],
[1.0, 1.0, 1.0, 1.0, 1.0, ..., 1.0, 1.0, 1.0, 1.0, 1.0]]] shape=[15, 2, 15], strides=[30, 15, 1], layout=C (0x1), const ndim=3

let arr = Array3::from_elem((3,3,3),1.0);println!("{:?}", arr);

[[[1.0, 1.0, 1.0],
[1.0, 1.0, 1.0],
[1.0, 1.0, 1.0]],
[[1.0, 1.0, 1.0],
[1.0, 1.0, 1.0],
[1.0, 1.0, 1.0]],
[[1.0, 1.0, 1.0],
[1.0, 1.0, 1.0],
[1.0, 1.0, 1.0]]] shape=[3, 3, 3], strides=[9, 3, 1], layout=C (0x1), const ndim=3

The number of elements before ellipses is managed by the PRINT_ELEMENTS_LIMIT constant.

LukeMathWalker · 2019-03-25T21:49:47Z


+#[derive(Debug)]
+enum ArrayDisplayMode {
+ // Array is small enough to me printed without omitting any values.


Typo: to me printed

LukeMathWalker · 2019-03-25T22:03:09Z

The output looks quite nice!
I have left some comments pointing at a bunch of functions in std that could help in visually denoising the new function.
It would also be great to have some tests for these new routines, both for the selection of printing mode and the formatting itself.

- fixed typo in ArrayDisplayMode comment - updated ArrayDisplayMode constructor to use shape instead of array - fixed output for complex dimensions

SuperFluffy · 2019-03-27T10:50:46Z

 mod arrayformat;
 mod data_traits;

+pub use arrayformat::PRINT_ELEMENTS_LIMIT;


I'd refer to not reexport this public constant at the top level of the crate, but rather reexport it from some config or integration_test module.
As far as I can tell the motivation for making it pub is entirely for making it usable within the integration tests at tests/format.rs?

Awesome. I think that's better. I understand your motivation to have a way of configuring formatting similar to numpy's set_printoptions, but we wouldn't be able to actually accomplish that through const PRINT_ELEMENT_LIMIT. I am actually not sure right now if that was possible at all for a user of the library to accomplish at compile time.

SuperFluffy · 2019-03-27T10:54:05Z

I personally don't like that the implementation detail PRINTS_ELEMENTS_LIMIT is made public. Can the integration tests at tests/format.rs be made into unit tests under arrayformat.rs? Or is there a reason to have them as integration tests?

andrei-papou · 2019-03-27T11:18:41Z

@SuperFluffy I can move them to arrayformat.rs. But as far as I understand to convert those tests to real unit tests I should test array_format_v2 directly, not Display, Debug, etc. implementations. And I'm not really sure how to mock f: &mut fmt::Formatter and mut format: F arguments for the array_format_v2 function. Any help here is appreciated.

Although I agree that those tests should be unit tests, it would be good to give a user some control over PRINT_ELEMENTS_LIMIT. For example, numpy allows configuring this number using set_printoptions (see edgeitems argument).

andrei-papou · 2019-04-08T14:34:25Z

@LukeMathWalker @SuperFluffy any more comments on the changes? I think all the previous issues are fixed

LukeMathWalker · 2019-04-12T07:35:51Z

Sorry for the late reply @andrei-papou, it has been a crazy week.
I will try to set aside some time this weekend to go over the PR one last time.

LukeMathWalker · 2019-04-16T21:11:36Z

I have opened a PR with some observations @andrei-papou.

Smart debug formatting review

andrei-papou · 2019-04-17T11:17:41Z

@LukeMathWalker I tried to make the function more modular by extracting parts of the functionality to other functions and by reducing the number of boolean flags. I also got rid of the explicit second for loop. I hope the code is a bit more readable now. Please take a look

…sional

LukeMathWalker · 2019-04-23T09:11:00Z

I have opened a PR on your branch, trying out a different approach to solve this issue @andrei-papou - let me know what you think of it.

A different approach

LukeMathWalker · 2019-04-25T07:02:48Z

I am happy to merge this into master. Do you want to have a look @jturner314?

jturner314 · 2019-04-25T16:17:13Z

@LukeMathWalker Sure, I'm happy to review it, but feel free to go ahead and merge it if it looks good to you. I won't be able to review it until late next week or so. (It's the end of the semester right now, so I'm busy with exams.) Thanks for working with @andrei-papou on this PR.

LukeMathWalker · 2019-04-26T08:27:17Z

I am happy to call the shots and go ahead with this one - I'd like to avoid consuming your time on PRs I am confident reviewing (we need it for all the others were I have little context 👀).
Good luck with your exams @jturner314!

jturner314 · 2019-04-26T23:55:48Z

Thank you. :)

andrei.papou added 2 commits March 24, 2019 21:53

Implemented a function for smarter debug formatting.
fe4365c

Fixed existing tests
2450f50

andrei-papou mentioned this pull request Mar 24, 2019
Smarter Debug formatting for arrays #398
Closed

LukeMathWalker reviewed Mar 25, 2019
View reviewed changes

Comment threadsrc/arrayformat.rs Outdated

LukeMathWalker reviewed Mar 25, 2019
View reviewed changes

Comment threadsrc/arrayformat.rs Outdated

LukeMathWalker reviewed Mar 25, 2019
View reviewed changes

Comment threadsrc/arrayformat.rs Outdated

LukeMathWalker mentioned this pull request Mar 26, 2019
Are we ndarray yet? #597
Open
31 tasks

andrei-papou added 6 commits March 26, 2019 14:34

Some fixes:
3474d3f
- fixed typo in ArrayDisplayMode comment - updated ArrayDisplayMode constructor to use shape instead of array - fixed output for complex dimensions

Implemented axis shrinking for n-dim arrays
2a8a7c7

All the PR issues are fixed. Tests are needed.
1b5da67

Fixed the tests
fbf8cac

Added tests for 1- and 2-dimensional array with overflow
51c4e11

Try to fix 1.31 build failing
0d11708

SuperFluffy reviewed Mar 27, 2019
View reviewed changes

PRINT_ELEMENTS_LIMIT is now private
18c868a

LukeMathWalker added 8 commits April 16, 2019 20:39

Remove dead code
3aa8abe

Rename format_array_v2 to format_array
17f86bb

Simplify code using ndim
d48c5a9

Access last element directly
6c636dc

Add a test to specify behaviour for zero-dimensional arrays
88d53e3

Add a test for empty arrays
b962a7b

Test zero-length axes
29a8f70

We already know that ndim > 0
27c4a92

Andrewand others added 2 commits April 17, 2019 10:43

Merge pull request#2from LukeMathWalker/smart-debug-formatting-review
49b4338
Smart debug formatting review

Made the formatting logic more modular
b535f28

Simplify element access
2a8a3b0

LukeMathWalker mentioned this pull request Apr 20, 2019
Is there a way to use index_axis on a reference to ArrayBase? #620
Closed

LukeMathWalker added 8 commits April 20, 2019 17:03

Almost there
5fd9b2f

Clone for the win
f60185c

Cast to 1d array
198d689

First and last elements have to be special cased. Done for multidimen…
c1036fb
…sional

Extract in a separate function
5336b10

Tests are passing
4fa8a62

Minor improvements
e659390

Formatting
0abff2a

LukeMathWalkerand others added 3 commits April 23, 2019 20:16

Remove closure
e049abd

Use custom enum
83c9f08

Merge pull request#3from LukeMathWalker/refactoring-smarter-debugging
72e05d7
A different approach

LukeMathWalker merged commit 61b4835 into rust-ndarray:masterApr 26, 2019

bluss mentioned this pull request Sep 10, 2019
Provide a way to print the whole array #705
Closed

Uh oh!

Conversation

andrei-papou commented Mar 24, 2019

Uh oh!

LukeMathWalkerMar 25, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LukeMathWalker commented Mar 25, 2019

Uh oh!

SuperFluffyMar 27, 2019

Choose a reason for hiding this comment

Uh oh!

andrei-papouMar 28, 2019

Choose a reason for hiding this comment

Uh oh!

SuperFluffyMar 28, 2019

Choose a reason for hiding this comment

Uh oh!

SuperFluffy commented Mar 27, 2019

Uh oh!

andrei-papou commented Mar 27, 2019

Uh oh!

andrei-papou commented Apr 8, 2019

Uh oh!

LukeMathWalker commented Apr 12, 2019

Uh oh!

LukeMathWalker commented Apr 16, 2019

Uh oh!

andrei-papou commented Apr 17, 2019

Uh oh!

LukeMathWalker commented Apr 23, 2019

Uh oh!

LukeMathWalker commented Apr 25, 2019

Uh oh!

jturner314 commented Apr 25, 2019

Uh oh!

LukeMathWalker commented Apr 26, 2019

Uh oh!

jturner314 commented Apr 26, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants