Uh oh!
There was an error while loading. Please reload this page.
gdb Pretty Print: generic encoded was failing on reference/pointer types - #33612
Conversation
rust-highfive
commented
May 13, 2016
Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @nikomatsakis (or someone else) soon. If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes. Please see the contribution instructions for more information. |
nikomatsakis
commented
May 13, 2016
Hmm, do we have any unit testing for these pretty printers? |
michaelwoerister
commented
May 13, 2016
We had tests for this at some point: I don't remember why they were removed. |
michaelwoerister
commented
May 13, 2016
@bors: r+ |
bors
commented
May 13, 2016
📌 Commit 04baf25 has been approved by |
bors
commented
May 15, 2016
…lwoerister
gdb Pretty Print: generic encoded was failing on reference/pointer types
If you debug this program using **gdb**
```rust
fn main() {
let x = 10;
let y = Some(&x);
// additional code
}
```
And you try to print **y**'s value from the debugger, you get the following:
```
(gdb) print y
Python Exception <class 'gdb.error'> Cannot convert value to int.:
$1 = {RUST$ENCODED$ENUM$0$None = Some = {0x7fff5fbff97c}}
```
What happens is that inside **debugger_pretty_printers_common.py** the method `is_null_variant` doesn't have any special handling for pointer values so it ends up calling `.as_integer()` on `discriminant_val` (which holds a pointer) and fails.
Considering it needs to handle pointers and return _true_ when the pointer is _null_, I modified the `.as_integer()` method in **gdb_rust_pretty_printing.py** to take pointers into consideration.
After this modification **gdb** prints **y** like this:
```
(gdb) print y
$1 = Some = {0x7fff5fbff97c}
```
Now, it would be nice to print something useful (instead of a pointer address) but the pretty printer doesn't currently handle references/pointers so that's a completely different subject.
If you debug this program using gdb
And you try to print y's value from the debugger, you get the following:
What happens is that inside debugger_pretty_printers_common.py the method
is_null_variantdoesn't have any special handling for pointer values so it ends up calling.as_integer()ondiscriminant_val(which holds a pointer) and fails.Considering it needs to handle pointers and return true when the pointer is null, I modified the
.as_integer()method in gdb_rust_pretty_printing.py to take pointers into consideration.After this modification gdb prints y like this:
Now, it would be nice to print something useful (instead of a pointer address) but the pretty printer doesn't currently handle references/pointers so that's a completely different subject.