STL: Visualize more types

[StephanTLavavej]

It's been a while since we've done a "visualizer pass", looking at all of the types that have been added to the STL in recent years, and determining whether their representations are comprehensible in the debugger, or whether we need to improve how they're displayed.

std::filesystem::path, shared_ptr<T[]>, and atomic_flag probably need to be visualized, but the list is likely longer. We should also consider future and span.

std::flat_set (currently on the feature/flat_map feature branch) needs a visualizer.

Also tracked by Microsoft-internal VSO-602066 / AB#602066.

[CaseyCarter]

Per the Debugger team, the debugger now understands char8_t; we should ensure that std::u8string and std::u8string_view visualize properly.

[mwinterb]

std::u8string_view works great since the visualizer for std::basic_string_view doesn't do anything overly special. The std::basic_string visualizers, however, are specialized for each known character type, char, wchar_t, unsigned short, char16_t, and char32_t. It looks like adding an AlternativeType entry for char8_t under the char specialization is enough.

First block is for std::basic_string<char8_t>, second block is for std::basic_string<char8_t>::iterator

Comparing files woo.stl.natvis and STL.NATVIS
***** woo.stl.natvis
  <Type Name="std::basic_string&lt;char,*&gt;">
    <Intrinsic Name="size" Expression="_Mypair._Myval2._Mysize" />
***** STL.NATVIS
  <Type Name="std::basic_string&lt;char,*&gt;">
    <AlternativeType Name="std::basic_string&lt;char8_t,*&gt;" />
    <Intrinsic Name="size" Expression="_Mypair._Myval2._Mysize" />
*****

***** woo.stl.natvis
      <AlternativeType Name="std::_String_const_iterator&lt;std::_String_val&lt;std::_Simple_types&lt;char&gt; &gt; &gt;" />
      <SmartPointer Usage="Indexable">_Ptr,na</SmartPointer>
***** STL.NATVIS
      <AlternativeType Name="std::_String_const_iterator&lt;std::_String_val&lt;std::_Simple_types&lt;char&gt; &gt; &gt;" />
      <AlternativeType Name="std::_String_iterator&lt;std::_String_val&lt;std::_Simple_types&lt;char8_t&gt; &gt; &gt;" />
      <AlternativeType Name="std::_String_const_iterator&lt;std::_String_val&lt;std::_Simple_types&lt;char8_t&gt; &gt; &gt;" />

      <SmartPointer Usage="Indexable">_Ptr,na</SmartPointer>
*****

Without those changes, i.e. what it looks like in 16.8.0 Preview 6.0:

[CaseyCarter]

It looks like adding an AlternativeType entry for char8_t under the char specialization is enough.

Color me surprised. I wonder if we could merge the other known-encoding types (char32_t, char16_t, perhaps even wchar_t on Windows) into the base visualizer as well?

[mwinterb]

Color me surprised. I wonder if we could merge the other known-encoding types (char32_t, char16_t, perhaps even wchar_t on Windows) into the base visualizer as well?

For the basic_string visualizers, there's this comment: Hard coding _BUF_SIZE for clang-cl compatibility; clang-cl as of 7.0.1 does not emit S_CONSTANT to get _BUF_SIZE. May 2019 seems to be when clang-cl was updated to emit S_CONSTANTs: https://reviews.llvm.org/D61926, which was probably for clang-9 at the earliest based on the releases page. Since there's only one STL.natvis file for VS2019 regardless of toolset chosen, and VS2019 seems to allow targeting the VS2015 toolset, presumably support for clang-7 and clang-8 is still required.
However, that may not be a limiting factor: removing all customizations for basic_string and replacing the hard coded bufSize intrinsic with this expression <Intrinsic Name="bufSize" Expression="16 / sizeof($T1) &lt; 1 ? 1 : 16 / sizeof($T1)" /> seems to work:

Natvis: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\Packages\Debugger\Visualizers\stl.natvis(750,6): Successfully parsed expression '16 / sizeof(char) < 1 ? 1 : 16 / sizeof(char)' in type context 'std::basic_string<char,std::char_traits<char>,std::allocator<char> >'.
Natvis: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\Packages\Debugger\Visualizers\stl.natvis(750,6): Successfully parsed expression '16 / sizeof(char16_t) < 1 ? 1 : 16 / sizeof(char16_t)' in type context 'std::basic_string<char16_t,std::char_traits<char16_t>,std::allocator<char16_t> >'.
Natvis: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\Packages\Debugger\Visualizers\stl.natvis(750,6): Successfully parsed expression '16 / sizeof(char32_t) < 1 ? 1 : 16 / sizeof(char32_t)' in type context 'std::basic_string<char32_t,std::char_traits<char32_t>,std::allocator<char32_t> >'.
Natvis: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\Packages\Debugger\Visualizers\stl.natvis(750,6): Successfully parsed expression '16 / sizeof(wchar_t) < 1 ? 1 : 16 / sizeof(wchar_t)' in type context 'std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> >'.

(Short strings also still visualize as expected. And since this isn't with Preview, I couldn't verify u8string still works, but that'd be surprising if it didn't work in the latest Preview given that AlternativeType did.)
This is with using a week-old version of clang from HEAD, but since there's no usage of _BUF_SIZE in the natvis file I don't think that's coming into play.
However, the /Zc:wchar_t- option to treat wchar_t as a type alias for unsigned short limits the full utility, since a UTF16 customization is still necessary. Avoiding all of the AlternativeTypes for wchar_t and char16_t does have the benefit of being easier on the eyes and maintenance since there are fewer < and > to deal with. There may also be concerns with user defined character types, which I think would not be visualized at all today, but I don't know what's "desired".

Attached so that others can double-check my claims:
stl.natvis.txt

[AnjuDel]

Added new project to track this
https://github.com/microsoft/STL/projects/10

[mwinterb]

As a note in case someone else searches for this, u8string is visualized as of #1856.

[amasciotta]

Hi! I wrote this for std::chrono::system_clock::time_point. It's just a starting point, but maybe it can help in some way

<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
  <Type Name="std::chrono::time_point&lt;std::chrono::system_clock,std::chrono::duration&lt;__int64,std::ratio&lt;1,10000000&gt; &gt; &gt;">
    <DisplayString> {_MyDur._MyRep/36000000000}:{(_MyDur._MyRep - 36000000000*(_MyDur._MyRep/36000000000))/6000000000}{(_MyDur._MyRep - 6000000000*(_MyDur._MyRep/6000000000))/600000000}:{(_MyDur._MyRep - 600000000*(_MyDur._MyRep/600000000))/100000000}{(_MyDur._MyRep - 100000000*(_MyDur._MyRep/100000000))/10000000}.{(_MyDur._MyRep - 10000000*(_MyDur._MyRep/10000000))/1000000}{(_MyDur._MyRep - 1000000*(_MyDur._MyRep/1000000))/100000}{(_MyDur._MyRep - 100000*(_MyDur._MyRep/100000))/10000}{(_MyDur._MyRep - 10000*(_MyDur._MyRep/10000))/1000}{(_MyDur._MyRep - 1000*(_MyDur._MyRep/1000))/100}{(_MyDur._MyRep - 100*(_MyDur._MyRep/100))/10} hh:mm:ss</DisplayString>
    <Expand/>
  </Type>
</AutoVisualizer>

It produces something like this: 259798:00:00.000000 hh:mm:ss

[amasciotta]

A little improvement

<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
  <Type Name="std::chrono::time_point&lt;std::chrono::system_clock,std::chrono::duration&lt;__int64,std::ratio&lt;1,10000000&gt; &gt; &gt;">
    <DisplayString>{(_MyDur._MyRep / (24 * 60 * 60 * 10000000ull))} days {(_MyDur._MyRep % (24 * 60 * 60 * 10000000ull))/(10 * 60 * 60 * 10000000ull)}{(_MyDur._MyRep % (4 * 60 * 60 * 10000000ull))/(60 * 60 * 10000000ull)}:{(_MyDur._MyRep % (60 * 60 * 10000000ull))/(10 * 60 * 10000000ull)}{(_MyDur._MyRep % (10 * 60 * 10000000ull)) / (60 * 10000000ull)}:{(_MyDur._MyRep % (60 * 10000000ull)) / (10 * 10000000ull)}{(_MyDur._MyRep % (10 * 10000000ull)) / 10000000ull}.{(_MyDur._MyRep % 10000000) / 1000000}{(_MyDur._MyRep % 1000000) / 100000}{(_MyDur._MyRep % 100000) / 10000}{(_MyDur._MyRep % 10000) / 1000}{(_MyDur._MyRep % 1000) / 100}{(_MyDur._MyRep % 100) / 10}{_MyDur._MyRep % 10} hh:mm:ss</DisplayString>
    <Expand/>
  </Type>
</AutoVisualizer>

Now it prints 1095 days 00:00:00.0000000 hh:mm:ss

[amasciotta]

I managed to print the year and the day of the year. Added some comments to explain the computation.

Now it prints year: 1985 DoY: 364 23:48:59.1234567 hh:mm:ss

<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
  <Type Name="std::chrono::time_point&lt;std::chrono::system_clock,std::chrono::duration&lt;__int64,std::ratio&lt;1,10000000&gt; &gt; &gt;">
      <!--
      The computation of the year is performed in this way:
      1. we divide the years into 4-years batches, ending with a leap year: 69+70+71+72, 73+74+75+76, etc
      2. we compute which of these batches the current year belongs to:
        a. days_since_1970 = (_MyDur._MyRep / (24 * 60 * 60 * 10000000ull)).
        b. days_since_1969 = days_since_1970 + 365
        c. days_in_a_batch = (365 * 3 + 366)
        d. batch_index = days_since_1969 / days_in_a_batch
      3. we determine which year of the current batch the current year belongs to:
        a. day_in_the_batch = days_since_1969 % days_in_a_batch (note the % instead of / aas in 2.d)
        b. year_in_the_batch_index = day_in_the_batch <= 365 * 3 ? day_in_the_batch / 365 : 3
      4. we sum everyting together, starting from 1969:
        year = 1969 + 4 * batch_index + year_in_the_batch_index
      
      The computation of the day of the year is performed in this way:
	  1. If day_in_the_batch <= 365 * 3, then we compute day_in_the_batch % 365
	  2. Otherwise, we compute day_in_the_batch - 3 * 365
	  
	  NOTE: this computation ignores the non-leap years multiples of 100. Since 2000 is also multiple of 400, it is a leap year anyway.
	        Therefore this makes the computation valid until 2099.
       -->
        <DisplayString>year: {
		1969 +
		4 * (((_MyDur._MyRep / (24 * 60 * 60 * 10000000ull)) + 365) / (365 * 3 + 366)) +
		(((_MyDur._MyRep / (24 * 60 * 60 * 10000000ull)) + 365) % (365 * 3 + 366) &lt;= (3 * 365) ? (((_MyDur._MyRep / (24 * 60 * 60 * 10000000ull)) + 365) % (365 * 3 + 366)) / 365 : 3)
		} DoY: {
		((_MyDur._MyRep / (24 * 60 * 60 * 10000000ull)) + 365) % (365 * 3 + 366) &lt;= (3 * 365) ? (((_MyDur._MyRep / (24 * 60 * 60 * 10000000ull)) + 365) % (365 * 3 + 366)) % 365 : (((_MyDur._MyRep / (24 * 60 * 60 * 10000000ull)) + 365) % (365 * 3 + 366)) - 365 * 3
		} {
		(_MyDur._MyRep % (24 * 60 * 60 * 10000000ull))/(10 * 60 * 60 * 10000000ull)}{(_MyDur._MyRep % (4 * 60 * 60 * 10000000ull))/(60 * 60 * 10000000ull)}:{(_MyDur._MyRep % (60 * 60 * 10000000ull))/(10 * 60 * 10000000ull)}{(_MyDur._MyRep % (10 * 60 * 10000000ull)) / (60 * 10000000ull)}:{(_MyDur._MyRep % (60 * 10000000ull)) / (10 * 10000000ull)}{(_MyDur._MyRep % (10 * 10000000ull)) / 10000000ull}.{(_MyDur._MyRep % 10000000) / 1000000}{(_MyDur._MyRep % 1000000) / 100000}{(_MyDur._MyRep % 100000) / 10000}{(_MyDur._MyRep % 10000) / 1000}{(_MyDur._MyRep % 1000) / 100}{(_MyDur._MyRep % 100) / 10}{_MyDur._MyRep % 10
		} hh:mm:ss
	</DisplayString>
    <Expand/>
</Type>
</AutoVisualizer>

[gh-tma2]

Can we please get visualizer support for time_point into the visualizers shipped with VS2022? I'm tired at looking at _MyDur {_MyRep=16457922110000000 }

[StephanTLavavej]

gh-tma2 This is tracked by the Visualizers Project - I've added a note to the time_point entry that you've requested it.

[samrrr]

I managed to print the year and the day of the year. Added some comments to explain the computation.

Now it prints year: 1985 DoY: 364 23:48:59.1234567 hh:mm:ss

<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
  <Type Name="std::chrono::time_point&lt;std::chrono::system_clock,std::chrono::duration&lt;__int64,std::ratio&lt;1,10000000&gt; &gt; &gt;">
      <!--
      The computation of the year is performed in this way:
      1. we divide the years into 4-years batches, ending with a leap year: 69+70+71+72, 73+74+75+76, etc
      2. we compute which of these batches the current year belongs to:
        a. days_since_1970 = (_MyDur._MyRep / (24 * 60 * 60 * 10000000ull)).
        b. days_since_1969 = days_since_1970 + 365
        c. days_in_a_batch = (365 * 3 + 366)
        d. batch_index = days_since_1969 / days_in_a_batch
      3. we determine which year of the current batch the current year belongs to:
        a. day_in_the_batch = days_since_1969 % days_in_a_batch (note the % instead of / aas in 2.d)
        b. year_in_the_batch_index = day_in_the_batch <= 365 * 3 ? day_in_the_batch / 365 : 3
      4. we sum everyting together, starting from 1969:
        year = 1969 + 4 * batch_index + year_in_the_batch_index
      
      The computation of the day of the year is performed in this way:
	  1. If day_in_the_batch <= 365 * 3, then we compute day_in_the_batch % 365
	  2. Otherwise, we compute day_in_the_batch - 3 * 365
	  
	  NOTE: this computation ignores the non-leap years multiples of 100. Since 2000 is also multiple of 400, it is a leap year anyway.
	        Therefore this makes the computation valid until 2099.
       -->
        <DisplayString>year: {
		1969 +
		4 * (((_MyDur._MyRep / (24 * 60 * 60 * 10000000ull)) + 365) / (365 * 3 + 366)) +
		(((_MyDur._MyRep / (24 * 60 * 60 * 10000000ull)) + 365) % (365 * 3 + 366) &lt;= (3 * 365) ? (((_MyDur._MyRep / (24 * 60 * 60 * 10000000ull)) + 365) % (365 * 3 + 366)) / 365 : 3)
		} DoY: {
		((_MyDur._MyRep / (24 * 60 * 60 * 10000000ull)) + 365) % (365 * 3 + 366) &lt;= (3 * 365) ? (((_MyDur._MyRep / (24 * 60 * 60 * 10000000ull)) + 365) % (365 * 3 + 366)) % 365 : (((_MyDur._MyRep / (24 * 60 * 60 * 10000000ull)) + 365) % (365 * 3 + 366)) - 365 * 3
		} {
		(_MyDur._MyRep % (24 * 60 * 60 * 10000000ull))/(10 * 60 * 60 * 10000000ull)}{(_MyDur._MyRep % (4 * 60 * 60 * 10000000ull))/(60 * 60 * 10000000ull)}:{(_MyDur._MyRep % (60 * 60 * 10000000ull))/(10 * 60 * 10000000ull)}{(_MyDur._MyRep % (10 * 60 * 10000000ull)) / (60 * 10000000ull)}:{(_MyDur._MyRep % (60 * 10000000ull)) / (10 * 10000000ull)}{(_MyDur._MyRep % (10 * 10000000ull)) / 10000000ull}.{(_MyDur._MyRep % 10000000) / 1000000}{(_MyDur._MyRep % 1000000) / 100000}{(_MyDur._MyRep % 100000) / 10000}{(_MyDur._MyRep % 10000) / 1000}{(_MyDur._MyRep % 1000) / 100}{(_MyDur._MyRep % 100) / 10}{_MyDur._MyRep % 10
		} hh:mm:ss
	</DisplayString>
    <Expand/>
</Type>
</AutoVisualizer>

11%4 is 3 (you need 1)