P2675R1 Improving `std::format`'s Width Estimation

[StephanTLavavej]

WG21-P2675R1 Improving std::format's Width Estimation

No feature-test macro is mentioned.

[achabense]

I hope to work on this.

According to the paper, I need to look for all ranges marked ;W or ;F (with awareness of default-value) in
https://www.unicode.org/Public/UCD/latest/ucd/EastAsianWidth.txt, or the equivalent, currently latest version
https://www.unicode.org/Public/15.0.0/ucd/EastAsianWidth.txt
and combine them with U+4DC0 – U+4DFF, U+1F300 – U+1F5FF and U+1F900 – U+1F9FF to make a table, then update _Width_estimate_intervals:

STL/stl/inc/format

Lines 981 to 982 in 8d18fec

	inline constexpr char32_t _Width_estimate_intervals[] = { // Per N4928 [format.string.std]/12
	0x1100u, 0x1160u, 0x2329u, 0x232Bu, 0x2E80u, 0x303Fu, 0x3040u, 0xA4D0u, 0xAC00u, 0xD7A4u, 0xF900u, 0xFB00u, 0xFE10u,

Is it enough to complete this feature by ~?

1. Introduce a parser to /tools, taking the .txt file as input to make the new table
2. Update _Width_estimate_intervals, and do optimizations if necessary

[achabense]

main...achabense:STL:P2675R1

I'm confused by this:
20000..2FFFF, 30000...3FFFF
vs
20000..2FFFD, 30000...3FFFD
By taking the second one(that in the EastAsianWidth.txt file), I can now reproduce exactly the same result as in Annex: Exhaustive list of changes in the paper.

C++20 vs C++23

Was 1, now 2:
231A-231B
23E9-23EC
23F0
23F3
25FD-25FE
2614-2615
2648-2653
267F
2693
26A1
26AA-26AB
26BD-26BE
26C4-26C5
26CE
26D4
26EA
26F2-26F3
26F5
26FA
26FD
2705
270A-270B
2728
274C
274E
2753-2755
2757
2795-2797
27B0
27BF
2B1B-2B1C
2B50
2B55
A960-A97C
16FE0-16FE4
16FF0-16FF1
17000-187F7
18800-18CD5
18D00-18D08
1AFF0-1AFF3
1AFF5-1AFFB
1AFFD-1AFFE
1B000-1B122
1B132
1B150-1B152
1B155
1B164-1B167
1B170-1B2FB
1F004
1F0CF
1F18E
1F191-1F19A
1F200-1F202
1F210-1F23B
1F240-1F248
1F250-1F251
1F260-1F265
1F680-1F6C5
1F6CC
1F6D0-1F6D2
1F6D5-1F6D7
1F6DC-1F6DF
1F6EB-1F6EC
1F6F4-1F6FC
1F7E0-1F7EB
1F7F0
1FA70-1FA7C
1FA80-1FA88
1FA90-1FABD
1FABF-1FAC5
1FACE-1FADB
1FAE0-1FAE8
1FAF0-1FAF8

Was 2, now 1:
2E9A
2EF4-2EFF
2FD6-2FEF
2FFC-2FFF
3040
3097-3098
3100-3104
3130
318F
31E4-31EF
321F
3248-324F
A48D-A48F
A4C7-A4CF
FE53
FE67
FE6C-FE6F
FF00

The full interval list looks horrible:

new list for _Width_estimate_intervals

0x1100u, 0x1160u, 0x231Au, 0x231Cu, 0x2329u, 0x232Bu, 0x23E9u, 0x23EDu, 0x23F0u, 0x23F1u, 0x23F3u, 0x23F4u,
0x25FDu, 0x25FFu, 0x2614u, 0x2616u, 0x2648u, 0x2654u, 0x267Fu, 0x2680u, 0x2693u, 0x2694u, 0x26A1u, 0x26A2u,
0x26AAu, 0x26ACu, 0x26BDu, 0x26BFu, 0x26C4u, 0x26C6u, 0x26CEu, 0x26CFu, 0x26D4u, 0x26D5u, 0x26EAu, 0x26EBu,
0x26F2u, 0x26F4u, 0x26F5u, 0x26F6u, 0x26FAu, 0x26FBu, 0x26FDu, 0x26FEu, 0x2705u, 0x2706u, 0x270Au, 0x270Cu,
0x2728u, 0x2729u, 0x274Cu, 0x274Du, 0x274Eu, 0x274Fu, 0x2753u, 0x2756u, 0x2757u, 0x2758u, 0x2795u, 0x2798u,
0x27B0u, 0x27B1u, 0x27BFu, 0x27C0u, 0x2B1Bu, 0x2B1Du, 0x2B50u, 0x2B51u, 0x2B55u, 0x2B56u, 0x2E80u, 0x2E9Au,
0x2E9Bu, 0x2EF4u, 0x2F00u, 0x2FD6u, 0x2FF0u, 0x2FFCu, 0x3000u, 0x303Fu, 0x3041u, 0x3097u, 0x3099u, 0x3100u,
0x3105u, 0x3130u, 0x3131u, 0x318Fu, 0x3190u, 0x31E4u, 0x31F0u, 0x321Fu, 0x3220u, 0x3248u, 0x3250u, 0xA48Du,
0xA490u, 0xA4C7u, 0xA960u, 0xA97Du, 0xAC00u, 0xD7A4u, 0xF900u, 0xFB00u, 0xFE10u, 0xFE1Au, 0xFE30u, 0xFE53u,
0xFE54u, 0xFE67u, 0xFE68u, 0xFE6Cu, 0xFF01u, 0xFF61u, 0xFFE0u, 0xFFE7u, 0x16FE0u, 0x16FE5u, 0x16FF0u, 0x16FF2u,
0x17000u, 0x187F8u, 0x18800u, 0x18CD6u, 0x18D00u, 0x18D09u, 0x1AFF0u, 0x1AFF4u, 0x1AFF5u, 0x1AFFCu, 0x1AFFDu, 0x1AFFFu,
0x1B000u, 0x1B123u, 0x1B132u, 0x1B133u, 0x1B150u, 0x1B153u, 0x1B155u, 0x1B156u, 0x1B164u, 0x1B168u, 0x1B170u, 0x1B2FCu,
0x1F004u, 0x1F005u, 0x1F0CFu, 0x1F0D0u, 0x1F18Eu, 0x1F18Fu, 0x1F191u, 0x1F19Bu, 0x1F200u, 0x1F203u, 0x1F210u, 0x1F23Cu,
0x1F240u, 0x1F249u, 0x1F250u, 0x1F252u, 0x1F260u, 0x1F266u, 0x1F300u, 0x1F650u, 0x1F680u, 0x1F6C6u, 0x1F6CCu, 0x1F6CDu,
0x1F6D0u, 0x1F6D3u, 0x1F6D5u, 0x1F6D8u, 0x1F6DCu, 0x1F6E0u, 0x1F6EBu, 0x1F6EDu, 0x1F6F4u, 0x1F6FDu, 0x1F7E0u, 0x1F7ECu,
0x1F7F0u, 0x1F7F1u, 0x1F900u, 0x1FA00u, 0x1FA70u, 0x1FA7Du, 0x1FA80u, 0x1FA89u, 0x1FA90u, 0x1FABEu, 0x1FABFu, 0x1FAC6u,
0x1FACEu, 0x1FADCu, 0x1FAE0u, 0x1FAE9u, 0x1FAF0u, 0x1FAF9u, 0x20000u, 0x2FFFEu, 0x30000u, 0x3FFFEu,

I find it challenging to update test cases for this... I'm mistaken by a test case not for _Unicode_width_estimate...

Do I need to update the table directly, or keeping the old table in C++20 mode?

[frederick-vs-ja]

Do I need to update the table directly, or keeping the old table in C++20 mode?

I believe we should change the table in C++20 mode, as P2675R1 is essentially the resultion of LWG-3780. MSVC STL gererally implement resolutions of LWG issues in old modes when they are merely fixes of old features.