Uh oh!
There was an error while loading. Please reload this page.
Bump the pinned standard library, and check its string handling - #1240
Merged
Conversation
Moves the pin from 16729fa to 98b1140. The library turns multibyte support on by default as of this range, and works the engine's behaviour out at runtime rather than being told: it cuts a character in half to see how a partial slice is represented, and slices a literal byte by byte to enumerate every continuation byte. That rests on a string being a sequence of bytes, which the interpreter now agrees with. Nothing runs the library's own tests, so a bump was otherwise only checked for still compiling. StdLibStringTests covers the part this one turns on: a multibyte length in bytes, a position inside a character reported as not a boundary, and ascii unaffected. The middle one is the detection working end to end, since it only answers that way if slicing produced the partial bytes the library expects to find.
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves the pinned standard library from
16729fato98b1140, twenty-one commits on.This is the bump that was blocked until the interpreter agreed with the game on byte semantics. The library enables multibyte support by default over this range and works the engine's behaviour out at runtime rather than being told:
String.wurstcuts a character in half to see how a partial slice is represented, and slices a 64 character literal byte by byte to enumerate every continuation byte. All of that rests on a string being a sequence of bytes.Why there are tests attached to a version bump
Nothing in this repository runs the standard library's own test functions, so a bump has only ever been checked for still compiling. That is a weak signal for a change whose whole point is behaviour: the library's multibyte detection is designed to degrade quietly to an ascii-only path when it cannot find what it probes for, so a version where it silently gave up would look exactly like a version where it worked.
StdLibStringTestschecks the part this bump turns on:lengthOfAMultibyteStringIsInBytes—"ä".length()is 2 and"aä".length()is 3, so the library counts what the game counts.aPositionInsideACharacterIsNotABoundary— both ends of"ä"are boundaries and the position between its two bytes is not. This is the detection working end to end: it only answers this way if slicing actually produced the partial bytes the library expects, and it is the assertion that would fail on the silent fallback.everyPositionInAnAsciiStringIsABoundary— ascii is unaffected.Worth noting the library reaches the right answer by a different route than it does in game. The engine collapses every half-character slice to one marker string with a constant hash; byte-accurate, a lead byte hashes as itself and misses
PARTIAL_CHAR_HASH, then falls through to the continuation byte table, is not found there, and is correctly reported as a boundary. Both paths agree on the answer, soisCharBoundaryis right either way, butPARTIAL_CHAR_DETECTABLEis true at compiletime while matching a narrower set of slices than in game.Full suite green on the new pin.