Unnormalized template literal types

[tpict]

Search Terms

template literal types, normalization, string validation

Suggestion

An unnormalized keyword that can be applied to template literal types to opt out of normalization. Instead of producing a union representing the cross product of each variable in the template, the original declaration is preserved. The template literal type is partially expanded when compared to another type, so the checker can avoid calculating the complete cross product. For example:

type Digit = 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9;
type Zip = unnormalized `${Digit}${Digit}${Digit}${Digit}${Digit}`;

For an assignment where the second digit is invalid, this would reduce the number of comparisons by orders of magnitude and spare the memory overhead of calculating the cross product:

const example: Zip = "9F999"; // checker reports error after only ~11 comparisons

I believe a feature of this sort is warranted as the current template literal syntax has a curious disconnect with the "pattern literal" syntax. The Zip example above is currently unfeasible because it produces a very large union for a relatively simple constraint. In comparison, something like

type Hello = `hello ${string}`;

has no such problem in spite of having infinitely more permutations.

I'm completely open to nomenclature*, syntax & behaviour suggestions, but I feel strongly that this should be an opt-in behaviour and not implicit in any way, as it represents the user selecting one feature set (string validation) as opposed to another (type unions).

*emphasis here–whichever way this syntax is expressed, it should communicate to the user that the result will not be a type union, which is the typical behaviour. Maybe validate would be better.

Use Cases

This would be useful for cases where the type is intended for validation only of a string, and the user doesn't require the type to be compatible with features of union types such as mapped types, distributive types, type narrowing. The examples below show a common topic in relevant PRs: validating that a string matches a 24 bit hex value. I believe that users who want to strictly type strings with millions of permutations are unlikely to need to apply features of union types to them... though I would love to see what 16.7 million grouped if/else statements look like.

Examples

type Hex = 0 | 1 | 2 | ... "F";
type HexColor = unnormalized `#${Hex}${Hex}${Hex}${Hex}${Hex}${Hex};`

// Compare types on assignment
const valid: HexColor = "#FFFFFF";
const invalid: HexColor = "#FFGFFF";

// unnormalized keyword propagates to template literal types:
type Template = `${"red | "blue"} ${HexColor}`;
// equivalent:
type Template = unnormalized `${"red" | "blue"} #${Hex}${Hex}${Hex}${Hex}${Hex}${Hex}`;

// likewise when the union is taken with another template literal:
type Template = `${"red" | "blue"}` | HexColor;
// equivalent:
type Template = unnormalized `${"red" | "blue" | `#${Hex}${Hex}${Hex}${Hex}${Hex}${Hex}`}`;

// For distributive types, comparing an unnormalized template literal always produces the RHS...
type Filter<T, U> = T extends U ? T : never;
type Filtered = Filter<HexColor, "#FFFFFF">; // equivalent to HexColor
// ...unless compared to itself:
type Filtered = Filter<HexColor, HexColor>; // equivalent to never

// Type narrowing behaves as it does with string, number etc:
if (valid === "#FFFFFF") {
 // valid is of type "#FFFFFF" in this context
} else {
 // valid is still of type HexColor in this context
}

// No-op on types where the cross product is of length one:
type Simple = unnormalized `${string}`;
// equivalent:
type Simple = `${string}`;

Pain Points

• What happens if you take the intersection of an unnormalized template literal type with another template literal type?
• How do we communicate the difference between normalized and unnormalized template literal types?
• How do we put an upper bound on the complexity of unnormalized template literal types?

Checklist

My suggestion meets these guidelines:

• This wouldn't be a breaking change in existing TypeScript/JavaScript code
• This wouldn't change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
• (?) This feature would agree with the rest of TypeScript's Design Goals.

[andrewbranch]

It seems like your use cases are already covered by existing type system features, which is usually far preferable to introducing new syntax. Is there anything besides ergonomics that something like this lacks that your suggestion addresses?

[tpict]

I guess I should add that the proposal is specifically for dealing with assignability. Generics definitely get the job done for isolated strings, but things get hairy if you want to use the validation in an interface declaration, array of validated strings etc.

[andrewbranch]

How would you propose determining if one unnormalized template literal type is assignable to another? Or to a union of string literal types? It feels like it would be really easy to create a case that has to do a significant amount, if not all, of the combinatorics that we limit today. If that can happen, doesn’t it defeat much of the purpose of this proposal? A lot of this discussion is taking place (or has already taken place) over at #6579, which I think aims to solve the exact same problem.

[tpict]

Firstly, let me adjust the proposed syntax as I think that will make things clearer:

type Hex = `${unnormalized 0 | 1 | 2 | 3... "F"}`;
type HexColor = `#${Hex}${Hex}${Hex}${Hex}${Hex}${Hex}`;

type Example = `${unnormalized "A" | "B"} ${"C" | "D"}`;
// expands to `${unnormalized "A" | "B"} C` | `${unnormalized "A" | "B"} D` 

The idea is that the type behaves in the same way as pattern literal types, but allows for a union of literals in place of a primitive. I would expect the same assignability rules, except that when two TemplateLiteralTypeSpans are compared, we check for an overlap in their union types.

The goals are definitely aligned with #6579, though my intent is to suggest a minor syntax addition in order to close the perceived gap in behavior between template literal types and pattern literal types.

Say I want to ensure 5 space-separated values in a string:

type Placeholder = string;
type MyString = `${Placeholder} ${Placeholder} ${Placeholder} ${Placeholder} ${Placeholder}`;

This is valid syntax in TS nightly and it does the validation I asked for. I can't distribute over this type, I can't narrow it through control flow. I can take a union with another type literal, I can't take an intersection.

Say I want to limit what any of these five values can be. I change Placeholder to the union of digits 0-9. Suddenly, TS is trying to provide a different set of features: the resultant type is expected to be distribution-compatible, support narrowing and intersections... and it fails to do so, because the cross product of the values is too large. All I wanted was to check a string value, I didn't need the other stuff!

I will look through comments on performance considerations of #6579 because that issue might be enough to shoot this suggestion down. Still, I think it's worth pointing out that the two extant syntaxes discussed have divergent behaviors, one of which results in limitations in service of a feature set that the user might not have wanted anyway.

[andrewbranch]

I would expect the same assignability rules, except that when two TemplateLiteralTypeSpans are compared, we check for an overlap in their union types.

I don’t think that’s sufficient, but notably, it appears that pattern literal types simply aren’t assignable to each other unless they’re identical: https://www.typescriptlang.org/play?ts=4.1.0-dev.20200924#code/C4TwDgpgBAKhDOwCMUC8UAGASA3ogTgJYB2A5gL5QCCGAUKJLAsAExqZS4EkXV20ATCAGMANgEN80URGBQIAD3EBbMDIBcTREgDcgkRKlQZcxSrUQWmuIhZ6zqmW3QOL9pY+guPFu7SA

I can take a union with another type literal, I can't take an intersection.

As an aside, intersections seem to work as expected: https://www.typescriptlang.org/play?ts=4.1.0-dev.20200924#code/C4TwDgpgBAKhDOwCMUC8UAGASA3ogTgJYB2A5gL5QCCGAUKJLAsAExqYBCuBJF1dtAMYB7YoigAPAFxNEKAGSzW7AEQdqK2kA

[andrewbranch]

“Conditional Assignability” discussed in the design meeting today (#40779) would subsume this proposal.

[tpict]

That's a really neat idea. It's syntactically way clearer than either of my suggestions, and I think TS in general would benefit from having an inference mechanism like that for assignments. Thanks for sharing this!