[PoC] Limited Abstract Generics

[Girgias]

This is a proof of concept for a limited abstract generic types feature set, as those can be, and are, resolved at compile/linking time.

Implementation

Depends on:

• Zend: Use pointer to zend_type for variance checks #18257 Not needed it turns out
• Zend: Add MUTABLE zend_type foreach macros and const qualifiers #18256
• Zend: Inherit interfaces early #18622

The implementation is relatively dumb, and partially based on arnaud-lb#4 for parser/AST/compile shenanigans.

The generic types (name and constraint) are stored on the CE in a new generic_parameters list field.

The bound types are also stored on the CE as a HashTable:

lc_interface_name: HashTable<int|string, zend_type>
int: positionally bound type
string: positional bound type associated to its template name

This means that this implementation cannot be extended to support concrete generics (i.e. generics on a concrete instantiable class), as those need to be tied to the instance of the CE, not the CE itself.

The generic types must be:

• invariant
• Not be part of a composite type
• Only on interfaces
• No support for type declarations

It is possible to extend an interface with generic types, so that a sub-interface can reuse the same generic parameter.
If one of the generic parameters of the interface being extended has a type constraint, this type constraint must be repeated on the child interface. As the type constraints for interfaces are checked when extending.

ToDos

• opcache support (currently leaks memory)
• Fix some type binding issues for implicit and explicitly implemented interfaces
• Properly support this for internal interfaces

Benefits

Although the lack of type declarations can make this unsound, in that a generic type T of an interface I<T : C> is no better than the type constraint C (which is mixed) by default.
It does "solve" the primary need of wanting never to be useable for parameter types, being able to specify the actual type, and thus have engine type checking, on the concrete implementation. As this is currently prevented by LSP variance rules.

One use case would be to use generic types K, V on the new interfaces of my Container/Offset RFC instead of mixed.

Future scopes

• Lifting restricting on generic parameters needing to be used standalone (i.e. make T|null valid)
• Add support for abstract class
• Add support for traits
• Add type declarations
• Allow variance of generic parameters
• Optional generic parameters

[withinboredom]

You are almost to full generics here :) ...

Invariant is a good default (and usually the default for any generics). You only need a proper constraint resolver (this is partly why I was working on type trees in #18189, which would let you resolve covariant/contravariant constraints very easily). I'm working on that for zend_type the last couple of weeks -- which is far more complex. Potentially, between the two of us, we could enable something powerful here.

I don't know if your intent is to get to full generics from here, but this is pretty similar to a couple of other experiments I've done.

[Girgias]

I was not really planning on going full generics. As the main issue with them from my understanding is determining the type to be bound to at runtime in a way that is not terrible for ergonomics and performance. Maybe @arnaud-lb could shed a bit more light.

I didn't even think of a constraint resolver, but a few other people mentioned it and have an idea how to implement it, so will do that soon.

[arnaud-lb]

Interesting!

In term of functionalities that has some similarities with @nikic's "purely abstract" generics [1] as well as @derickr Collections [2], in that we can not parameterize types at the point of use, but types can extend/implement parameterized types.

One implication is that we can not use a type-with-assoc-types in type declarations, because this is not allowed:

functionf(I<T: Foo> $i) {}

and this is unsound if I has assoc types:

functiong(I$i) {}

Therefore, currently this seems most useful in traits and abstract classes? Could you expand on the relation with the Container/Offset RFC?

Allowing assoc types in traits or abstract classes seems possible, but this increases complexity to a level comparable to [1], as assoc types on properties, method signatures, or method bodies would be handled at runtime (at least on abstract classes).

Allowing I<T: Foo> in type decls seems possible. Some things to consider would be how T is allowed to vary, and the fact that this increases the complexity of sub-type checking against a type-with-assoc-types. I think that variance should be specified in the type statement (and default to invariant) rather than determined implicitly, to avoid unintended variance changes. E.g. T is invariant here:

interface I {
type T; // invariant by defaultfunctionfoo(T): T;
}

here as well:

interface J {
type T; // invariant by default (covariant would be allowed)functionfoo(): T;
}

but covariant here:

interface K {
type out T; // covariantfunctionfoo(): T;
}

I was not really planning on going full generics. As the main issue with them from my understanding is determining the type to be bound to at runtime in a way that is not terrible for ergonomics and performance. Maybe @arnaud-lb could shed a bit more light.

I confirm. There are some difficulties [3]:

• Type inference is hard to achieve
• Big-O complexity of type checking can be quadratic or worse when checking compound types against compound types

[1] PHPGenerics/php-generics-rfc#45
[2] https://wiki.php.net/rfc/collections
[3] arnaud-lb#4

[Girgias]

Could you explain the unsoundness argument a bit more? I am struggling to see it.

This experiment was mainly prompted about the discussion of allowing never as a parameter type (#18016) where the main motivation seems to be able to define an interface:

<?phpinterface I {
publicfunctionset(never$offset, never$value);
publicfunctionget(never$offset): mixed;
}

With the intention that any implementation of said interface would specialize the types to be "sensible" e.g.

<?phpclass ListOfAnimals implements I {
publicfunctionset(int$offset, Animal$value);
publicfunctionget(int$offset): Animal;
}

The proposal to allow never as parameter types seems to be hitting into the same unsoundness issue you are describing (i.e. we cannot know statically if the whole call chain is valid).

However, an associated type, even without being able to specify it in a type declaration, gives you at least the small guarantee that different methods use the same types:

interface I {
type K : int|string
type V : mixed;
publicfunctionset(K$offset, V$value);
publicfunctionget(K$offset): V;
}
class ListOfAnimals implements I {
publicfunctionset(int$offset, Animal$value);
publicfunctionget(int$offset): Animal;
}

This is basically also how it ties in to the Container/Offset RFC, because instead of needing to use mixed everywhere:

<?phpinterface DimensionReadable
{
publicfunctionoffsetGet(mixed$offset): mixed;
publicfunctionoffsetExists(mixed$offset): bool;
}
interface DimensionFetchable extends DimensionReadable
{
publicfunction &offsetFetch(mixed$offset): mixed;
}
interface DimensionWritable
{
publicfunctionoffsetSet(mixed$offset, mixed$value): void;
}
interface DimensionUnsetable
{
publicfunctionoffsetUnset(mixed$offset): void;
}
interface Appendable
{
publicfunctionappend(mixed$value): void;
}
interface FetchAppendable extends Appendable
{
publicfunction &fetchAppend(): mixed;
}

We could use a pair of associated type:

<?phpinterface DimensionReadable
{
type K;
type V;
publicfunctionoffsetGet(K$offset): V;
publicfunctionoffsetExists(K$offset): bool;
}
interface DimensionFetchable extends DimensionReadable
{
publicfunction &offsetFetch(K$offset): V;
}
interface DimensionWritable
{
type K;
type V;
publicfunctionoffsetSet(K$offset, V$value): void;
}
interface DimensionUnsetable
{
type K;
publicfunctionoffsetUnset(K$offset): void;
}
interface Appendable
{
type V;
publicfunctionappend(V$value): void;
}
interface FetchAppendable extends Appendable
{
publicfunction &fetchAppend(): V;
}

Where ArrayAccess, to keep BC, resolves both K and V to mixed.

My main concern with supporting traits, is that I would be hitting the same issue, that I haven't resolved yet, when trying to resolve self to the class name it is implemented at compile time. As doing so would, I think, remove a lot of type checking complexity as everything would just be reusing the same typing infrastructure.

I will also say that for this feature to be fully fleshed it does need to support property hooks, which might or might not be a challenge.

[arnaud-lb]

Could you explain the unsoundness argument a bit more? I am struggling to see it.

What I meant is that calling any method in I would be unsound, but now I get that's it's intended / it's the purpose.

Thank you for the explanations.

[withinboredom]

Type inference is hard to achieve

I have some ideas here. Here's one I'd probably tackle first as a proof-of-concept:

1. tack a bool on zvals: isAffirmedType or something.
2. on type checking: if the type of the zval matches the declared type, set isAffirmedType to true.

If the type is affirmed, then the type in the zval can be inferred, otherwise, it is an error.

functionfoo(SomeInterface$a) {
newGenericArray($a); // type error: type cannot be inferred from SomeConcreteType
}
foo(newSomeConcreteType());

It's not ideal, but it is pretty straightforward to reason about as a user.

Big-O complexity of type checking can be quadratic or worse when checking compound types against compound types

I'm working on this, but I lack a lot of practical knowledge of the engine -- but getting there, slowly but surely. Feel free to beat me to it.

[arnaud-lb]

@withinboredom this is an interesting idea as it makes inference works when the runtime and static types match. Unfortunately I think it’s unsound because calling foo() with a type accepted by its signature is an error.

[morrisonlevi]

When I saw my example in here, I got excited that you added basic union support. Nope! 😆

One day 🙏🏻

[Girgias]

As I said to Bob I really want to keep it as small as possible as it's already hurting my brain a bit! But this should be a rather easy limitation to lift :)