Caches failed service probes

[NickSdot]

Extracted from #22917.

Prevents unavailable optional test services from slowing the suite through repeated connection timeouts. PDO and SNMP availability probes cache the first failure for the duration of one test run, so later SKIPIF checks return immediately.

Only failures are cached; successful probes and test connections are unchanged. PDOs one second timeout applies only to the default availability probe. The caches are private, run-scoped and removed afterwards. A service becoming available mid run is detected on the next test-suite run.

Result

Values in seconds; measured on macOS.

mode	master	this PR	saved	speedup
-j1	285.246	8.991	276.255	31,72×
-j10	221.144	4.560	216.584	48,49×

Reproduction:

sapi/cli/php run-tests.php -q -j1 ext/pdo ext/pdo_dblib ext/snmp
sapi/cli/php run-tests.php -q -j10 ext/pdo ext/pdo_dblib ext/snmp

Result (after #22944 (comment) )

Added caching for ext/ldap, ext/pgsql, ext/mysqli, ext/pdo_mysql, ext/odbc.

mode	master	this PR	saved	speedup
-j1	313.740	35.177	278.563	8,91x
-j10	241.474	13.781	227.693	17.52x

Reproduction:

sapi/cli/php run-tests.php -q -j1 ext/pdo ext/pdo_dblib ext/snmp ext/ldap ext/pgsql ext/mysqli ext/pdo_mysql ext/odbc
sapi/cli/php run-tests.php -q -j10 ext/pdo ext/pdo_dblib ext/snmp ext/ldap ext/pgsql ext/mysqli ext/pdo_mysql ext/odbc

[TimWolla]

Without looking too deeply into it: There is already a SkipCache in the test runner, why doesn't it work?

[NickSdot]

Without looking too deeply into it: There is already a SkipCache in the test runner, why doesn't it work?

SkipCache is not actually related. This here actively caches expensive failures across parallel workers so that same following calls do not run into the same timeouts over and over again.

[arnaud-lb]

I believe it has the same purpose, but it's less efficient since it doesn't share its state between workers. However it's more general and avoids introducing complexity into individual tests, so if we could improve it, this would be better.

We could update SkipCache so that it shares its state between workers. E.g. store results in individual files /tmp/run-tests-skipif-{pid}/{key}/{code hash} instead of this->skips[$key][$code].

[NickSdot]

Went back and forth on the comments... Maybe my brain is not braining, but I still don't see how SkipCache is related or could do what is proposed here even if we share it between workers.

SkipCache caches SKIPIF by its code/key. What is proposed here caches failed service probes between different SKIPIF. Changing SkipCache to file based would share some same SKIPIF between workers, but different SKIPIF running the same probes would still not share anything!? The -j1 result proves that, though. All tests already use the same in-memory SkipCache yet we get a huge speedup. Sharing between workers cannot produce that gain.

Agreed on the in-test complexity. But we will not fully get around having something that tells when different tests run the same probe? I pushed an update that makes this more general (ext/ldap, ext/pgsql, ext/mysqli, ext/pdo_mysql, ext/odbc now included), keeps complexity in helpers, reduced the diff from 42 to 14 files, and is now 31,72x and 48,49x instead of 12.66× and 17.34× (on the initial reproduction).

---

Asides:

• sharing SkipCache between workers could be useful on its own. Something we could explore in a separate PR.
• the CI label can be removed; it was added when I excluded tests for debugging.

[arnaud-lb]

I agree, you are right!

I did a first review, will continue after your feedback.

[arnaud-lb]

Looks good to me. Only two suggestion comments: let me know if you want to apply them. Otherwise I will merge.

[NickSdot]

@arnaud-lb I consider this done. Would appreciate if you would prioritise #22995 before any others to give me a faster CI feedback loop.

[arnaud-lb]

Thank you!