Add a test suite under Tests - #103
Conversation
Nine console test programs plus a script that builds and runs them with every Delphi installation it finds, for Win32 and Win64. Nothing in FastMM5.pas changes; this is purely additive. Each test exits with 0 when all of its checks passed and with the number of failures otherwise, so no test framework is needed and the runner (or a CI step) only has to look at the exit code. RunTests.ps1 exits with the number of failed runs. Covered: the block size classes including reallocations across their boundaries, debug mode basics, usage accounting per size class (both leaks and unbounded address space growth), the mode transition contract from pleriche#85, double free handling from pleriche#73, the corruption scan from pleriche#102 in both directions (detection must work, false positives must not happen), corrupted size fields in the debug header, and multithreaded stress with cross thread frees. Verified with Delphi 10 Seattle and Delphi 13.1, Win32 and Win64: 36 of 36 runs pass.
pleriche
commented
Jul 25, 2026
Looks good. One thing I would prefer though is if the list of compilers to use was external, e.g. in CompilerPaths.txt (added to .gitignore) so the script doesn't need to be edited. If there's no CompilerPaths.txt then it would be really awesome if the script could detect and use the latest compiler automatically. We use MSBuild for our build process with the latest installed compiler. It's a crude batch file, but it does the job: |
RunTests.ps1 no longer carries a table of paths that has to be edited. It reads the installed versions from the registry (HKCU and HKLM, including the 32 bit view) and from %EmbarcaderoRoot% if that is set, and uses the newest one by default. -AllCompilers runs all of them, -ListCompilers just reports what was found. Where the list has to be pinned - other install locations, a specific set of versions, or a machine where the registry cannot be read - a CompilerPaths.txt next to the script takes over, with one installation root per line and an optional "Name = Path". All of its entries are used, so it doubles as "run against exactly these". It is in .gitignore, so a local setup never shows up as a change. The reported name of an installation is its IDE version, and next to it the compiler version from the registry, which is the number the CompilerVersion guards in the sources refer to. Verified on Win32 and Win64 with Delphi 10 Seattle, 11.3 Alexandria, 12.3 Athens and 13.1 - 72 of 72 runs pass - and with the automatic path, CompilerPaths.txt, -Only, -AllCompilers and a deliberately broken entry.
TetzkatLipHoka
commented
Jul 25, 2026
Done - pushed, and the hard coded table is gone entirely. Automatic by default. The installed versions are read from the registry ( The name is the IDE version and next to it the compiler version, taken from CompilerPaths.txt takes over when it exists, with one installation root per line and an optional Environment variables are expanded, One deliberate limitation worth mentioning: only the Embarcadero era versions are found automatically, because those are the ones registered under Verified on Win32 and Win64 with Delphi 10 Seattle, 11.3 Alexandria, 12.3 Athens and 13.1: 72 of 72 runs pass. That is a good deal broader than the two versions the first push claimed, and it came for free once selecting compilers stopped being a manual step. I also exercised the paths themselves: automatic detection, |
Nothing in the suite links an .obj, so this changes nothing for a current Delphi, but it lets the same command line serve a compiler whose RTL declares external routines from .obj files - which is what makes the script usable unchanged in a fork that also targets older versions through CompilerPaths.txt.
Versions before the platform aware RTL layout keep their DCUs directly in "lib" rather than in "lib\win32\release". Falling back to it means the same script serves those too when they are listed in CompilerPaths.txt, and it is inert for every version that has the per platform directories.
TetzkatLipHoka
commented
Jul 25, 2026
Two small follow-up commits, both of which I would understand you reverting - they are there because they let one script serve older compilers listed in
With those two, and nothing else, the suite builds and passes under Delphi 7 as well, which is a useful canary for the If you would rather the script stayed strictly XE3+, say so and I will drop both; the automatic detection and |
pleriche
commented
Jul 25, 2026
Great work, thanks! I've merged the pull request. |
The correctness test predates the suite that came in with pleriche#103, so it had its own scaffolding and its own name. It is now FastMM5Test_FillPattern: it uses FastMM_TestUtils, reports one check per size rather than one per byte position, and names the first missed offset when a size fails. RunTests.ps1 runs it with everything else - all 34,949 byte positions take about a fifth of a second, so it needs no shortened quick variant. fillbench.dpr and Measure.ps1 become FastMM5Bench_FillPattern and MeasureFillPattern.ps1. They stay outside the suite on purpose, since they report a time rather than a pass or a fail, and the Bench prefix says so. The harness no longer hard codes the directory the two builds live in. README-sse2.md records janrysavy's independent reproduction on Ryzen 9 7950X and Core i7-8750H, and his finding that a straightforward Win64 integration costs the small sizes 3.58% through register saves and code movement alone, even though they never execute the vector path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nine console test programs plus a script that builds and runs them with every Delphi installation it finds, for Win32 and Win64. Nothing in FastMM5.pas changes - this is purely additive, and if you never run it, it costs nothing.
The motivation is #102: the corruption scan silently stopped seeing small debug blocks, and it stayed unnoticed because there was nothing that would have run the same check across all three size classes after a change.
FastMM5Test_ScanCoverageis exactly that check, and it fails on the affected commits and passes on either side of them.How it works
No test framework. Each program is a plain console application that prints one line per check and exits with 0 when everything passed, otherwise with the number of failed checks.
RunTests.ps1builds and runs them all and exits with the number of failed runs, so it drops straight into a CI step:The
$Compilerstable at the top of the script is the only thing that needs adjusting for a different machine.What is covered
FastMM5Test_DebugModeFastMM5Test_SizeClassesFastMM5Test_UsagePerSizeClassFastMM5Test_ModeTransitionFastMM5Test_DoubleFreeFastMM5Test_ScanCoverageFastMM5Test_ScanRaceFastMM5Test_ScanHeaderBoundsUserSize/StackTraceEntryCount, i.e. the fields that decide where the scan reads; each case must give a clean report rather than an A/V.FastMM5Test_MultiThreadStressFastMM_TestUtils.pasholds the assertions and the exit code convention. It also clearsFastMM_MessageBoxEventsandFastMM_LogToFileEventsfor the duration of a run, since several tests corrupt blocks on purpose and a modal dialog would hang an unattended run.Testing
36 of 36 runs pass: Delphi 10 Seattle and Delphi 13.1, each Win32 and Win64.
Two notes
The
Tests/README.mdrecords two things that cost me time and would otherwise be rediscovered by whoever writes the next test: corrupting a freed small block is not observable (raising the report allocates the exception object, which is handed exactly that block, so the process dies before any handler runs - use medium or large blocks there), and a corruption test that happens to use a large block proves nothing about the small block path, which is how #102 stayed hidden.The sources carry
{$if CompilerVersion >= ...}guards in a few places so the same files also build on older compilers in a fork. They are inert on XE3 and later; happy to strip them if you would rather not carry them.