Maybe it is deterministic now?

[nojaf]

I'm just curious at this point.

[vzarytovskii]

Compiler build is deterministic, but it problem is still there, right? Pdb may have differences when graph checking is enabled?

[nojaf]

Compiler build is deterministic, but it problem is still there, right? Pdb may have differences when graph checking is enabled.

I'm not quite sure. I was able to generate the same pdb about 50 times in a row on my machine.
For various projects (FSharp.Core, FCS, Fantomas.Core, my reproduction of #14703 project GraphSample)
These all worked for me: mvid is stable, file-hash of binary and pdb file are stable.

Used script:

#r "nuget: CliWrap, 3.6.0"
#r "nuget: System.Reflection.Metadata"
#r "System.Security.Cryptography"openSystemopenSystem.IOopenSystem.Reflection.MetadataopenSystem.Reflection.PortableExecutableletgetMvid refDll =use embeddedReader =new PEReader(File.OpenRead refDll)letsourceReader= embeddedReader.GetMetadataReader()letloc= sourceReader.GetModuleDefinition().Mvid
letmvid= sourceReader.GetGuid(loc)
mvid
letgetFileHash filename =use sha256 = System.Security.Cryptography.SHA256.Create()use stream = File.OpenRead(filename)lethash= sha256.ComputeHash(stream)
BitConverter.ToString(hash).Replace("-","")openCliWrapletargsFile=// FileInfo(@"C:\Users\nojaf\Projects\main-fantomas\src\Fantomas.Core\Fantomas.Core.args.txt")// FileInfo(@"C:\Users\nojaf\Projects\fsharp\src\Compiler\FSharp.Compiler.Service.args.txt")// FileInfo(@"C:\Users\nojaf\Projects\fsharp\src\FSharp.Core\FSharp.Core.args.txt")
FileInfo(@"C:\Users\nojaf\Projects\graph-sample\GraphSample.args.txt")lettotal=50typeCompilationResultInfo={
Mvid:Guid
BinaryFileHash:string
PdbFileHash:string option}overridex.ToString()=letmvid= x.Mvid.ToString("N")letpdb=match x.PdbFileHash with| None ->""| Some pdb -> $", pdb: {pdb}"
$"mvid: {mvid}, binary: {x.BinaryFileHash}{pdb}"[<RequireQualifiedAccess>]typeCompilationResult<'TResultwhen'TResult:equality>=| None
| Stable ofresult:'TResult*times:int| Unstable ofinitial:'TResult*times:int*variant:'TResultletoldFiles(argsFile:FileInfo)=letoutputFile=
File.ReadAllLines(argsFile.FullName)|> Array.tryPick (fun line ->ifnot(line.StartsWith("-o:"))then
None
elseletobjPath= line.Replace("-o:","")letobjPath=if File.Exists objPath then
objPath
else
Path.Combine(argsFile.Directory.FullName, objPath)
FileInfo(objPath)|> Some
)match outputFile with| None -> Seq.empty
| Some outFile ->seq{yield! Directory.EnumerateFiles(outFile.Directory.FullName,"*.dll")yield! Directory.EnumerateFiles(outFile.Directory.FullName,"*.pdb")}for file in oldFiles argsFile do
File.Delete(file)letruns=(CompilationResult.None,[1..total ])||> List.fold (fun(prevResult: CompilationResult<CompilationResultInfo>)idx->match prevResult with| CompilationResult.Unstable _-> prevResult
|_->tryletargs= $"@{argsFile.Name}"
Cli
.Wrap(@"C:\Users\nojaf\Projects\safesparrow-fsharp\artifacts\bin\fsc\Release\net7.0\win-x64\publish\fsc.exe")
.WithWorkingDirectory(argsFile.DirectoryName)
.WithArguments($"\"{args}\" --test:GraphBasedChecking --test:DumpCheckingGraph --debug:portable")// --debug-
.ExecuteAsync()
.Task.Wait()letbinaryPath=letbinaryPath= File.ReadAllLines(argsFile.FullName).[0].Replace("-o:","")if File.Exists binaryPath then
binaryPath
else
Path.Combine(argsFile.DirectoryName, binaryPath)letbinary= FileInfo(binaryPath)letbinaryHash= getFileHash binary.FullName
letmvid= getMvid binary.FullName
letpdbFile:FileInfo option =letpath= Path.ChangeExtension(binary.FullName,".pdb")ifnot(File.Exists path)then
None
else
Some(FileInfo(path))letresult={
Mvid = mvid
BinaryFileHash = binaryHash
PdbFileHash = pdbFile |> Option.map (fun fi -> getFileHash fi.FullName)}
printfn $"Compiled %02i{idx}, write date %A{binary.LastWriteTime}, result: {result}"letrenameToRun(file:FileInfo)=letdifferentPath=
Path.Combine(
file.Directory.FullName,
$"{Path.GetFileNameWithoutExtension(file.Name)}-{idx}{file.Extension}")
File.Move(file.FullName, differentPath)
renameToRun binary
Option.iter renameToRun pdbFile
match prevResult with| CompilationResult.Unstable _| CompilationResult.None _-> CompilationResult.Stable(result,1)| CompilationResult.Stable(prevResult, times)->if prevResult <> result then
CompilationResult.Unstable(prevResult, times, result)else
CompilationResult.Stable(prevResult, times +1)with ex ->
printfn "%s" ex.Message
prevResult
)
printfn "%A" runs

At this point, I really really want someone to point out to me where there still is an actual problem.
For all we know is the problem we saw pre #14703 is just related to the usage of Proto or some other weird compiler-only factor.

[nojaf]

Determinism failed for the following binaries:
FSharp.Compiler.Service.dll (old hash: 5DE8920DD1BD7BD52C169EAC0AFD8C10; new hash: 040C66612A969FBD913FD36A05F2B3A1)
Archiving failure information
Please send D:\a_work\1\s\artifacts\log\Debug\determinism.zip to compiler team for analysis

Ok, that settles it 🙃

[vzarytovskii]

Determinism failed for the following binaries:
FSharp.Compiler.Service.dll (old hash: 5DE8920DD1BD7BD52C169EAC0AFD8C10; new hash: 040C66612A969FBD913FD36A05F2B3A1)
Archiving failure information
Please send D:\a_work\1\s\artifacts\log\Debug\determinism.zip to compiler team for analysis

Ok, that settles it 🙃

Oh...Let me download it and dig around pdbs. I suspect it may be as simple as ordering of locals for document.

[vzarytovskii]

Okay, in the specific run it was failing, the difference is in optimization data for sure, havent' looked at PDB yet

[nojaf]

Can that optimization data be written to disk as a separate file?

[vzarytovskii]

Can that optimization data be written to disk as a separate file?

You can save it from ILSpy

[vzarytovskii]

Ok, my bad, it was not an optimization data, but signature data, again 1 byte difference (with offset of one AB -> AC).

[vzarytovskii]

DLLs themselves differ in MVID and a bunch more (PDB perhaps)

[vzarytovskii]

Alright, PDBs have a handful of changes, not sure which parts just yet:

[nojaf]

Ok, when compiling for Release, I see inconsistent results for FSharp.Compiler.Service locally.
Sometimes the mvid, occasionally only the file hash of the binary.

[nojaf]

Superseded by #15003