Consolidate Compiler ErrorMessage tests under one suite.

[vzarytovskii]

As part of #7075

[KevinRansom]

Minor tweaks, then it's good to go , thanks

[smoothdeveloper]

In my experience, tests asserting error messages are easier to maintain when they use the baseline files.

Adjusting the test code when they are using CompilerAssert.CompileWithErrors is more tedious than running a diff on .err and .bsl files. It is also true for authoring new tests, and making sure the test covers all the compiler output, not just cherry picking on subset of messages (which can still hide potential regressions).

I've added TypeCheckWithErrorsAndOptionsAgainstBaseLine to make it convenient to use the baseline system under the new testing infrastructure, it may be worth for new tests asserting on error messages to use this system instead:

• tests/FSharp.Compiler.ComponentTests/ErrorMessages/ConfusingTypeName.fs
• tests/FSharp.Compiler.ComponentTests/ErrorMessages/AccessOfTypeAbbreviationTests.fs

fsharp/tests/FSharp.Test.Utilities/CompilerAssert.fs

Lines 449 to 478 in d53716c

	static memberTypeCheckWithErrorsAndOptionsAgainstBaseLine options (sourceDirectory:string)(sourceFile:string)=
	lock gate <|fun()->
	letabsoluteSourceFile= System.IO.Path.Combine(sourceDirectory, sourceFile)
	letparseResults,fileAnswer =
	checker.ParseAndCheckFileInProject(
	sourceFile,
	0,
	SourceText.ofString (File.ReadAllText absoluteSourceFile),
	{ defaultProjectOptions with OtherOptions = Array.append options defaultProjectOptions.OtherOptions; SourceFiles =[|sourceFile|]})
	|> Async.RunSynchronously
	Assert.IsEmpty(parseResults.Errors, sprintf "Parse errors: %A" parseResults.Errors)
	match fileAnswer with
	| FSharpCheckFileAnswer.Aborted _-> Assert.Fail("Type Checker Aborted")
	| FSharpCheckFileAnswer.Succeeded(typeCheckResults)->
	leterrorsExpectedBaseLine=
	letbslFile= Path.ChangeExtension(absoluteSourceFile,"bsl")
	ifnot(File.Exists bslFile)then
	// new test likely initialized, create empty baseline file
	File.WriteAllText(bslFile,"")
	File.ReadAllText(Path.ChangeExtension(absoluteSourceFile,"bsl"))
	leterrorsActual=
	typeCheckResults.Errors
	|> Array.map (sprintf "%A")
	|> String.concat "\n"
	File.WriteAllText(Path.ChangeExtension(absoluteSourceFile,"err"), errorsActual)
	Assert.AreEqual(errorsExpectedBaseLine.Replace("\r\n","\n"), errorsActual.Replace("\r\n","\n"))

This would also lend well into something where we cover all actual error messages the compiler can produce (which isn't the case, we've seen a regression on match on DU cases), which can also be useful for future work on compiler documentation (description of each error message with samples and explanation to fix).

[baronfel]

regarding the last point about compiler error message documentation I 100% took samples and phrasing from the baseline tests because they were such an easy to read resource.

[vzarytovskii]

@smoothdeveloper

In my experience, tests asserting error messages are easier to maintain when they use the baseline files.

Adjusting the test code when they are using CompilerAssert.CompileWithErrors is more tedious than running a diff on .err and .bsl files. It is also true for authoring new tests, and making sure the test covers all the compiler output, not just cherry picking on subset of messages (which can still hide potential regressions).

I've added TypeCheckWithErrorsAndOptionsAgainstBaseLine to make it convenient to use the baseline system under the new testing infrastructure, it may be worth for new tests asserting on error messages to use this system instead:

• tests/FSharp.Compiler.ComponentTests/ErrorMessages/ConfusingTypeName.fs
• tests/FSharp.Compiler.ComponentTests/ErrorMessages/AccessOfTypeAbbreviationTests.fs

It makes sense to me, I wouldn't do it as a part of this PR thought, this one is just moving tests from older suites to a new one.
I will make it as a separate though.

[smoothdeveloper]

@vzarytovskii sorry I didn't catch the tests I mentioned were just moved.

What I say makes most sense for new tests around error / warnings, or at the time change to the actual message makes the test red.