Simplified compiler test framework by vzarytovskii · Pull Request #9721 · dotnet/fsharp

vzarytovskii · 2020-07-20T16:17:52Z

First round of implementation of #9720

There are still a bunch of stuff to implement, but first, we need to have a new base for it.

abelbraaksma · 2020-07-20T17:57:10Z

Not really a review here, just a suggestion:

May I suggest we improve withWarning to something like withWarning (Warning 213, (Line 2, Col 1, Line 2, Col 6)), or similarly, withWarning (Warning 213, Pos.From(2, 1), Pos.To(2, 6))? I find these notoriously hard to read when working with tests (though everything here is already a biiig improvement!).

vzarytovskii · 2020-07-20T18:24:03Z

Not really a review here, just a suggestion:
May I suggest we improve withWarning to something like withWarning (Warning 213, (Line 2, Col 1, Line 2, Col 6)), or similarly, withWarning (Warning 213, Pos.From(2, 1), Pos.To(2, 6))? I find these notoriously hard to read when working with tests (though everything here is already a biiig improvement!).

Yeah, that's the plan - I will add a verbose versions (looks pretty much like you suggested), as well as fewer small functions which are easier to compose (like withErrorCode 123 |> onPosition ... |> withMessage ... etc), so you have freedom to how to write and how vebose you want it :)

Also - I would love to chat about the more complex testing scenarios (like one you've encountered in ExprTests).

abelbraaksma · 2020-07-20T18:32:53Z

Also - I would love to chat about the more complex testing scenarios (like one you've encountered in ExprTests).

Sure, any time, you can ping me on Slack (just remember I'm on European time ;) ) and we can set up a call.

vzarytovskii · 2020-07-20T18:35:01Z

Also - I would love to chat about the more complex testing scenarios (like one you've encountered in ExprTests).
Sure, any time, you can ping me on Slack (just remember I'm on European time ;) ) and we can set up a call.

Yeah, I'm in Prague myself :)
Will ping you there some time during the week then.

abelbraaksma · 2020-07-20T18:42:41Z

Oh I love Prague! I come there at least once a year to visit friends and for a conf. I know it quite well, and of course the 🍺 🍻 🍺

…f netcoreapp31)

cartermp · 2020-07-22T15:54:25Z


+ // NOTE: This function will not clean up all the compiled projects after itself.
+ // The reason behind is so we can compose verification of test runs easier.
+ // TODO: We must not rely on the filesystem when compiling


How will that be resolved?

I have a separate issue for tackling that.
I'm thinking on the suite teardown level we can clean up all projects/sources we've created.

Co-authored-by: Phillip Carter <pcarter@fastmail.com>

KevinRansom

Looks, great. I have suggested a few things to consider in the next update. But they shouldn't hold up merging this PR.

Nice work

Kevin

KevinRansom · 2020-07-23T00:46:48Z

+ Warnings = []
+ Errors = [] }
+
+ let private typecheckFSharpSource (fsSource: FSharpCompilationSource) : CompilationResult =


Is it possible we would like have tests with multiple source files?

Yep, it's tracked in #9746

…ions + fixed options composition.

* [WIP] Simplified compiler API for testing * [WIP] Support of references, for F#, C# projects. Added more asserts. * Added more examples + ignoreWarnings support * Added typecheck + baseline test helpers. * Added running functionality * Moved a bunch of compiler tests to a new suite * Changed error message * [WIP] Added CSharp compilation + referencing * Added some simple interop tests * [WIP] Process MetadataReference recursively * [WIP] Added netcorapp31 support for tests * [WIP] Termporary revert to using netcoreapp30 in the tests (instead of netcoreapp31) * Added types to errors/warnings as oppose to just having tuples * Fixed tests to fit newer API * Revert libraries upgrade * Revert change netcoreapp31 -> netcoreapp30 * Update tests/FSharp.Test.Utilities/Compiler.fs Co-authored-by: Phillip Carter <pcarter@fastmail.com> * Addressed some PR comments * Fixed missing styling inconsistensies * Addressed feedback: recursive module instead of forward type declarations + fixed options composition. * Changed Option.isSome -> defaultArg Co-authored-by: Phillip Carter <pcarter@fastmail.com>

vzarytovskii added 11 commits July 7, 2020 19:33

[WIP] Simplified compiler API for testing
1ca38f5

[WIP] Support of references, for F#, C# projects. Added more asserts.
b66d8ae

Added more examples + ignoreWarnings support
b34ca95

Added typecheck + baseline test helpers.
9e4561e

Added running functionality
e1f6ee9

Moved a bunch of compiler tests to a new suite
54c313f

Changed error message
7a6fe9b

[WIP] Added CSharp compilation + referencing
dcf30fa

Added some simple interop tests
285d6e8

[WIP] Process MetadataReference recursively
ed00d7e

[WIP] Added netcorapp31 support for tests
a91288c

vzarytovskii changed the title ~~[WIP] Simplified compiler test framework~~[WIP] [Don't review] Simplified compiler test frameworkJul 20, 2020

vzarytovskii added 5 commits July 21, 2020 18:23

[WIP] Termporary revert to using netcoreapp30 in the tests (instead o…
d8f4542
…f netcoreapp31)

Added types to errors/warnings as oppose to just having tuples
8bf7ded

Fixed tests to fit newer API
8799253

Revert libraries upgrade
2f36dc4

Revert change netcoreapp31 -> netcoreapp30
56e8c92

vzarytovskii changed the title ~~[WIP] [Don't review] Simplified compiler test framework~~Simplified compiler test frameworkJul 22, 2020

vzarytovskii marked this pull request as ready for review July 22, 2020 15:36

jonsequitur reviewed Jul 22, 2020
View reviewed changes

Comment threadtests/FSharp.Compiler.ComponentTests/Interop/SimpleInteropTests.fs Outdated

cartermp reviewed Jul 22, 2020
View reviewed changes

vzarytovskiiand others added 3 commits July 22, 2020 17:57

Update tests/FSharp.Test.Utilities/Compiler.fs
3e10cdb
Co-authored-by: Phillip Carter <pcarter@fastmail.com>

Addressed some PR comments
957dd77

Fixed missing styling inconsistensies
aa70025

KevinRansom approved these changes Jul 23, 2020
View reviewed changes

KevinRansom closed this Jul 23, 2020

KevinRansom reopened this Jul 23, 2020

vzarytovskii added 2 commits July 23, 2020 10:19

Addressed feedback: recursive module instead of forward type declarat…
a1e4293
…ions + fixed options composition.

Changed Option.isSome -> defaultArg
cb9e1cb

vzarytovskii merged commit 3f011ec into dotnet:masterJul 23, 2020

Uh oh!

Conversation

vzarytovskii commented Jul 20, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

abelbraaksma commented Jul 20, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vzarytovskii commented Jul 20, 2020

Uh oh!

abelbraaksma commented Jul 20, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vzarytovskii commented Jul 20, 2020

Uh oh!

abelbraaksma commented Jul 20, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cartermpJul 22, 2020

Choose a reason for hiding this comment

Uh oh!

vzarytovskiiJul 22, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KevinRansom left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

KevinRansomJul 23, 2020

Choose a reason for hiding this comment

Uh oh!

vzarytovskiiJul 23, 2020

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

vzarytovskii commented Jul 20, 2020•
edited
Loading

abelbraaksma commented Jul 20, 2020•
edited
Loading

abelbraaksma commented Jul 20, 2020•
edited
Loading

abelbraaksma commented Jul 20, 2020•
edited
Loading

vzarytovskiiJul 22, 2020•
edited
Loading