Skip to content

Improve the reliability of the tests - #2620

Merged
KevinRansom merged 3 commits into
dotnet:masterfrom
KevinRansom:testreliability
Mar 18, 2017
Merged

Improve the reliability of the tests#2620
KevinRansom merged 3 commits into
dotnet:masterfrom
KevinRansom:testreliability

Conversation

@KevinRansom

@KevinRansomKevinRansom commented Mar 16, 2017

Copy link
Copy Markdown
Contributor

The test failure occurs when calling peverify on the built assembly. The usual readon for that is a missing or incorrect version of fsharp.core.dll. An incorrect version of fharp.core.dll can occur because of one left hanging about in the directory at test build time.

This change ensures that we :

  1. Delete an fsharp.core.dll if it exists prior to the test being run.

  2. At the end of the test deleting it, so it isn't left hanging about.

  3. Slight reformat of some of the test code paths to ensure a consistent cleanup approach.

Kevin

@KevinRansom

KevinRansom commented Mar 16, 2017•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

Tests Passed: Trying again: @dotnet-bot test this please

@KevinRansom

KevinRansom commented Mar 16, 2017

Copy link
Copy Markdown
ContributorAuthor

Tests Passed(Except Ubuntu): Trying again: @dotnet-bot test this please

@KevinRansom

KevinRansom commented Mar 16, 2017

Copy link
Copy Markdown
ContributorAuthor

Tests Passed: Trying again: @dotnet-bot test this please

1 similar comment
@KevinRansom

KevinRansom commented Mar 16, 2017

Copy link
Copy Markdown
ContributorAuthor

Tests Passed: Trying again: @dotnet-bot test this please

@KevinRansom

KevinRansom commented Mar 16, 2017

Copy link
Copy Markdown
ContributorAuthor

Flakey test failed, different test case but same code.

  1. Failed : FSharp-Tests-Core+GeneratedSignatureTests.array-GENERATED_SIGNATURE

@KevinRansom

KevinRansom commented Mar 17, 2017

Copy link
Copy Markdown
ContributorAuthor

Tests Passed: With extra logging, trying again: @dotnet-bot test this please

2 similar comments
@KevinRansom

KevinRansom commented Mar 17, 2017

Copy link
Copy Markdown
ContributorAuthor

Tests Passed: With extra logging, trying again: @dotnet-bot test this please

@KevinRansom

KevinRansom commented Mar 17, 2017

Copy link
Copy Markdown
ContributorAuthor

Tests Passed: With extra logging, trying again: @dotnet-bot test this please

smoothdeveloper
smoothdeveloper reviewed Mar 17, 2017
View reviewed changes
Comment threadtests/fsharp/test-framework.fs Outdated
log "rm %s" path
let p = path |> getfullpath dir
if File.Exists(p) then File.Delete(p)
if File.Exists(p) then (log "rm %s" p) |> ignore; File.Delete(p); else (log "not found: %s p") |> ignore

@smoothdevelopersmoothdeveloperMar 17, 2017

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't checked the code in VS, but are the ignore required?

let's just put this on separate lines instead of ;.

I like the use of IDisposable to make it work in a scope.

@KevinRansomKevinRansomMar 17, 2017

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure ... when I am convinced it is actually .... reliable :-)

@KevinRansom

KevinRansom commented Mar 17, 2017

Copy link
Copy Markdown
ContributorAuthor

Tests Passed: With extra logging, trying again: @dotnet-bot test this please

@KevinRansom

KevinRansom commented Mar 17, 2017

Copy link
Copy Markdown
ContributorAuthor

Tests Passed (apart from Ubuntu): With extra logging, trying again: @dotnet-bot test this please

@KevinRansom

KevinRansom commented Mar 18, 2017

Copy link
Copy Markdown
ContributorAuthor

Tests Passed (apart from FSHARPQA?): With extra logging, trying again: @dotnet-bot test this please

@KevinRansom
KevinRansom merged commit 140440a into dotnet:masterMar 18, 2017
cloudRoutine pushed a commit to cloudRoutine/fsharp that referenced this pull request Mar 19, 2017
@KevinRansom@cloudRoutine
Improve the reliability of the tests (dotnet#2620)
2086125
* Improve the reliability of the tests
* Add extra logging
* Address cr
@KevinRansom
KevinRansom deleted the testreliability branch March 31, 2017 18:54
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@smoothdevelopersmoothdevelopersmoothdeveloper left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@KevinRansom@smoothdeveloper@msftclas