Skip to content

cross-crossgen job, reduce workitem size - #33436

Merged
jashook merged 4 commits into
dotnet:masterfrom
jashook:reduce_crossgen_compairison_size
Mar 11, 2020
Merged

cross-crossgen job, reduce workitem size#33436
jashook merged 4 commits into
dotnet:masterfrom
jashook:reduce_crossgen_compairison_size

Conversation

@jashook

@jashookjashook commented Mar 10, 2020

Copy link
Copy Markdown
Contributor

The work item should be ~1/2 the size of the older workitem. The largest difference will be the bin/* folder will now be ommitted.

Workitem size goes from 750m to 350m, decided to print the elapsed runtime as well into the console log. Which will help with #33175

@Dotnet-GitSync-BotDotnet-GitSync-Bot added the area-Infrastructure-coreclr Only use for closed issues label Mar 10, 2020
@jashookjashook linked an issue Mar 10, 2020 that may be closed by this pull request
@jashook
jashook requested review from echesakov, jkoritzinsky and trylek and removed request for trylekMarch 11, 2020 15:28
@jashookjashook closed this Mar 11, 2020
@jashookjashook reopened this Mar 11, 2020
Comment threadeng/pipelines/coreclr/templates/crossgen-comparison-job.yml
@jashook

Copy link
Copy Markdown
ContributorAuthor

Console looks good to me, run time on device is 7 minutes. Total time is ~14 minutes, so ~7 minute copy and unpack time.

@jashook

Copy link
Copy Markdown
ContributorAuthor

@jashook

Copy link
Copy Markdown
ContributorAuthor

The sooner this is merged the more capacity we can give back to the lab @echesakovMSFT ptal

@echesakovechesakov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Thank you @jashook!

Comment threadeng/pipelines/coreclr/templates/crossgen-comparison-job.yml
@jashook

Copy link
Copy Markdown
ContributorAuthor

The job can put its own results (for native architecture scenario) inside the work item directory. I am talking about results for cross-architecture scenario that are only read during compare step and can be placed inside CorrelationPayloadDirectory .

This makes sense and saves a minute+ Will do this is a follow up change though.

@jashookjashook closed this Mar 11, 2020
@jashookjashook reopened this Mar 11, 2020
@echesakov

Copy link
Copy Markdown
Contributor

Console looks good to me, run time on device is 7 minutes. Total time is ~14 minutes, so ~7 minute copy and unpack time.

@jashook That probably means that there is no much room left for improvement other than parallelizing across multiple Helix machines?

@jashook
jashook merged commit 871e874 into dotnet:masterMar 11, 2020
@jashook

Copy link
Copy Markdown
ContributorAuthor

@jashook That probably means that there is no much room left for improvement other than parallelizing across multiple Helix machines?

I still would like to get #33175 merged. I believe we can drop the execution time down by half. In general to me executing on multiple machines seems like a waste if the time to download is larger than the execution time.

@jashook

Copy link
Copy Markdown
ContributorAuthor

There is more pruning that can happen in the core_root directory. #33438 tracks this.

@jashook
jashook deleted the reduce_crossgen_compairison_size branch March 11, 2020 18:15
@ghostghost locked as resolved and limited conversation to collaborators Dec 10, 2020
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-Infrastructure-coreclrOnly use for closed issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make work item payload smaller for crossgen-comparison job

4 participants

@jashook@echesakov@jkoritzinsky@Dotnet-GitSync-Bot