Skip to content

Allow --noCheck to be commandLine option - #58839

Merged
Sheetal Nandi (sheetalkamat) merged 3 commits into
mainfrom
noCheck
Jun 14, 2024
Merged

Allow --noCheck to be commandLine option#58839
Sheetal Nandi (sheetalkamat) merged 3 commits into
mainfrom
noCheck

Conversation

@sheetalkamat

Copy link
Copy Markdown
Member

Works on top of #58626 and #58838

Adds checkPending to buildInfo so as to handle if there was noCheck pending. This does not add "noCheck" as semantic affecting diagnostics because builder can handle that and save the diagnostics between runs with and without "noCheck"

Comment threadsrc/compiler/commandLineParser.ts
@jakebailey

Copy link
Copy Markdown
Member

This will close #29651, right?

@sheetalkamat

Copy link
Copy Markdown
MemberAuthor

Yes. We would report ssyntax errors though while emitting.

@fatcerberus

Copy link
Copy Markdown

ssyntax errors

Oh no, I hadn't realized some of the TypeScript devs were snakes

@sheetalkamat

Copy link
Copy Markdown
MemberAuthor

TypeScript Bot (@typescript-bot) test it

@typescript-bot

TypeScript Bot (typescript-bot) commented Jun 13, 2024

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

CommandStatusResults
test top400✅ Started✅ Results
user test this✅ Started✅ Results
run dt✅ Started✅ Results
perf test this faster✅ Started👀 Results

@typescript-bot

Copy link
Copy Markdown
Contributor

Hey Sheetal Nandi (@sheetalkamat), the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

Comment threadsrc/compiler/builder.ts Outdated
@typescript-bot

Copy link
Copy Markdown
Contributor

Sheetal Nandi (@sheetalkamat) Here are the results of running the user tests with tsc comparing main and refs/pull/58839/merge:

Everything looks good!

cancellationToken: CancellationToken | undefined,
semanticDiagnosticsPerFile?: BuilderProgramState["semanticDiagnosticsPerFile"],
): readonly Diagnostic[] {
if (state.compilerOptions.noCheck) return emptyArray;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it funky that a noCheck with -b fetches different diagnostics than a noCheck normally does? This is going to skip all the program construction errors that noCheck would normally still report, no?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. If typechecking is skipped the semantic and program diagnostics are skipped for that file too.. (Look at getBindAndCheckDiagnostics as well as getProgramDiagnostics in program)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, so this early bail here just skips the per-file iteration that'll internally bail on every individual file anyway; got it. It's not really clear that that's guaranteed to be the case.

@typescript-bot

Copy link
Copy Markdown
Contributor

Sheetal Nandi (@sheetalkamat)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-Unions - node (v18.15.0, x64)
Errors3030~~~p=1.000 n=6
Symbols62,15362,153~~~p=1.000 n=6
Types50,24250,242~~~p=1.000 n=6
Memory used193,532k (± 0.93%)194,770k (± 0.92%)~192,257k196,095kp=0.378 n=6
Parse Time1.57s (± 0.52%)1.58s (± 0.48%)~1.57s1.59sp=0.120 n=6
Bind Time0.87s (± 0.87%)0.86s (± 1.40%)~0.85s0.88sp=0.502 n=6
Check Time11.38s (± 0.44%)11.39s (± 0.12%)~11.37s11.41sp=0.377 n=6
Emit Time3.25s (± 0.91%)3.28s (± 0.69%)~3.24s3.30sp=0.059 n=6
Total Time17.07s (± 0.37%)17.11s (± 0.21%)~17.06s17.16sp=0.296 n=6
angular-1 - node (v18.15.0, x64)
Errors55~~~p=1.000 n=6
Symbols944,104944,104~~~p=1.000 n=6
Types407,044407,044~~~p=1.000 n=6
Memory used1,218,567k (± 0.00%)1,218,586k (± 0.00%)~1,218,527k1,218,651kp=0.521 n=6
Parse Time7.97s (± 0.47%)7.97s (± 0.60%)~7.89s8.02sp=0.809 n=6
Bind Time2.23s (± 0.66%)2.23s (± 0.63%)~2.21s2.25sp=0.870 n=6
Check Time36.34s (± 0.34%)36.24s (± 0.23%)~36.13s36.31sp=0.261 n=6
Emit Time17.82s (± 0.87%)17.88s (± 0.67%)~17.79s18.11sp=0.470 n=6
Total Time64.36s (± 0.37%)64.32s (± 0.27%)~64.14s64.64sp=1.000 n=6
mui-docs - node (v18.15.0, x64)
Errors55~~~p=1.000 n=6
Symbols2,132,3012,132,301~~~p=1.000 n=6
Types926,081926,081~~~p=1.000 n=6
Memory used2,114,925k (± 0.00%)2,114,963k (± 0.01%)~2,114,765k2,115,123kp=0.689 n=6
Parse Time7.84s (± 0.57%)7.85s (± 0.34%)~7.82s7.89sp=0.808 n=6
Bind Time2.75s (± 0.50%)2.74s (± 0.67%)~2.72s2.77sp=0.249 n=6
Check Time84.10s (± 0.31%)84.31s (± 0.47%)~83.70s84.88sp=0.378 n=6
Emit Time0.16s (± 2.58%)0.16s (± 5.58%)~0.15s0.17sp=0.787 n=6
Total Time94.86s (± 0.29%)95.05s (± 0.42%)~94.44s95.63sp=0.378 n=6
self-build-src - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,229,5421,229,778+236 (+ 0.02%)~~p=0.001 n=6
Types260,916260,978+62 (+ 0.02%)~~p=0.001 n=6
Memory used2,370,937k (± 2.60%)2,406,525k (± 3.06%)+35,588k (+ 1.50%)2,346,363k2,497,315kp=0.031 n=6
Parse Time7.47s (± 0.95%)7.41s (± 0.59%)~7.36s7.47sp=0.199 n=6
Bind Time2.77s (± 0.77%)2.77s (± 0.38%)~2.76s2.79sp=0.806 n=6
Check Time49.62s (± 0.48%)49.56s (± 0.27%)~49.42s49.79sp=0.689 n=6
Emit Time3.88s (± 2.53%)3.79s (± 1.88%)~3.70s3.90sp=0.128 n=6
Total Time63.73s (± 0.40%)63.53s (± 0.31%)~63.31s63.79sp=0.149 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,229,5421,229,778+236 (+ 0.02%)~~p=0.001 n=6
Types260,916260,978+62 (+ 0.02%)~~p=0.001 n=6
Memory used2,421,421k (± 0.02%)2,448,000k (± 2.51%)+26,579k (+ 1.10%)2,422,078k2,573,671kp=0.008 n=6
Parse Time6.27s (± 0.76%)6.26s (± 0.35%)~6.23s6.29sp=0.748 n=6
Bind Time2.01s (± 0.58%)2.02s (± 0.87%)~1.99s2.04sp=0.514 n=6
Check Time40.69s (± 0.75%)40.59s (± 0.28%)~40.43s40.75sp=1.000 n=6
Emit Time3.23s (± 4.46%)3.14s (± 2.36%)~3.06s3.28sp=0.199 n=6
Total Time52.22s (± 0.93%)52.01s (± 0.36%)~51.86s52.36sp=0.230 n=6
self-compiler - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols258,665258,684+19 (+ 0.01%)~~p=0.001 n=6
Types104,931104,933+2 (+ 0.00%)~~p=0.001 n=6
Memory used428,295k (± 0.01%)428,407k (± 0.01%)+112k (+ 0.03%)428,352k428,472kp=0.005 n=6
Parse Time3.32s (± 0.53%)3.32s (± 0.52%)~3.29s3.34sp=0.870 n=6
Bind Time1.30s (± 0.84%)1.30s (± 1.13%)~1.29s1.32sp=0.863 n=6
Check Time18.09s (± 0.41%)18.09s (± 0.32%)~17.99s18.14sp=0.872 n=6
Emit Time1.38s (± 1.91%)1.39s (± 1.26%)~1.36s1.41sp=0.413 n=6
Total Time24.09s (± 0.29%)24.10s (± 0.22%)~24.02s24.14sp=0.810 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors3535~~~p=1.000 n=6
Symbols224,565224,565~~~p=1.000 n=6
Types93,73493,734~~~p=1.000 n=6
Memory used369,493k (± 0.02%)369,539k (± 0.03%)~369,464k369,726kp=0.298 n=6
Parse Time2.76s (± 0.50%)2.79s (± 0.97%)+0.03s (+ 1.15%)2.76s2.83sp=0.029 n=6
Bind Time1.60s (± 0.83%)1.60s (± 1.74%)~1.58s1.65sp=0.402 n=6
Check Time15.66s (± 0.30%)15.65s (± 0.35%)~15.59s15.74sp=0.808 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time20.02s (± 0.25%)20.04s (± 0.29%)~19.96s20.09sp=0.872 n=6
vscode - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols2,853,8422,853,842~~~p=1.000 n=6
Types968,167968,167~~~p=1.000 n=6
Memory used3,020,759k (± 0.00%)3,020,800k (± 0.00%)~3,020,736k3,020,891kp=0.335 n=6
Parse Time16.77s (± 0.24%)16.79s (± 0.34%)~16.70s16.84sp=0.226 n=6
Bind Time5.11s (± 1.72%)5.10s (± 0.26%)~5.09s5.12sp=0.252 n=6
Check Time89.33s (± 0.34%)89.67s (± 0.72%)~88.87s90.62sp=0.471 n=6
Emit Time29.25s (± 0.56%)29.19s (± 1.65%)~28.37s29.56sp=0.378 n=6
Total Time140.45s (± 0.32%)140.75s (± 0.43%)~139.85s141.45sp=0.378 n=6
webpack - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols266,914266,914~~~p=1.000 n=6
Types108,685108,685~~~p=1.000 n=6
Memory used411,357k (± 0.02%)411,340k (± 0.01%)~411,285k411,416kp=1.000 n=6
Parse Time3.18s (± 0.26%)3.17s (± 0.37%)~3.15s3.18sp=0.666 n=6
Bind Time1.41s (± 0.78%)1.42s (± 0.59%)~1.41s1.43sp=0.652 n=6
Check Time14.39s (± 0.56%)14.41s (± 0.34%)~14.35s14.46sp=0.572 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time18.97s (± 0.41%)19.00s (± 0.26%)~18.94s19.05sp=0.574 n=6
xstate-main - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols523,429523,429~~~p=1.000 n=6
Types177,955177,955~~~p=1.000 n=6
Memory used461,442k (± 0.07%)461,515k (± 0.07%)~461,209k461,958kp=0.378 n=6
Parse Time3.15s (± 0.84%)3.16s (± 0.77%)~3.11s3.17sp=0.934 n=6
Bind Time1.19s (± 0.43%)1.19s (± 0.83%)~1.18s1.20sp=0.931 n=6
Check Time18.13s (± 0.36%)18.11s (± 0.60%)~17.98s18.23sp=0.936 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time22.47s (± 0.33%)22.45s (± 0.45%)~22.34s22.59sp=0.873 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

Developer Information:

Download Benchmarks

// Because this is only output file in the program, we dont need fileId to deduplicate name
latestChangedDtsFile?: string | undefined;
errors: true | undefined;
checkPending: true | undefined;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this exactly derived from the options? Like, isn't this just options.noCheck? Eveywhere I'm looking in this PR, this is either set if options.noCheck in the buildinfo is present, or unset if it isn't. I guess NonIncrementalBuildInfo doesn't have options stored, but honestly, we should probably just always store them all.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No its not same as options.noCheck since it takes into account run with --noCheck, one without noCheck and then --noCheck without any updates in between does not update tsbuildInfo to mark it as pending. So its really if we havent done semanticdiagnostics calculation

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, it just seems like that's the same check we do for differing diagnostics-affecting options anyway? Like this is kinda redundant? (Though technically isn't since it's not using that mechanism with this PR.)

You run once without <flag> set, those options get cached into the .buildinfo, then you do a run with <flag> set - we compare the old <flag> value (nonextant) with the new <flag> value (present), and if <flag> affects diagnostics, mark the build as needing a .buildinfo update, minimally, and some subset of diagnostics and emit updated (as appropriate for the flag). At least for --incremental. Adding noCheck would, then, trigger a typecheck, yes, but it'd be a basically free no-op typecheck (since the flag's function is to skip it). Removing the flag then, would, naturally, also trigger a new typecheck (and you'd get the errors back). Just seems weird we wouldn't reuse that logic for non-incremental, and instead special-case the option like this. Doesn't seem like it's actually skipping anything beyond a few function calls that'd skip their work internally with the flag set anyway, and is duplicating what already happens inside createBuilderProgramState so long as the options from the old non-incremental build are actually serialized - which is why it seems so much like we're just hoisting the option out of the options list and into the root buildinfo.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s because we don’t want to cached diagnostics that happened as part of checking when running with no check..

This helps with keeping the cache .. if you see the diff of second commit you should see that we are now able to retain diagnostics of unchanged files

we do exactly same thing as part of pending emit so we don’t have to emit already emitted files

@weswighamWesley Wigham (weswigham)Jun 14, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I get what we're doing here that's special - we're trying to reuse the .buildinfo from an old non-noCheck build for another non-noCheck without a noCheck build in the middle clobbering it.

That feels kinda weird, since it's basically like we're ignoring the noCheck build even happened and not updating the state, but I kinda get why you'd want that - so long as the state on disk is from the non-noCheck build, you can start a noCheck (for emit) and non-noCheck (for checking) build in parallel off it.

Personally, I assumed you'd want separate .buildinfos for you non-noCheck and noCheck build parts if you were trying to ensure they didn't clobber one another as you parallelized things. This kinda feels like it's trying to enable running the two builds with different options collaboratively off of a single .buildinfo (since if the noCheck runs first, it'll leave the diagnostics part to the next build, while if the noEmit part runs first, it'll leave the emit part for the next build to fill in), which feels kinda odd. Both orders work in serial under this scheme, but if you actually parallelize it, one very likely could clobber the other's .buildinfo output regardless. So I guess it's kinda neat that we can do this, but I'm not sure why we'd need to. If you are running them in serial, the options themselves are going to skip the relevant work, so skipping them at the orchestrator level is a bit redundant and just lets you run them in arbitrary order repeatedly without being penalized with a rebuild when you toggle emit/check on and then off again, but in parallel, you need separate buildinfos to prevent race conditions between the processes anyway, which were never going to trigger a rebuild unless there was an actual substantive change in the first place.

I guess the TL;DR is:
Do we actually care if

tsc -b . --noCheck
tsc -b . --noEmit
tsc -b . --noCheck
tsc -b . --noEmit

reuses diagnostics from the first --noEmit run on the second --noEmit line? These kind of partial output builds are only useful in parallel, and in parallel, you need separate .buildinfos for safety anyway. I'd expect we'd wanna enable and recommend something like

tsc -b . --noCheck
tsc -b . --noEmit --tsBuildInfoFile ${configDir}/noemit.buildinfo
tsc -b . --noCheck
tsc -b . --noEmit --tsBuildInfoFile ${configDir}/noemit.buildinfo

instead, since that'll work even if you spawn the differently-option'd tsc -bs in parallel.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do care about that and there is buildInfo fidelity built into it. Eg currenly we do support tsc -b --noEmit followed by tsc -b so it doesnt typecheck again.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are not doing this for parallel builds but subsequent builds. Eg imagine running tsc -b --noCheck -w while debugging and once done last running tsc -b

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's fine to have this support, I'm just not sure it's gonna help many people in ways that aren't gonna lead them to a pit of failure. Like you see the sequential builds with different options smartly don't trigger rebuilds, so then naturally if you're a watch user you spawn one window with tsc -b --noCheck -w and one with tsc -b --noEmit -w as your super simple easy top-level parallelization, and that bricks the state, since the two builds will repeatedly clobber one another, rather than cooperate to build a single more complete state like the sequential builds do.

@typescript-bot

Copy link
Copy Markdown
Contributor

Sheetal Nandi (@sheetalkamat) Here are the results of running the top 400 repos with tsc comparing main and refs/pull/58839/merge:

Everything looks good!

// Because this is only output file in the program, we dont need fileId to deduplicate name
latestChangedDtsFile?: string | undefined;
errors: true | undefined;
checkPending: true | undefined;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's fine to have this support, I'm just not sure it's gonna help many people in ways that aren't gonna lead them to a pit of failure. Like you see the sequential builds with different options smartly don't trigger rebuilds, so then naturally if you're a watch user you spawn one window with tsc -b --noCheck -w and one with tsc -b --noEmit -w as your super simple easy top-level parallelization, and that bricks the state, since the two builds will repeatedly clobber one another, rather than cooperate to build a single more complete state like the sequential builds do.

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Author: TeamFor Uncommitted BugPR for untriaged, rejected, closed or missing bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@sheetalkamat@jakebailey@fatcerberus@typescript-bot@weswigham