Design Meeting Notes, 11/22/2019

[DanielRosenwasser]

.mjs Input Files

#27957

• Node.js shipped module support recently

• How's this work?

  • In the Node runtime, as a CommonJS consumer, you can't interact with ESM at all.
  • require always does a CJS resolution
  • In a module, you can only import other modules, and they need to have extensions.

• Idea: if what Node has today is the best that it will ever deliver...

  • We'd have to redo basically everything in the CJS resolution scheme we do.
  • We'd need to add .mjs resolution, but error on that.
  • There will need to be a module format output.
  • Also, import(...) would...need to not be imported.

• Wait, why would you want any of this?

  • You want to migrate consuming code (?)

• Should we emit .mjs files?

  • Shouldn't emit .ts files to .mjs files because that'd be a massive breaking change.
  • Need a new module field?
  • Need .mts and .cts?
    • .mtsx and .ctsx?
    • .d.mts and .d.cts?
  • This is absolutely unreasonable.
    • Need to be able to disambiguate format of the original inputs.
    • Could it be in-band?
      • File extensions mean you don't need to read the file contents.

• As long as we don't rewrite imports (and we won't because it's error prone and requires whole-program knowledge), then we need the disambiguators.

• Take the following example

  foo/package.json
   foo/index.js (this is ESM)
   foo/index.cjs (this is CJS)
  

  • require("foo") breaks because it'll resolve to foo/index.js
  • import "foo" doesn't work because it won't resolve to /index.js because Node thinks it's too magical for ESM.

• This logic is extremely complex.

  • "The combinatorics of this are extremely absurd."
  • Can we come up with a smaller set of what is supported?
    • By doing that, it provides a prescriptive direction for users who ask "how do I write ES modules in Node and TypeScript?"
  • But have to be smart about what we "leave on the table" and what that breaks.
    • Also want to leave things open for later additions.

• Conclusion:

  • .mjs in an import path is doable long-term, but not
  • Node ESM support as a whole will need to be scoped out.
    • .mjs output is troublesome.

--emitExtensions and Importing .ts Files

#35148

• We're not rewriting paths.
  • Not now, not tomorrow, not ever.
• So is fixing this just a matter of relaxing the check?
  • Sounds like it.
  • But the check stops people from shooting themselves in their feet, so don't want to just confuse people more.
• Unclear what the original use-case really is. Linked issues have very different goals/use-cases.
  • Proposal: emit files without .ts extension to get custom file extension #30076
    • [[wat.bmp]]
• It sounds like the only use-case is deno
  • So "moduleResolution": "deno"?
  • Uh, but what's your emit??
  • How do you deal with https:// paths?
    • Maybe don't.
    • Can simulate this with path mapping/import maps.

PnP Resolution

#35206

• What are the problems with pnp?
  • Executes arbitrary .js file.
  • The defaults make little sense in the editor - means that users need to configure their editors to load tsserver with yarn. Very opt-in.
  • Also, while guidance is to stick to one version of package manager, users sometimes mix/match package managers and different versions.
  • Kind of hard to be sure what doing the right thing means here that doesn't require the same level of configuration as an LS plugin that overrides resolution

Reorder Extension Priorities

#34713

• Out of time.

[MicahZoltu]

We're not rewriting paths.
Not now, not tomorrow, not ever.

I have seen this several times but haven't seen details about it. Can someone help explain it for us plebs why this is "100% off the table, never to be considered, never to be discussed, never to be brought up"?

I suspect there is a good reason for this, but it would really help me accept it if I could read the discussion that lead to such a hardline stance. I understand it is hard, but given the other options it feels like we really should be exploring all possibilities fully, including hard ones.

[Jack-Works]

We're not rewriting paths.
Not now, not tomorrow, not ever.

I have seen this several times but haven't seen details about it.

Agree, if not rewriting the extension name, then how to emit valid code for Node module resolution and deno?

[RyanCavanaugh]

I have seen this several times but haven't seen details about it. Can someone help explain it for us plebs why this is "100% off the table, never to be considered, never to be discussed, never to be brought up"?

Let's say you write some code in a JavaScript file

var x = y + z;

If you paste that code into a TypeScript file, it means

var x = y + z;

This the fundamental promise of TypeScript: You can take some JavaScript code, put it in a TypeScript file, and it means the same thing meant before. I would argue it is the most important design principle we have, because it's the only reason JS -> TS migration is possible and the only reason JS <-> TS interop is sane.

This is why we don't have extension methods, even though it would be nice.
This is why we don't have operator overloading, even though it would be nice.
This is why we don't have exceptions on missing property access, even though it would be nice.
This is why we don't have automatic binding of this on class methods, even though it would be nice.
This is why we don't have native integers, even though it would be nice.
This is why we don't have exceptions on divide-by-zero, even though it would be nice.
This is why we don't have automatic prefixing of this. on class members, even though it would be very nice.

We've kept that promise on literally any random JS snippet you can think of, and while some JS code may have type errors, it does the same thing at runtime it does before. Changing your file extension from .js to .ts will not break your program.

Now you can say "Well that promise sucks, you should break it and just rewrite paths because rewriting paths is the best possible thing a programming language can provide". OK, fine.

Our stance has always been that you should write the import path you want to appear in the emitted code, and add path mapping or other configuration to make TypeScript understand what that path should resolve to. This is 100.0% consistent with the idea that you should write the JavaScript you want to appear in the emitted code and add type annotations or assertions to make TypeScript understand what the type intent of the code is. In fact, it's not even a different principle at all, because an import statement is JavaScript and we have already established that you should write the JavaScript you want to be emitted.

The thing I see every time is that people go through this cycle:

1. Start with working import paths
2. Add path mapping to their tsconfig to create aliases
3. Change their import paths to use those aliases
4. The program stops working because the import paths are now wrong

The right fix is to either:

• Not do step 2 in the first place
• Tell your loader about the aliases you invented in step 2 so it can handle them

TypeScript isn't here to provide a module aliasing system. If your loader supports such a system, great, use path mapping to describe it. TypeScript isn't obligated to invent new ways of adding epicycles to module resolution, and doing so would violate one of our core principles.

But again, let's say you think that rule is bad. What happens if we start rewriting import paths tomorrow?

Well, today, you write the import path that you want to appear in the emitted JavaScript file. You have two paths to consider here:

• The import path you wrote, which you can trivially understand will appear in the output
• The file that TypeScript resolved this to

This is already insanely complicated. paths, baseUrl, baseUrls, the module resolution setting, the location of the originating file, the effect of symlinks, file extension priorities, the fact that there are really three different kinds of paths each with their own independent resolution strategies, etc. etc..

Tomorrow, if you write an import path, you now have three things under consideration:

• The import path you wrote
• The file TypeScript resolved that to
• What the path that should appear in the output is

This is taking a complicated 2-dimensional math problem and moving it to 3 dimensions. People can already barely understand the interaction of include, exclude, and module imports - do we really want to make this the most complicated configuration system ever imagined? The user confusion will be unending.

That said, this is 0% about the difficulty of such a system. The existing resolution system is already difficult; we are not afraid of doing difficult things. We are opposed to doing things that violate our fundamental promise that the JS code you wrote is the JS code you get.

If you find yourself trying to write a valid runtime path, and can't get TypeScript to understand how to resolve that path to the target module, tell us about it. We have half a dozen module resolution flags and will keep adding more until any valid runtime path you write can be reasonably resolved to a corresponding file or declaration.

Conversely, don't tell us that you wrote an invalid runtime path and want us to fix it for you! This is the same as saying that you wrote var x = y + z; and want TypeScript to emit something different because you really wanted some other JavaScript - that's not what we do, and not what we've ever done.

[Jack-Works]

I got the principal now, and I propose a change in my --emitExtension PR at #35148 (comment)
Does it seems okay now?

[Jamesernator]

My summary on Node ES Modules and TypeScript incompatibilities

Dynamic modules / Named imports from CommonJS

Currently import * as React from 'react'; and import { PureComponent } from 'react'; as handled by default by TypeScript is not compatible.

Only import React from 'react'; is supported (the whole CJS module as default export).

Why is this the case?

Dynamic modules has not gained consensus within the working group.

Alternative solutions have not been accepted either.

TS Author compromise

esModuleInterop can be enabled for users wishing to output ESM.

Problems with compromise

import * as React from 'react'/import { PureComponent } from 'react'; is not treated as an error but will error in Node's implementation when targeting "module": "esnext".

Required extensions

Extensions are required, current auto-suggestions do not add extensions automatically and still report the code as OK which will definitely cause confusion.

Why is this the case?

Extensionless imports failed to gain consensus although discussion is still ongoing.

TS Author compromise

Authors can add extensions themselves for ES output.

Problems with compromise

Authors must take care to add another package.json if they have multiple targets as .js cannot be used for both targets.

package.json exports

New feature typescript doesn't understand. Basically a package can be published with a package.json like:

{
 "name": "my-package",
 "exports": {
 "/foo": "./dist/foo.js"
 }
}

And consumers can then do:

import foo from "my-package/foo";

Why is this the case?

New feature to support authors of packages to freely change their package structure without affecting consumers.

TS Author Compromise

Only import foo from './node_modules/my-package/dist/foo.js' works for now.

Problems with compromise

This is different to actual supported "my-package/foo.js", feature simply can't be used without TS support.

.mjs/.cjs/"type": "module"

Currently TypeScript uses existence of import/export to distinguish modules and comnmonjs.

Node however uses package.json "type": "module" | "commonjs" to determine whether or not .js is CJS or ESM. .cjs and .mjs are always treated as CJS and ESM respectively regardless of "type".

Some already existing modules cannot be used in TypeScript because of this (e.g. idlize).

Why is this the case?

Automatic detection was rejected due to various hazards. I can't find the actual reference for when this decision was finalized.

Instead dual packages will be replaced with (the currently experimental) conditional exports if no require(esm) or other such solution is resolved by January 2020.

TS Author Compromise

There's no compromise if author's want to use .mjs/.cjs authored packages. Currently import IdleValue from 'idlize/IdleValue.mjs' is simply not supported.

[Jamesernator]

This is already insanely complicated. paths, baseUrl, baseUrls, the module resolution setting, the location of the originating file, the effect of symlinks, file extension priorities, the fact that there are really three different kinds of paths each with their own independent resolution strategies, etc. etc..

I have never even considered using this features for this reason. However existing complexity doesn't mean the alternative system increases complexity. It may be the case that within roots that have a tsconfig.json with such a feature they cannot use baseUrl/baseUrls/paths/etc.

Speaking for myself, all I really want a is a tool that turns a graph like this:

Into one where the .ts files are re-mapped to whatever format I want.

Maybe I want modules:

Any maybe I want CommonJS with .cjs to sit alongside side the other modul graph:

---

In these graphs the specifier always points to the type of resource I want, but what I want .ts to be converted to might vary depending on target. Relying on ./noextension doesn't even work if those output graphs go into the same directory like so:

In such a graph ./a resolves to multiple but given a resolution order it'll always resolve to one of them. So simply outputting the specifier with no transform doesn't work.

[weswigham]

I can't find the actual reference for when this decision was finalized.

Because it hasn't been. There are just individuals with opinions - I'm one of them.

[Jamesernator]

Because it hasn't been. There are just individuals with opinions - I'm one of them.

Well some agreement must've been achieved to unflag the current implementation (even if it is still experimental), this seems like a pretty strong signal that the core implementation of modules is nearing readiness even if there's still rough edges.

Also the problems will still be present even with auto detection, as TypeScript will still need to learn to understand the extensions if it wants to be able to support importing packages that use .mjs.

Although I do think this is one of the lesser issues compared to the others I mentioned as TypeScript authors can always decide just to publish non-mixed types for the time being.

[weswigham]

It is how it is because it's easier to add extension resolution back in than to remove it. That's what we could agree on.

[hybrist]

In the Node runtime, as a CommonJS consumer, you can't interact with ESM at all.

Just to clarify: You can't require ESM but you can import() ESM using dynamic import.

[Jack-Works]

I got the principal now, and I propose a change in my --emitExtension PR at #35148 (comment)
Does it seems okay now?

I have fully rewritten the pr, for any one interested, see #35148