Slow compilation likely caused by too many types

[nscarcella]

TypeScript Version: >= 3.7.5 (reproduceable in @latest and @next)

Search Terms: slow compilation typecheck types typecount

After refactoring the model of our project to rely more heavily in classes we found a very significant increase in the typecheck time. Using the --diagnostics flag we noticed that our project seems to be producing over 2 million types (which takes around 37s to check). Half of those types came from our model definition, although that sounded like way too much, since we only have around 30 classes with little to none methods each.

Weirdest part is that the issue seems to mainly be cause by two particular method definitions. Removing these methods reduces the produced types from +900K to around 52K (around 1/6 of the original time). I'm not sure if we are doing something we are not supposed to in these methods or there is something else going on, but I would very much appreciate your insights.

I tried to purge our codebase so I could show you the smallest demo possible. You can find the trimmed example here.

The impact of the issue is, of course, less impressive, but commenting the copy method reduces the type count from 521K to arun 24K:

Code

abstract class $Node<S extends Stage> {

  ...
  
  // DELETE ME 
  copy(delta: Partial<Payload<this>>): this {
    return new (this.constructor as any)({ ...this, ...delta })
  }

  transform<R extends Stage = S>(tx: (node: Node<R>) => Node<R>): Node<R> {
    const applyTransform = (value: any): any => {
      if (typeof value === 'function') return value
      if (Array.isArray(value)) return value.map(applyTransform)
      // COMMENT THIS LINE SO THE CODE COMPILES AGAIN
      if (isNode<S>(value)) return value.copy(mapObject(applyTransform, tx(value as any)))
      if (value instanceof Object) return mapObject(applyTransform, value)
      return value
    }
    return applyTransform(this)
  }

}

Output WITH copy method

> tsc --noEmit --diagnostics

Files:            96
Lines:         32232
Nodes:        119346
Identifiers:   43635
Symbols:       48073
Types:        521712
Memory used: 260840K
I/O read:      0.01s
I/O write:     0.00s
Parse time:    0.50s
Bind time:     0.26s
Check time:    5.96s
Emit time:     0.00s
Total time:    6.72s

Output WITHOUT copy method

> tsc --noEmit --diagnostics

Files:           96
Lines:        32228
Nodes:       119301
Identifiers:  43620
Symbols:      44377
Types:        24409
Memory used: 90363K
I/O read:     0.01s
I/O write:    0.00s
Parse time:   0.41s
Bind time:    0.21s
Check time:   0.98s
Emit time:    0.00s
Total time:   1.60s

tsconfig.json

{
  "compilerOptions": {
    "target": "es2017",
    "module": "commonjs",
    "outDir": "dist/temp",
    "lib": [
      "es2020"
    ],
    "declaration": true,
    "strict": true,
    "removeComments": true,
    "moduleResolution": "node",
    "resolveJsonModule": true,
    "esModuleInterop": true,
    "experimentalDecorators": true
  },
  "files": [
    "src/model_demo.ts"
  ],
  "exclude": [
    "node_modules/**/*"
  ]
}

Despite my best efforts the issue still requires a couple dozens of subclasses to clearly manifest so please excuse the lengthy example.

Expected behavior: A less time consuming compilation

Actual behavior: Lots of types cause lots of check delay

Playground Link: Here

[weswigham]

For each of the 29 Node union members, we map them thru mapObject's type into new anonymous types like {prop1: any, prop2: any}. For each of those new types, we then have to compare it against Partial<Payload<this>>, which is ends up being an intersection of 29 members - each of which is the result of a deep series of conditionals and mappings; but the end result is an intersection of 29 unique generic mapped types (all with the same pair of apparent properties). Because they're all generic mapped types, we need to take an expensive comparison path in structuredTypeRelatedTo where we calculate the overlapping keys and construct source[overlapping keys], which we then compare with the mapped type template. When source is a union, and the overlapping keys are a union, this results in source length * overlapping keys length number of brand new types, all of which then get decomposed structurally and compared to the target template (which we then do 29 times because again, 29 essentially identical but ID-unique types - each time producing new types because the mapped type parameter is unique in each instance).

I've just put up a PR (#39696) that adds a fastpath for common cases where the mapped type template is similar to {[P in K]: Obj[P]} (like Pick) where we avoid making all the intermediate types for source[P] by recognizing that the P exactly matches the P in a template Obj[P] early on (therefore we just need to compare source to Obj).

[nscarcella]

Thanks @weswigham, that was fast!

[nscarcella]

@weswigham do you have any insights on how to further improve the times? Maybe explicitly defining some of the calculated types (or expressing them some other, more efficient, way) to ease the checker workload? I would love to have a more verbose diagnostic to detect these bottlenecks.

[weswigham]

Avoiding building types property-by-property with Omit-like helpers helps a lot, tbh (what did everyone in the ORM/react community do before we had that capability). While we can it means we make that many more types (which can end up getting multiplied by some identities or constructions). So you gotta be aware that being that accurate has a real checking cost (much moreso than just comparing one interface to another!).

@amcasey was working on a thing to detect which expressions in a project specifically induce the largest costs, if you really care about compilation time - I think he has a PR up right now.

[amcasey]

The PR as a whole is red (need to update some baselines), but there's a build here: #37785 (comment)