Rethink AutoImportProvider automatic limits

[andrewbranch]

There have been a few issues with "includePackageJsonAutoImports": "auto" recently.

Today, we scan your project's package.json for dependencies not yet included in your TS program, scanning those dependencies' package.jsons for entrypoints to include as root files for the auxiliary program that collects additional auto-importable files. By default, we limit this to 10 entrypoints before giving up and not making an auto import provider at all. I think this was intended to be 10 packages, not 10 entrypoints. At the time the feature was added, most packages only had one detectable entrypoint ("main"/"types"). Now, it's quite common to see packages with more than 10 "exports" subpaths, which effectively guarantees that the feature will be disabled. #53116 (comment) shows an example of this.

On the flip side, once we've added our (10 or fewer) root files, there's no limit to how many files they might transitively pull in via imports. AutoImportProvider size was an editor startup time issue for the Teams codebase, not because of package.json inclusion, but because of referenced project inclusion (which has no limiter in place). At any rate, a limiter on root files is not a great solution for preventing costly project load, since a single file could pull in an unbounded number of others.

I'm interested in experimenting to see what would happen if we limit the auto import provider program's module resolution in certain ways, and then relax the limits on adding entrypoints. We could try limiting the depth of module resolution (e.g. root files can pull in other files, but then those files cannot pull in any more). At least, we could prevent explosions of dependencies by not allowing a node_modules package to transitively include any files outside of itself. Experimentation is needed, but off the top of my head, I think this wouldn't remove any available auto-imports except ones generated by export * from "other-package", since we don't (shouldn't) need to know the contents of other-package to generate an auto-import for foo after seeing export { foo } from "other-package".

I'm less sure what to do about referenced project inclusion, since projects often have include globs instead of distinct entrypoints, so it's very possible to add a huge number of root files even without performing any module resolution.