Slow start times due to use of barrel files

[greyscalemotif]

🐛 Bug Report

It is unclear that large dependency graphs caused by the use of barrel files can slow test initialization dramatically. There are some tangential mentions of this in the documentation, but it is not outlined in a clear and direct manner. This likely leads to a lot of inefficiency in many projects.

Here are the most important points to stress:

• Jest simulates Node's require cache to allow for isolation and mocking
• The require cache is built independently for every test suite
• All dependencies of a file are included, even if unused

It is the last bullet that leads to the largest reduction in efficiency, due mainly to barrel files. Barrel files are index files that re-export the exports of other files in a directory. They make it possible to import multiple related dependencies in a single import statement. The downside of this is that Jest sees all of the re-exported contents of the barrel file as dependencies and crawls through them, used or not.

Reducing the use of barrel files can help quite a bit in reducing the amount of time it takes before a test can start. This is especially true of dependencies pulled from NPM packages. Packages are typically developed with a root index file that acts as a barrel file for the entire package. If the package is rolled up into a single file at build time, there is only one file for the Jest runtime to open and parse. If the package publishes its source files independently, without a rollup stage, the Jest runtime will need to open and parse every file in the package independently.

In an enterprise setting, there are often internally developed tools and libraries. These packages can grow to be fairly large, and given their internal use, it can be tempting to provide the source files as the output of the packages. In fact, this can improve tree shaking when building the applications that depend on them. Jest, however, can suffer greatly in this environment because the raw number of files can grow without bounds. This problem becomes exponentially worse when overly tight internal dependency semvers reduce the ability of package managers to de-duplicate their installs.

Resolving these issues can lead to tremendous decreases in Jest's test suite initialization and this should be highlighted in the documentation. Barrel files can have a huge impact on the number of files that the Jest runtime needs to parse, and that is not clear without a deep dive into the way dependencies are evaluated. Sharing this knowledge more broadly could make an already fantastic test runner that much better and improve the quality of many products that rely upon it in the process.

To Reproduce

As this is a documentation issue and not a code issue, this may not apply. However, in the spirit of completeness:

• Read the Jest documentation
• Experience slow test start times in enterprise scale development
• Be unaware of the impact that barrel files can have on running Jest

Expected behavior

The expectation is that the Jest documentation should more explicitly explain the impact that barrel files can have on test start times.

Link to repl or repo (highly encouraged)

Given that this issue is manifested in large scale applications with many inter-related dependencies, it is not feasible to provide a replication of the issue.

envinfo

N/A

[gilamran]

On our project we have ~100 test suites, ~600 tests. it take about 6 minutes to run all the tests.
The tests are very fast (less than 50ms. per test), but the startup time is very slow (And annoying)

I've created a script that changes all the imports from barrel files to explicit import, and the tests run 10x faster (Still not optimal, but shows the problem)

Anyone with a solution/workaround/idea?

[SimenB]

I'm happy to take a PR outlining this issue. 👍

---

In Jest itself we've made lots of imports lazy to avoid this issue, but doing across the entire codebase requires the use of babel (or some other code transform). Might be an OK workaround in tests, but won't help in node_modules as it's not transformed by default (you can have Jest transform node_modules if you want, but at some point there are diminishing returns since you then have to wait for the file to be transformed).

[fadi-george]

@gilamran could you please share that script for changing the barrel files import? I'm in a similar bind where changing one reusable component is running 100s of tests. I've tried using babel transform plugin imports but it was not working for me.

[gilamran]

@fadi-george sorry I don't have the script anymore (Maybe I can look it up in the git history), but I've found even faster solution.
I'm using esbuild to bundle ALL my tests + code into one huge file, then I run jest on this file only. so instead of 350sec. with jest as it is, it's taking 30sec tops.

I had to do some juggling to make esbuild and jest work together, like importing svgs etc.
you can see the script here

[fadi-george]

@gilamran ah interesting approach, thank you for sharing! I'll definitely play around with that idea.

[fadi-george]

@gilamran I assume you are doing something like this? Have you ran into any memory issues?

node build-tests && jest ./src/bundle.spec.ts

bundle.spec.ts:

import testBundle from './tests/bundle';

test('should pass', () => {
  testBundle();
});

[fadi-george]

I believe I fixed my memory issues but now I ran into this problem with jest mocks and mockImplementations failing.

[fadi-george]

@gilamran did you use any jest.mock calls? For me, it forces my tests to fail

import { someAction } from 'actions/something';
jest.mock('actions/something');

It might be supported later on with esbuild or jest, I tried top-level-await but I couldn't get it to work

jest.mock('actions/something');
const { someAction } = await import('actions/something');

[gilamran]

I don't use jest.mock at all (And I think that it's bad practice in most cases).
I also don't think that it's possible to use when doing the esbuild bundle. The bundle already includes all the code, and I think that jest.mock is overriding the import which doesn't exist anymore...

[rsslldnphy]

I have made a simple repo that reproduces this issue using a barrel file provided by a large third-party library, in this case Material UI icons: https://github.com/rsslldnphy/jest-barrel-files

On my machine, the test that imports icons as import * as Icons from "@mui/icons-material"; takes about 7 seconds to run, compared to fractions of a second for the test that doesn't.

I do not have a good understanding of the mechanics of what goes on (with treeshaking etc) when importing files in this way, but I'm not encountering this slowness issue the other tools I'm using - is there a way to make jest aware of code that can be ignored?

[PupoSDC]

@rsslldnphy for material specifically, which is a quite big library, I isolate the parts of the library my code actually uses in a re-exported barrel file:

// components/index.ts

export { default as AppBar, type AppBarProps } from "@mui/material/AppBar";
export { default as Button, type ButtonProps } from "@mui/material/Button";
// ...

export { default as DateRangeIcon } from "@mui/icons-material/DateRange";
export { default as CheckCircleIcon } from "@mui/icons-material/CheckCircle";
export { default as LocalShippingIcon } from "@mui/icons-material/LocalShipping";
// ... keep in mind that even 50-100 icons is only a small part of the 2000+ icons material exports

To be fair, we did this to mark to other developers in the team what parts of the material ui they could use without having to have a talk with our UI/UX designer beforehand. But it also mitigates this issue as a side effect.

[superjose]

On our project we have ~100 test suites, ~600 tests. it take about 6 minutes to run all the tests. The tests are very fast (less than 50ms. per test), but the startup time is very slow (And annoying)

I've created a script that changes all the imports from barrel files to explicit import, and the tests run 10x faster (Still not optimal, but shows the problem)

Anyone with a solution/workaround/idea?

I've had this problem. Problem is barreling, and the way the import mapping works.

Unfortunately, it seems that transpilers (TypeScript), bundlers (WebPack, SnowPack, Vite, Rollup) can't differentiate exactly which is the import you're trying to use, and it will pull all the other files that were specified in the barrel index file. I'm currently refactoring a somewhat big app because I was waiting for 3 minutes until I ran 3 tests, and it was because my entire app was barreled.

[rsslldnphy]

@rsslldnphy for material specifically, which is a quite big library, I isolate the parts of the library my code actually uses in a re-exported barrel file:

// components/index.ts

export { default as AppBar, type AppBarProps } from "@mui/material/AppBar";
export { default as Button, type ButtonProps } from "@mui/material/Button";
// ...

export { default as DateRangeIcon } from "@mui/icons-material/DateRange";
export { default as CheckCircleIcon } from "@mui/icons-material/CheckCircle";
export { default as LocalShippingIcon } from "@mui/icons-material/LocalShipping";
// ... keep in mind that even 50-100 icons is only a small part of the 2000+ icons material exports

To be fair, we did this to mark to other developers in the team what parts of the material ui they could use without having to have a talk with our UI/UX designer beforehand. But it also mitigates this issue as a side effect.

Nice idea on both counts @PupoSDC! May well implement this. Will be a bit of a faff to set up and maintain with icons especially but not a bad trade-off at all for mitigating this issue. Thanks!

[Grohden]

Not sure if it will help, but here's my 2 cents:

I face problems with barrel files not only in jest, but on some external and internal libs that have optional peer dependencies. Including these libs which do all exports from a barrel causes compile errors due to indirect import of unused components that use optional not installed dependencies

To fix this, I was using the already mentioned babel-plugin-transform-imports, which IMO works great (although I had to fork it and fix import aliases issues)

I even tried to improve the transform imports solution writing babel-plugin-resolve-barrel-files, but its a simple solution for ESM modules only.

But I guess that for jest, people could try implementing a lazy import with mocks:

jest.mock('module-with-barrel', () => {

  const RequiredComponent = jest.requireActual('module-with-barrel/RequiredComponent').default;
  return {
     __esModule: true,
    RequiredComponent
    // or use a more lazy approach...
    //  get RequiredComponent() {
    //     return jest.requireActual('module-with-barrel/RequiredComponent').default;
    // }
   }
})

---

Unfortunately, it seems that transpilers (TypeScript), bundlers (WebPack, SnowPack, Vite, Rollup) can't differentiate exactly which is the import you're trying to use

They probably can, but since barrel files are normal source code files, they can execute side effects (read the tip) (eg: some dep exporting global stuff).. so it's more safe to just don't optimize unless you explicit tell them to do it (the case for the babel plugins and webpack configs)

[piotrroda]

I confirm. Using barrel imports extreamly slows down starting test suites. Test cases run very fast. Please manage this problem.

[boubou158]

Is there any solution for this? I have more than 3000 imports to change if i want to reverse the barrels import to full file path import, which would take very long time.

[gilamran]

No solution, but there's a workaround

[dsmalik]

Just a thought on this as recently i was facing a similar issue and the way i fixed it is by creating a custom barrel import transformer. The way it works is in first step we iterate all files to determine all the exports of files in project making a Map of import to lookup later.

Now when the test start to execute, using jest transform configuration, then execute a transform which uses the import lookup map created in first step to rewrite the import statements to specific imports statements and then jest executes on the transformed code.

Was able to significant improvement with this approach when using jest with ts-jest (isolatedModules enabled).

[boubou158]

How did you manage it? Do you have any example somewhere? I am very interested as i can't manage to reduce the time execution of my spec files due to the size of the project and barrel imports everywhere.

[dsmalik]

@boubou158 added a typescript based sample here with some readme- https://github.com/dsmalik/ts-barrel-import-transformer

[boubou158]

After spending a month trying to make esm working on our project trying to speed up the jest performance, it seems impossible to have 100% working. I am giving up on the esm option. I am now checking your solution (thanks a lot for that by the way !) but i am a little confused on how i could integrate this transformer with the transformer used by jest-preset-angular?

[boubou158]

@fadi-george sorry I don't have the script anymore (Maybe I can look it up in the git history), but I've found even faster solution. I'm using esbuild to bundle ALL my tests + code into one huge file, then I run jest on this file only. so instead of 350sec. with jest as it is, it's taking 30sec tops.

I had to do some juggling to make esbuild and jest work together, like importing svgs etc. you can see the script here

How would you apply your approach on an angular project? I am running esbuild on a single spec file with --bundle option, it is then throwing errors TypeError: Cannot read properties of null (reading 'ngModule') every where when i run this file with jest.
The barrel imports slow performance is definitely a major issue of using jest for big projects :(

[rendner]

We also face long startup times in our tests. After trying a lot of "solutions" mentioned on the internet we discovered that in our case the problem are the barrel files.

I created two simple unit tests in our code base. Test A which imports from barrel files and test B, in which these imports are replaced by named imports. Here are the results:

test	duration
A	~29s
B	91ms

To be able to dig deeper into the problem you have to understand what is going on during the long startup time.
For this I modified the class Runtime in node_modules/jest-runtime/build/index.js.

Modify Jest Runtime Class

Instead of a patch I link the code-pointers and the added code, in case you run a different jest version.
All modifications are marked with // MARKER to easily find them afterwards.

The file to modify is: node_modules/jest-runtime/build/index.js

Step 1

Add additional class properties to track the number of loaded modules.
code pointer
Add before the marked line:

// MARKER
this._LOAD_COUNTER = 0;
this._INDENTS = [];
// MARKER

Step 2

Count and log loaded modules.
code pointer 1
Add before the marked line:

// MARKER
this._LOAD_COUNTER += 1;
console.log(`${this._INDENTS.join('')}>>_loadModule`, this._LOAD_COUNTER, moduleName)
this._INDENTS.push('\t')
// MARKER

code pointer 2
Add before the marked line:

// MARKER
this._INDENTS.pop()
// MARKER

Outcome

After patching the Runtime class I got the following numbers for my simple unit tests:

test	loaded modules
A	17424
B	1503

And you can clearly see by the logged output that jest is importing everything in case of barrel files before running a test.

After adjusting only some of the imports (to use named imports) in our code base as a PoC, I was able to reduce the build time from 15 minutes to 6 minutes.

[krishnachaitanya137]

We have around 3200 test suites and 35k tests. We had the similar issue. @mui/icons-material is the culprit in our case. We simply mocked the package and time came down from 50 minutes to 18 minutes. We are still going through other packages but that's a significant time improvement for mocking just one package,
Haven't tried with babel-plugin-direct-import. But i'm concerned how modules array will look in larger projects which uses around 50 MUI icons.

[TSMMark]

Instead of a patch I link the code-pointers and the added code, in case you run a different jest version. All modifications are marked with // MARKER to easily find them afterwards.

This is fantastic. Thanks for sharing your patch.

I've got it printing out the results, but I'm not sure what to look for. I can see that it's certainly loading a ton of modules, but nothing stands out as something that can be optimized. Do you have any advice? Currently taking >20s to run a <3s test.

[rendner]

@TSMMark

I've got it printing out the results, but I'm not sure what to look for.

To investigate, you should select a simple and rather small test that has an unexpectedly long start time.

Maybe you can see the difference better in the following react example. As mentioned by others @mui/icons-material is a good candidate to slow down your tests massively.

If you are not already using @mui/icons-material in your project, add it as a devDependency for demo purposes.

Create a test tsx-file and add the following code:

import React from 'react'
// this import slows your test down
import { DoNotDisturbOn } from '@mui/icons-material'
// this import not
// import DoNotDisturbOn from '@mui/icons-material/DoNotDisturbOn'

test('should demo the problem', () => {
  <DoNotDisturbOn />
  expect(1).toBe(1)
})

The test with the second import for DoNotDisturbOn should start much faster because it imports far fewer modules.

Now run the single test with only one of the two imports for DoNotDisturbOn and compare the amount of loaded modules. Search in the output for @mui/icons-material to see what jest also imported from it.

With this knowledge, you should be able to find problematic imports in your codebase.

[sibelius]

is there a plugin or something that can solve all barrel files usages?

[baublet]

is there a plugin or something that can solve all barrel files usages?

Unfortunately not. There are several workarounds in this thread, but none are low-effort.

---

FYI, if you're looking at any of the above issues, here are some limitations and things to experiment with:

- Remove barrel files and only test what you can see

Sure-fire way to fix this. However, not feasible in larger codebases using barrel files liberally. (Plus, it's tilting at windmills. Build for the actual need, not the ideal.) In addition, mock things your test subject is using, so that your test is focused on exactly the logic you wrote. Let TypeScript make sure your interfaces continue to align, and if you're worried, you can have another layer of tests to ensure interop between modules. (We have a test capital-i, where we have 100% unit test coverage, and a comprehensive set of end-to-end tests that validate high-level module interop. Almost no integration tests because those often end up slower than our end-to-end tests!).

- Try out the various Babel plugins listed above

This may work for you on relatively small projects without using a lot of mocks and spies. However, most of these are opt-in, and are only useful if you can identify up-front where a lot of your heftier barrel files are. In many cases, this isn't feasible.

- Try another test runner

Something like Vite might work better, but most people won't have a 1:1 transition from Jest. Vite uses enforces strict modules (e.g., not mockable/spyable in the same ways Jest is). Also, it suffers from the same issues as those described below, since it also uses ESBuild. But you might have luck here if you don't do anything fancy with mocks/spies.

- ESBuild/SWC all the things

If you barely use Jest's mocking/spy features, this might be feasible. However, any alternative transformer to Babel will be a pain if you use them. Synthetic exports in babel are loosely constructed (e.g., they're mutable references, so can be replaced spied on). Tools like SWC and ESBuild adhere more to official specs for modules (e.g., modules are immutable, can't be monkey patched, etc.).

I have not yet tried Rollup for our tests. Might be worth a shot if someone has the energy, but I'd be surprised if it's any faster/better/useable than any of the above examples.

I have experimented with a custom Jest transformer, too, that uses Babel to hoist jest.mock calls; esbuild to transpile everything else; and comprehensive set of modifications to esbuild output that enables things like istanbul ignore lines (esbuild transpiles out all comments unless you mark them as legally required) and switches it to a loose module system with easy find/replace. But this also has limitations (e.g., if you're not careful, it can break your source maps; it still transpiles a million things, but at least esbuild is faster than babel at it). This is a fraught game of whack a mole. You have to know and account for every weird way people use mocks and spies ahead of time, and with thousands of tests, there are probably a couple hundred combinations to account for 😭

---

In my case, we're a bit stuck. We have a huge codebase with tons of mocks and spies (we require 100% code coverage, and suggest that developers have 1:1 test -> adjacent test file with 100% coverage, so our codebase is at least half test code). This has been excellent for maintaining high code quality based on the standards when we began (circa 2019). But the barrel file issue is a thorny one. In new code, we don't allow them, but we've got thousands of tests that drag because of this.

To put it simply: there's no easy solution here. In hindsight, it was probably engineering malpractice to suggest the use of barrel files, and we're paying for our past oversight! 😆

The only clear way forward at the moment is to suck it up, deal with slow tests for a while, and refactor them as you get the opportunity. I've been experimenting on the side for my team for years looking for ways to make this better, especially as our lazier engineers realize how easy we've made it to write thick tests that exercise huge swaths of the platform. Yeah, it's less test code, but it takes two minutes to get feedback. Some folks don't mind waiting, but I sure do.

Despite our best efforts, even the best engineers will take shortcuts to shave off a few minutes from their tickets, and make everyone pay for that in the time it takes to run tests. Amortized over several years and several dozen developers, and you're looking at 3 minute startup times for a single, 3ms test that ensures an if statement works correctly in a 12 line file.

I almost think Jest (or some other tool) needs a way to elevate how much code a certain test is using and give us the ability to set upper limits. It's a waste of time and CPU to load 20,000 modules to test a couple of logical branches. No matter what Jest does to make this better, without the automatic tooling for us tech leads to set upper bounds on this, I don't foresee this being all that fixable with some out of the box solution... 🤔

[mikerentmeister]

I have taken a stab at writing my own jest transformer to accomplish this. I'm using an Nx mono repo with about 11 Angular libraries inside (v17.1), I have about 1600 unit tests, and it takes about 5 minutes to run. The jest transformer I've created will correctly replace all of the paths in my ts config with the import pointing to the actual file of where it's exported. However, my solution doesn't work when it comes to transforming 3rd-party paths like @angular/core, @ngrx/store, etc., because ts.createProgram will only return declaration files for any path I pass in that points to a node_module file. With this transformer only changing local imports, it's only shaved about 30 seconds off the execution time.

If there is anyone who might be able to lend a helping hand with this transformer, I would be very appreciative. I'm sure it could be very beneficial to others coming here as well. I only started working with the TypeScript compiler api a few days ago, so I'm sure that my solution is far from optimal. modifyImportDeclaration is where I would need the most help. I was also looking into SWC, to see if they had an api that would do something like ts.createProgram and ts.transpileModule, but I couldn't find anything.

// jest.preset.js 

{
  ...
  transform: {
    '^.+\\.(ts|mjs|js|html)$': [
      '<rootDir>/jest-import-transformer.js'
    ]
  },
  ...
}

// jest-import-transformer.js

var fs = require('fs');
const path = require('path');
const mkdirp = require('mkdirp');
var angularPreset = require('jest-preset-angular');
const presetConfig = {
  // NOTE: All tsconfig.spec.json files used have their target set to ES2016 to deal with the following problem
  // https://github.com/nrwl/nx/issues/15390#issuecomment-1457792176
  tsconfig: '<rootDir>/tsconfig.spec.json',
  stringifyContentPathRegex: '\\.(html|svg)$',
  isolatedModules: true
};
var baseTransformer = angularPreset.default.createTransformer(presetConfig);
const ts = require('typescript');
const tsConfig = require('./tsconfig.base.json');
const pathCache = {};
const filePaths = {};

function extractAllExportStatements(filePath) {
  const program = ts.createProgram([filePath], {
    target: tsConfig.compilerOptions.target,
    module: tsConfig.compilerOptions.module
  });
  const output = {};
  for (const sourceFile of program.getSourceFiles()) {
    if (!sourceFile.isDeclarationFile) {
      // Walk the tree to search for exported nodes
      if (!filePaths[sourceFile.fileName]) {
        filePaths[sourceFile.fileName] = path
          .relative(__dirname, sourceFile.fileName)
          .replaceAll('\\', '/');
      }
      const fileName = filePaths[sourceFile.fileName];
      ts.forEachChild(sourceFile, node => visit(node, fileName));
    }
  }

  return output;

  function visit(node, fileName) {
    // Only consider exported nodes
    if (!isNodeExported(node)) {
      return;
    }

    if (ts.isModuleDeclaration(node)) {
      // This is a namespace, visit its children
      ts.forEachChild(node, visit);
    } else if (node.name) {
      output[node.name.text] = fileName;
    } else if (ts.isVariableStatement(node)) {
      output[node.declarationList?.declarations[0]?.name?.text] = fileName;
    } else if (ts.isExportDeclaration(node)) {
      output[node.exportClause?.name?.text] = fileName;
    } else {
      debugger;
    }
  }

  function isNodeExported(node) {
    return (
      (ts.getCombinedModifierFlags(node) & ts.ModifierFlags.Export) !== 0 ||
      (!!node.parent && node.parent.kind === ts.SyntaxKind.SourceFile) ||
      node.exportClause?.name?.text // export * as [Name] syntax
    );
  }
}

function ensureBarrelIsCached(oldPath, barrelPath, options) {
  if (!pathCache[barrelPath]) {
    // Not in the cache, let's check the cacheDirectory
    const cacheFilePath = path.join(options.config.cacheDirectory, oldPath + '.json');
    if (fs.existsSync(cacheFilePath)) {
      pathCache[barrelPath] = JSON.parse(fs.readFileSync(cacheFilePath));
    } else {
      // https://github.com/microsoft/TypeScript/wiki/Using-the-Compiler-API#using-the-type-checker
      pathCache[barrelPath] = extractAllExportStatements(barrelPath);
      const dirName = path.dirname(cacheFilePath);
      mkdirp.sync(dirName);
      fs.writeFileSync(cacheFilePath, JSON.stringify(pathCache[barrelPath]));
    }
  }

  return pathCache[barrelPath];
}

function invalidateAndReloadBarrelCache(oldPath, barrelPath, options) {
  const cacheFilePath = path.join(options.config.cacheDirectory, oldPath + '.json');
  if (fs.existsSync(cacheFilePath)) {
    fs.rmSync(cacheFilePath, { force: true, recursive: true });
  }
  pathCache[barrelPath] = null;
  return ensureBarrelIsCached(oldPath, barrelPath, options);
}

function modifyImportDeclaration(node, src, fileName, options) {
  const oldPath = node.moduleSpecifier.text;
  // only transform paths defined in our tsConfig
  const barrelPath = tsConfig.compilerOptions.paths[oldPath]?.[0];
  let barrelExports;
  if (barrelPath) {
    barrelExports = ensureBarrelIsCached(oldPath, barrelPath, options);
  }

  if (!barrelExports) {
    return src;
  }

  if (node.importClause && node.importClause.namedBindings) {
    var directImports = [];
    for (importNode of node.importClause.namedBindings.elements) {
      importNode = importNode.getFullText().trim();
      if (!barrelExports[importNode]) {
        barrelExports = invalidateAndReloadBarrelCache(oldPath, barrelPath, options);
        if (!barrelExports[importNode]) {
          console.warn('Import not found!', importNode, fileName);
          break;
        }
      }

      var relPath = path
        .relative(fileName, __dirname + '/' + barrelExports[importNode])
        .replaceAll('\\', '/')
        .replace('.ts', '')
        .replace('../', '');
      directImports.push(`import {${importNode}} from '${relPath}';`);
    }
    if (directImports.length) {
      var transformedFileSrc = src.replace(node.getFullText(), directImports.join('\r\n'));
      return transformedFileSrc;
    }
  }
  return src;
}

function processFile(src, filename, options) {
  var transformedFileContent = src;
  ts.transpileModule(src, {
    compilerOptions: tsConfig.compilerOptions,
    fileName: filename,
    transformers: {
      before: [
        context => {
          return sourceFile => {
            function visit(node) {
              if (ts.isImportDeclaration(node)) {
                transformedFileContent = modifyImportDeclaration(
                  node,
                  transformedFileContent,
                  filename,
                  options
                );
                // TODO find a way to replace the node with multiple import nodes instead
              }
              return ts.visitEachChild(node, visit, context);
            }
            return ts.visitNode(sourceFile, visit);
          };
        }
      ]
    }
  });
  return transformedFileContent;
}

function hasDebuggerAttached() {
  // We don't want to run when the debugger is attached, because it would throw off sourcemaps
  return !!process.env.VSCODE_INSPECTOR_OPTIONS;
}

module.exports = {
  process(src, fileName, options) {
    let newSrc = src;
    if (
      !hasDebuggerAttached() &&
      fileName.endsWith('.ts') &&
      !ignoreFiles.some(f => fileName.includes(f))
    ) {
      newSrc = processFile(src, fileName, options);
    }
    return baseTransformer.process(newSrc, fileName, {
      ...options,
      transformConfig: presetConfig
    });
  }
};

const ignoreFiles = ['test-setup', 'test-polyfills'];

[gtsop-d]

@mikerentmeister I have also embarked on this endeavor myself.

I have been building a babel plugin that replaces all the import statements to point directly to the original exporter. I have also modified the mock calls so that it applies to jest.mock calls as well. Our codebase is a quite large CRA with around 500 test files. It takes ~700 sec to run without this plugin. With the plugin it runs in around ~330 sec

In order for my plugin to be practical I need to modify the parser I'm spawning so that it dynamically uses the jest/babel configuration of the project. As of right now I have sort of hardcoded my project's config within my plugin

[gtsop]

Here is a first version of the plugin I mentioned above (as gtsop-d). It is tested on our CRA app (React 16, jest 27) and roughly gives a 50% speed boost. It is quite alpha but I hope it can help you, feel free to post any issues there to get it working better on your codebases. Ideally you can start using it without modifying your codebase at all

https://www.npmjs.com/package/@gtsopanoglou/babel-jest-boost

Edit: If you've used the plugin above successfully, or have trouble using it, please open an issue and i'll try to help you. I intend for it be an actual solution to this problem we are discussing here, not just a tailor-made solution for my codebase. Thanks 🙏🏻

AUG 13 Update: We have been successfully using this plugin in our production codebase since it's release (3+ months as of writing this). We have upgraded to react 18 and it still works great.

OCT 30 Update: I now consider this plugin production ready. We've been continually using it in an ever evolving codebase for many months and it has proven to be an actual solution to the problem discussed here.

[srosato]

@gtsop I have tried your plugin quickly and so far on my nx monorepo project that uses barrel files everywhere, the performance gain is massive!

[Oize]

@gtsop your license stops us from trying your plugin. As it will probably do for any commercial project.

[gtsop]

@Oize Not sure I can think of any use cases other than speeding up your own tests as part of your internal tooling, in which case the licence doesn't block you at all. If for some reason you wish to sell jest test runs as a service then maybe there is a problem there.

[Oize]

I mean, afaik, I have to make open source any repo that has your package as a dependency. And I definitely can't make open source my company's monorepo xD. Maybe I am reading it wrong, idk.

[TSMMark]

@gtsop I'd love to use your plugin at work as well, where our product code is closed source. Can you please update the license to MIT or something less restrictive? Thanks for your work on this — I'm genuinely extremely excited to try this out and see if it helps

[gtsop]

@TSMMark @Oize The licence does not prevent you from using it as long as it's for internal tooling, and let's not make this thread about licencing, mongodb was agplv3 up to 2018 and had thousands of commercial users, I highly doubt anyone has a case that would trigger the copyleft clause for this plugin

[life-engineered]

Hey, I am facing a similar issue, I use ts-jest in my config, does anyone have any ideas on how to improve the performace.

We have ~1800 test cases, which take anywhere between 15-20 mins

[FogelAI]

Hi everyone,
I have developed a Babel plugin that resolves performance issues caused by barrel files.
You can check it out here: https://github.com/FogelAI/babel-plugin-transform-barrels
The plugin supports Jest, Webpack and Vite.
If you're wondering when and why it's recommended to use this plugin, please take a look at the articles on Potential issues with barrel files in Webpack and Potential issues with barrel files in Jest.

[guilhermecastros]

@FogelAI do you mind scaffolding a project on codesandbox or codepen using the plugin?

I followed the readme file & tried to apply in my project, and I didn't see any improvement. I might be doing something wrong 😞

The project that I tried is using react + jest + vite.

[FogelAI]

Hi @guilhermecastros
Could you please share your Jest and Vite configurations? Then I can try to help you.
Is this a public project? If so, could you share the repository?

[guilhermecastros]

It is not public, but I have it like this:

• Jest

• Vite

[FogelAI]

Try this in Jest:

  "transform": {
    "^.+.tsx?$": [
      "ts-jest",
      {
        "isolatedModules": true,
        "babelConfig": {
          "plugins": [
            [
              "babel-plugin-transform-barrels", 
              {
                "executorName": "jest"
              }
            ]
          ]
        }
      }
    ],
  },

Let me know if it helps.

[de-don]

@FogelAI I have tried your lib, for my codebase it make test results slower ;D

NX monorepo with 120+ libs, tests 290s (with plugin) vs 250s (without)

Isolated modules enabled.

[gtsop]

As a re-cap to this mega-thread:

1. Barrel files cause jest runs to be slower, but it is not a bug. It is the expected behavior
2. Babel plugins CAN actually solve this problem by re-wiring imports and jest.mock calls as demonstrated by both @FogelAI's plugin and mine, which seem to have followed similar approaches. They've shown demonstrable performance results in large codebases.

https://github.com/gtsop/babel-jest-boost
https://github.com/FogelAI/babel-plugin-transform-barrels

Since this problem is officially out of scope for jest, I am wondering whether jest should add a reference in the documentation for this topic and how to approach it.