Jest ESM - Total Guide To More Than 100% Faster Testing For Angular ⚡

Idea, Prompts, Composition & Design by Tomas Trajan, Gen by MindJourney

In my case, the final speed up was close to 290% from 50 seconds to only 17 seconds which is a huge win!

Finally made #jest #ESM work with #Angular including #CDK and harnesses, what an odyssey 😅😅😅

The last boss proved to be 'ProxyZone' and solution was to map to fesm2015 of CDK in the node_modules to downlevel async / wait 😅😅😅

Results are great though !

⌛ 50s -> 17s ⚡⚡ pic.twitter.com/OepIeMde6m

— Tomas Trajan 🇨🇭 (@tomastrajan) December 22, 2022

Jest runs tests in parallel, and we can even specify how many workers should be used to match the cores of our machine, which makes it really fast!

For these reasons, Jest became the go-to replacement for the Karma test runner which comes out of the box in Angular CLI workspaces.

This article focuses on Jest ESM integration in standard Angular CLI workspaces
using plain jest together with jest-preset-angular. There are many other ways
to use Jest with your Angular projects including @angular-builders/jest or out
of the box support in NX monorepo. That being said, the concepts and approaches
in this article should be helpful when trying to migrate or troubleshoot Jest ESM
in any of the above listed solutions!

ESM

When speaking about Jest ESM, we're talking about Jest working in mode where it
understands and uses EcmaScript Modules, especially the ESM import / export syntax.
This should be more than familiar as it's something we're using in Angular
TypeScript files since the beginning… (learn more)

import addDays from 'date-fns/addDays'; // ESM default import
import { Component } from '@angular/core'; // ESM import

@Component({
  /*...*/
})
export class AppComponent {} // ESM export

TLDR;

• Jest is fast until it isn't therefore we have to bother with ESM
• Basic Jest ESM setup is pretty straight forward
• There is an ideal Jest ESM scenario where it would just work if libraries implemented ESM correctly
• In real life Jest ESM breaks because most libs ship incorrect ESM
• The moduleNameMapper and transformIgnorePatterns are the main tools that will help is to fix all the Jest ESM problems
• We're going to progress through list of common problems and their solutions (please share yours to make it more complete in the comments)

Jest is fast, but somehow, it got really slow

One of the best things about Jest is that it's fast, especially compared to default Karma…

Why doesn't Angular team just replace Karma with Jest as a default test runner in the Angular CLI workspaces?

Jest build pipeline

As it turned out, Jest comes with it's own "standalone" build pipeline which is not really that modular. Because of that it is not really possible to make it consume an output produced by the something like ng build.

To provide Jest support, Angular team would have to maintain two completely separate build pipelines next to each other. This hopefully provides insight on their decision to NOT support Jest out of the box and why it makes sense!

This all means that the Jest setup for Angular project needs to be able to do basically everything that the Angular CLI does for ng build and more. It should also provide us with an understanding why the things are as complex as they are!

Besides that, it underlines the fact that most of these problems (and our need to deal with them) are caused by the architecture of Jest which makes it impossible to play nicely with other tooling (such as Angular CLI build pipeline), please keep that in mind as we sink deeper into the depths of this configuration pit! 😅

Dealing with complexity

Setting up separate Jest build pipeline to run ngcc for the libraries or compile Angular component including their templates would be pretty annoying, that's why we have libraries like jest-preset-angular which hides at least some of this complexity!

With Jest and the jest-preset-angular in place, the Jest Angular testing in most of the projects "just worked" and was pretty fast out of the box, at least until the release of Angular 12!

Jest testing of Angular 12 (and higher) got dramatically slower because Angular stopped shipping UMD bundles which were used by the Jest. This is because Jest now needs to additionally transpile Angular (and other Angular libs) in its own build pipeline

Angular 12 and the major Jest slowdown

The release of Angular 12 brought major update to the Angular Package Format and therefore to the way the Angular libraries are shipped. Check out the relevant Angular changelog entries:

• UMD bundles are no longer generated
• UMD bundles are no longer shipped in the Angular NPM packages

This means that if we explored node_modules of one of our Angular projects, we will see something like the following…

node_modules/
  @angular/
    core/
      // no more umd/ folder !!!
      esm2020/
      esm2015/
      fesm2020/  // folder
        core.mjs // js bundle with ESM import / export syntax and .mjs ext
      fesm2015/
        core.mjs

      // other sub-entries like testing/...
      package.json
      index.d.ts

Starting with version 12, Angular no longer ships UMD bundles and provides only ESM bundles instead!

This brings us back to the separate build pipeline of the Jest which works in the CommonJS (CJS) mode out of the box and ESM support is something that needs to be enabled and configured to make it work.

Because Angular 12 no longer delivers UMD bundles, if we tried to run our Jest test suite just after the upgrade to Angular 12, we would encounter a new error…

SyntaxError: Cannot use import statement outside a module

Jest encountered an unexpected token

Jest failed to parse a file. This happens e.g. when your code or
its dependencies use non-standard JavaScript syntax,
or when Jest is not configured to support such syntax.

Out of the box Jest supports Babel, which will be used to transform
your files into valid JS based on your Babel configuration.

By default "node_modules" folder is ignored by transformers.

It was pretty easy to make this work by adjusting the basic Jest setup. In particular, we had to add @angular (or even better .*\.mjs) to the transformIgnorePatterns (more on that later) of the Jest configuration file.

This would lead to a setup where Jest would first need to transpile whole Angular library from node_modules from ESM to CJS to be able to consume it without any issue.

This would apply not only to the Angular itself but to all other libraries built with Angular 12 like Angular Material or any other custom internal libraries which are pretty common in enterprise organizations.

---

Follow me on Twitter because that way you will never miss new Angular, NgRx, RxJs and NX blog posts and other cool frontend stuff!😉

---

Jest ESM basic setup

Now that we understand why default Jest setup got slower in the Angular CLI workspaces, let's explore how to make it fast again by enabling Jest ESM support!

As we stated previously, we're using jest-preset-angular to help us with the complexities of Jest / Angular integration.

This great library delivers also ESM focused presets which can be enabled
with ease. More so, there are plenty of examples in the library GitHub
repository which we can use as an inspiration on how exactly to set up
Jest in our Angular CLI workspaces!

Let's start at the beginning by installing the relevant dependencies
(you might need to check jest-preset-angular changelog to figure out
which version is compatible with your current version of Angular)

npm i -D jest @types/jest jest-preset-angular

Next, we can add a following npm script to our main package.json file.

{
  "scripts": {
    "test": "node --experimental-vm-modules --no-warnings node_modules/jest/bin/jest.js --config src/jest.config.mjs"
  }
}

Instead of just running jest as we would usually, we're running it explicitly
with node and setting multiple environment flag which might no longer be
necessary based on the version of node you're using in your environment.

Besides that, we're pointing to our Jest config file, in this case a src/jest.config.mjs.

Next, let's focus on the content of the Jest config file…

export default {
  // use esm preset (from jest-preset-angular )
  preset: 'jest-preset-angular/presets/defaults-esm',

  // use esm global setup (from jest-preset-angular)
  globalSetup: 'jest-preset-angular/global-setup.mjs',

  extensionsToTreatAsEsm: ['.ts'],

  // another setup file which we will create in the next step
  setupFilesAfterEnv: ['<rootDir>/jest.setup.ts'],

  globals: {
    'ts-jest': {
      // path might be different based on your workspace setup
      // <rootDir> represents the location of jest.config.mjs
      tsconfig: '<rootDir>/tsconfig.spec.json',
      stringifyContentPathRegex: '\\.(html|svg)$',
      useESM: true,
    },
  },

  // more on this later
  moduleNameMapper: {
    // eg when importing symbol (tslib) use content of the file (path)
    tslib: 'tslib/tslib.es6.js',
  },

  // perf (you might try various options based on the available cores)
  maxWorkers: '8',
};

The comments in the code example are hopefully self-explanatory,
basically, we're using ESM based presets from the jest-preset-angular
library and overriding the underlying ts-jest library config by setting
the useESM: true configuration property.

The last part of this basic setup is to introduce referenced jest.setup.ts file
which is in practice used to do things like provide global mocks
(eg for window.matchMedia or import some global libraries
which can make testing and debugging more pleasurable experience
like @angular-extensions/pretty-html-log ).

That being said, basic content will look like this…

import 'jest-preset-angular/setup-jest.mjs';

And that's it, that's the Jest ESM setup! Unfortunately things are not going to turn out as easy as they seem…

With this setup in place, the Jest will be able to consume Angular 12 ESM bundles ( .*\.mjs files ) natively without
the need to transpile them.

In the ideal world, Jest ESM would just work

As we have seen, basic Jest ESM setup is not really that wild, couple of config files, a strange flag or two, but all in all nothing too out of the ordinary…

If every library shipped correct ESM bundles (the way Angular does), this would have been all the config that is needed. Jest ESM would just pick up correct ESM bundle in the library folder in the node_modules,
and we would just enjoy great test performance!

First, were going to describe what such a correctly shipped ESM based library looks like (basically what Angular is already doing) and then we will explore all the cases where this goes wrong and how to fix them…

Ideal ESM library scenario

Disclaimer: The following is a rather limited understanding of ESM based on the experience of how the modules are being resolved by Jest and might differ from the ESM specification itself, please, feel free to point out additional resources in the comment to make this even better!

The list of scenarios bellow follows the way the Jest resolves which file should be loaded (imported) when it encounters an import statement in our source code.

In the ideal world, a library which delivers ESM based bundles would do one of the following:

• library ships ESM only and has "type": "module" in its package.json file while using .js extension for the ESM bundle files
• library ships ESM only and does NOT have "type": "module" in its package.json file while using .mjs extension for the ESM bundle files
• ships both ESM and CJS (and or UMD) bundles and uses correct extension per bundle type, namely .mjs for ESM and .cjs (or .js) for CJS
• library ships both ESM and CJS (and or UMD) bundles and has "type": "module" in its package.json and .js for ESM and .cjs for CJS

As we can see, there are plenty of options and all of them would work with Jest out of the box but unfortunately, many libraries do things differently.

This is understandable as the whole CJS to ESM migration in the node ecosystem is pretty messy and hard to get right in the first iteration, hopefully things will improve in the future where most libs start shipping ESM only or at least correct ESM.

---

Our toolbox

Before we go through the list of common issues and their solutions, we are going to take a quick detour to explore two main tools that will help us to fix all the problems that we will encounter.

The tools are the two configuration properties of the jest.config.mjs file, namely the moduleNameMapper and the transformIgnorePatterns.

The moduleNameMapper

This configuration property allows us to override which file is loaded when we try to import some symbol from a library in our source code.

For example when we try to import something like…

import { Observable } from 'rxjs';

What happens behind the scenes is that the module resolution mechanism will go to the node_modules/ folder and try to find rxjs/ folder.

After that, it expects to find a package.json file which will point to actual JavaScript files based on the provided configurations like exports map which will then point to something like "es2015": "./dist/esm/index.js".

The moduleNameMapper property in the jest.config.mjs allows us to override this behavior by pointing to a different file for such import.

For example…

{
  moduleNameMapper: {
    '^rxjs(/operators$)?$': '<rootDir>../../node_modules/rxjs/dist/bundles/rxjs.umd.js',
  }
}

In the example above we're mapping rxjs imports to the dist/bundles/rxjs.umd.js
instead of dist/esm/index.js which would have been used otherwise.
This might look confusing because we're trying to make it work with Jest ESM,
but we're going to discuss this specific case later.

The moduleNameMapper safety tip

The property itself represents a regular expression, and it is usually good idea to make it as explicit as possible, eh by using ^ and $ as not doing so could lead to a hard to debug problem.

For example, if we provided a rule with just rxjs,
it would match any import which would contain rxjs string such as
my-custom-rxjs.operators.ts which would then lead to errors like
Can't import myCustomOperator from my-custom-rxjs.operators.ts as the symbol was not exported.
This is because instead of importing our file, the import would be resolved
to that dist/bundles/rxjs.umd.js!

The transformIgnorePatterns

The second tool in our toolbox is the transformIgnorePatterns configuration property.

This one is pretty unintuitive, at least on the first look, as the real use case involves double negation.

The default value could be something like…

{
  transformIgnorePatterns: ['node_modules/'];
}

This means we DON'T want to transform anything inside the node_modules/ folder which makes sense as those libraries have already been pre-built.

The real config comes as the second negation where we "ignore things from the initially ignored pattern" with the help of regexp negative lookahead expression ?!.

Let's see an example

{
  transformIgnorePatterns: [
    // ignore everything in node_modules besides the cases when:
    // 1. it has tslib in its path
    // 2. the file ends with.mjs
    'node_modules/(?!(tslib|.*.mjs)',
  ];
}

The snipped above is something which we would use for Jest config to make it work with Angular 12 without the need to do the Jest ESM migration.

In general, we will use the transformIgnorePatterns to enable transpilation of some of the libraries which are not possible to handle in any other way. Every such library will have a negative impact on the overall testing performance.

Because of this, it's very important to regularly check if these libraries can be removed from the transformIgnorePatterns exclusion as they release new versions which might deliver proper ESM!

Great, now we should be familiar with the tools at our disposal and now is the time to explore all the common problems and their solutions when setting up Jest ESM for in our Angular CLI workspaces!

---

---

Problems and solutions

Lib doesn't have ESM (or is invalid) and has valid UMD / CJS

The most common example of this problem is rxjs. RxJs does deliver esm folders and bundles but unfortunately, those files have:

• .js extension
• package.json does NOT have "type": "module"

These two conditions add up to a situation where Jest expects that the content of these files is CJS (and or UMD) which is not the case and the process fails with the SyntaxError: Cannot use import statement outside a module.

To fix this, we can add the following entries to the moduleNameMapper.

{
  moduleNameMapper: {
    '^rxjs(\/operators)?$': '<rootDir>../../node_modules/rxjs/dist/bundles/rxjs.umd.js',
    '^rxjs/testing$': '<rootDir>../../node_modules/rxjs/dist/cjs/testing/index.js',
  }
}

The moduleNameMapper points to the RxJs UMD bundles which contain expected content.

Packages which ship correct UMD will work as long as moduleNameMapper points to a correct file!

Lib has invalid ESM AND invalid UMD

Sometimes libraries don't follow the way Jest resolves import at all which leads to a situation where we can't just point to a correct ESM (or UMD) bundle.

In that case, it is best to use moduleNameMapper to point to a bundle which uses ESM import / export syntax AND exclude the library from the transformIgnorePatterns.

Example of such library can be the tslib or @googlemaps/markerclusterer.

moduleNameMapper: {
    tslib: 'tslib/tslib.es6.js', // didn't figure out why this one works without full path
    '@googlemaps/markerclusterer': '<rootDir>../../node_modules/@googlemaps/markerclusterer/dist/index.esm.js',
  },
  transformIgnorePatterns: [
    'node_modules/(?!(tslib|@googlemaps/markerclusterer))',
  ],

With such setup, were pointing Jest to files which contain ESM and because the Jest would expect them to contain CJS because the way they are resolved we also have to add them to the transformIgnorePatterns.

Lib has valid ESM but invalid package.json configuration

Didn't really encounter this one yet but the solution would
be to just point Jest to a correct file with the help of moduleNameMapper.
For example, if a lib would deliver index.mjs which contains correct ESM
it should be enough to solve this with the help of moduleNameMapper.

{
  moduleNameMapper: {
    '^some-lib$': '<rootDir>../../node_modules/some-lib/index.mjs',
  }
}

Great! Now we know how to solve 3 most common generic problems caused by the configuration (and or distribution) problems of the libraries which try to ship ESM

---

The next problem is much more Angular specific because it's related to the way Angular uses Zone.js to handle change detection which clashes with the async / await syntax used by the popular Angular CDK test harness APIs…

The dreaded ProxyZone error

These problems happen in projects that are using Angular CDK test harnesses in conjunction to async / await syntax to write their own tests!

Running such tests with Jest ESM will yield the dreaded Expected to be running in 'ProxyZone', but it was not found. error.

Example of such test could look like this…

it('should check the checkbox if we set the value to true', async () => {
  component.initialValue = true;
  fixture.detectChanges();

  const checkbox = await loader.getHarness<MyOrgCheckboxHarness>(
    MyOrgCheckboxHarness,
  );

  expect(await checkbox.checked()).toBe(true);
});

The very reduced version of an explanation why this happens is that whenever
we await result of async operation in our test source code, what ever
happens after (on the next line) will happen in another zone which leads
to that error.

First step to solve this problem is to make sure that we're using
"target": "es2015" in our tsconfig.spec.json file which will cause
our test code to be down-leveled to that JavaScript language version
which was BEFORE introduction of async / await syntax which will in
practice lead to its replacement by a polyfill.

This adjustment solves the problem in our test code but the ProxyZone error remains!?

As it turns out, Jest ESM will consume @angular/cdk/fesm2020/cdk.mjs
instead of @angular/cdk/fesm2015/cdk.mjs even though we overridden it
in the tsconfig.spec.json (not completely sure why that is) and the
es2020 file of course contains plain async / await which leads to the error.

The solution to the second part of the problem is to override it in the moduleNameMapper!

{
  moduleNameMapper: {
    '^@angular/cdk$': '<rootDir>../../node_modules/@angular/cdk/fesm2015/cdk.mjs',
    '^@angular/cdk/testing$': '<rootDir>../../node_modules/@angular/cdk/fesm2015/testing.mjs'
  }
}

That way, the Jest ESM is going to consume files with downleveled
async / await and the ProxyZone error finally disappears!

Honorable mention: The moduleNameMapper matching of unexpected files

As mentioned previously, always be careful with properties
of the moduleNameMapper as they can lead to accidental matching
of your own source files and hard to trace bugs! Especially with
a short names like rxjs which can easily be part of the file
names in your source code.

You can always mitigate this risk by using ^ and $ to limit what gets matched by the regexp.

Honorable mentions: The "does not provide export error"

Some specific combination of versions of jest, jest-preset-angular, typescript and ts-jest can lead to the SyntaxError: The requested module does not provide an export name X error.
To solve this, you can employ workaround which involves including all .ts files in the test
tsconfig.spec.json file.

{
  "include": ["**/*.ts", "..."]
}

---

Real life results

After all of these fixes, I have achieved 53 seconds to 17 seconds speedup when running the whole test suite with Jest ESM which amounts to ~290% speed increase! ⚡

Embrace the speed!

Great! I hope you enjoy learning about how to significantly speed up your test suites in Angular CLI workspaces with Jest ESM and jest-preset-angular.

The nature of these problems means that they are pretty individual to the specific combination of used versions and libraries for a given project and therefore these approaches should be useful also when trying to make Jest ESM work when using other libraries such as @angular-builders/jest.

In general these approaches will help you to troubleshoot and fix problems as long as you have access and are able to override Jest config, especially the moduleNameMapper and the transformIgnorePatterns config properties.

Don't hesitate to ping me in the comments about the speed up you have been able to achieve by implementing these ideas! 😉

Also, don't hesitate to ping me if you have any questions using the article responses or Twitter DMs @tomastrajan

And never forget, future is bright

Obviously the bright Future! (📸 by Tomas Trajan )