Future of fake timer testing in a zoneless world

[yjaaidi]

Which @angular/* package(s) are relevant/related to the feature request?

core

Description

In order to test scenarios that involve timers (e.g. a component that uses setTimeout or setInterval), Angular provides means to manipulate time in tests using the fakeAsync function (+ associated functions like tick, flush, flushMicrotasks etc...).

While this approach offers a testing framework agnostic approach of manipulating time, it has the following limitations:

• It requires Zone.js, so by definition, it is not zoneless-ready.
• As it requires Zone.js, it also requires async/await to be downleveled during the build process.
• It doesn't provide a way of controlling the clock (e.g. set current date, then tick and finally observe the date change).
• It requires the system under test to be executed in a fake zone, which can make it more challenging to integrate with some untraditional testing patterns.

The most important point here is that this is not zoneless-ready.

---

• Proposed Solutions and Workarounds
  • Source code
  • 1. Using 3rd party fake timers
    • 1.1. Using 3rd party fake timers + "real" requestAnimationFrame
      • Pros & Cons
    • 1.2. Using 3rd party fake timers + fake rAF
      • Pros & Cons
  • 2. Using a microtask change detection scheduler for testing
    • Pros & Cons
  • 3. Introducing a Timer service
    • Pros & Cons
  • 4. Avoiding fake timers using configurable delays
• Comparison
• Side notes

Proposed Solutions and Workarounds

Source code

Examples of most of the different proposed solutions and alternatives presented below can be found here: https://stackblitz.com/edit/angular-zoneless-testing?file=README.md,src%2Fzoneless-fake-timers.spec.ts

1. Using 3rd party fake timers

The most common fake timer around is @sinonjs/fake-timers. Previously known as lolex. It is used under the hood in Jest, Vitest, etc...

The solutions below are based on this library as an example but in general, any fake timer library can be used and the same principles & problems apply. 😅

1.1. Using 3rd party fake timers + "real" requestAnimationFrame

The first problem we have is that we can't make any general assumption on what which functions will be faked by a fake timer library.

Just as an example, the default configurations can vary:

Tool	Default configuration for faking rAF
@sinonjs/fake-timers	true
Jest	true
Vitest	false

By making sure that rAF is not faked (which is harder than one might think, cf. below), users can rely on ComponentFixture#whenStable() making the test look something like this:

vi.useFakeTimers({ toFake: ['setTimeout'] });

test('should say hello after 1s', async () => {
  const fixture = TestBed.createComponent(GreetingsComponent);

  await vi.advanceTimersByTimeAsync(1000);

  await fixture.whenStable();

  expect(fixture.nativeElement.textContent).toBe('Hello!');
});

Warning

Given vitest defaults of not faking rAF, it might feel that vi.useFakeTimers() would be enough but it's not!
The problem is that the result will vary depending on the testing environment you are using (real browser, happy-dom, jsdom, etc...).

For example:
jsdom is using setInterval under the hood to emulate rAF (i.e. setInterval(callback, 1000 / 60)) but as setInterval is faked by default, rAF will be faked as well.
happy-dom uses setImmediate to emulate rAF. Luckily it grabs the real setImmediate function before it is faked so it will always work.

This means that while Angular could detect if rAF is faked and provide a warning, it would still not be enough.
Angular would have to implement heuristics to detect environments like jsdom where the provided implementation relies on setInterval and provide a warning if setInterval is faked in that case.

This also makes it impossible to fake both setTimeout and setInterval when using jsdom (except if jsdom is changed similarly to happy-dom in order to grab the original setInterval before it is faked).

Warning

By default Jest also fakes queueMicrotask which is used by Angular to schedule microtasks.

Users must make sure that queueMicrotask is not faked when using Jest by adding it to the doNotFake option.

Pros & Cons

• 👍 The test in itself is straightforward
• 👍 This doesn't require any framework changes.
• 🛑 By relying on rAF only to schedule change detection, tests will be a bit slower.
• 🛑 This approach is very fragile as it relies on the fact that rAF is not faked.
• 🛑 This is not perfectly symmetric to production as this is equivalent of implementing a change detection scheduler that only relies on rAF. This means that there are extreme cases where some macrotasks scheduled by the user could happen before the CD triggered by rAF while this wouldn't happen in production because CD would setTimeout would win the race. That being said, this is a really rare case that probably reflects some poor design choice.

1.2. Using 3rd party fake timers + fake rAF

With a fake timer that also fakes rAF, the app will simply never be stable as long as we don't flush the fake timers.

In other words, the following test will timeout:

vi.useFakeTimers({ toFake: ['setTimeout', 'requestAnimationFrame'] });

test('will timeout', async () => {
  const fixture = TestBed.createComponent(GreetingsComponent);

  await fixture.whenStable(); // this never resolves
});

More interestingly, (given the same fake timer configuration), the following test will also timeout:

test('will timeout', async () => {
  const fixture = TestBed.createComponent(GreetingsComponent);

  await vi.advanceTimersByTimeAsync(1000);
  await fixture.whenStable(); // this never resolves
});

while this one will pass:

test('will pass', async () => {
  const fixture = TestBed.createComponent(GreetingsComponent);

  await vi.advanceTimersByTimeAsync(1001);
  await fixture.whenStable();
  expect(fixture.nativeElement.textContent).toBe('Hello!');
});

Here is why:

• the call to vi.advanceTimersByTimeAsync(1000) will trigger the callback from the setTimeout(callback, 1000) in the component,
• by updating a signal, the callback, will notify the change detection scheduler to schedule a change detection cycle,
• this cycle will be scheduled with a race between setTimeout and rAF which are faked and will only be triggered if we advance by an extra millisecond.

This extra millisecond is introduced on purpose by @sinonjs/fake-timers (probably in order to provide more control when timers are "nested" or "chained").

Cf. https://github.com/sinonjs/fake-timers#:~:text=If%20called%20during%20a%20tick%20the%20callback%20won%27t%20fire%20until%201%20millisecond%20has%20ticked%20by.

& https://github.com/sinonjs/fake-timers/blob/341203310225bf5cd3d7396b2fcf276c5e218347/src/fake-timers-src.js#L645C58-L645C68

While there are workarounds to avoid this freeze, like advancing the timer millisecond by millisecond and checking the stability, they are not necessarily straightforward nor intuitive:

async function advanceTimeAndWaitStable(duration = 0) {
  await vi.advanceTimersByTimeAsync(duration);
  while (!fixture.isStable()) {
    await vi.advanceTimersByTimeAsync(1);
    duration++;
  }
  return duration;
}

If rAF is faked, users need a deep understanding of the fake timer library and zoneless change detection scheduling.

Pros & Cons

• 👍 This solution is less fragile than the previous one as it doesn't require the user to make sure that rAF is not faked.
• 👍 Tests are faster than using "real" rAF (Cf. solution 1.1.)
• 👍 This doesn't require any framework changes.
• 👍 This is a bit more symmetric to production than the previous solution.
• 🛑 Tests have to somehow advance the timer to trigger CD and therefore they are less straightfoward than a simple call to ComponentFixture#whenStable()
• 🛑 The extra millisecond can be annoying.

2. Using a microtask change detection scheduler for testing

By providing a microtask change detection scheduler instead of the zoneless one (setTimeout & rAF race), tests are more straightforward as extra change detection cycles will be triggered in the same timer tick (so no extra 1ms introduced by @sinonjs/fake-timers).

TestBed.configureTestingModule({
  providers: [
    ɵprovideZonelessChangeDetection(),
    {
      provide: ɵChangeDetectionScheduler,
      useExisting: TestingChangeDetectionScheduler,
    },
  ],
});

await vi.advanceTimersByTimeAsync(1000);
await fixture.whenStable();

expect(fixture.nativeElement.textContent).toBe('Hello!');

Pros & Cons

• 👍 Tests are straightforward.
• 👍 Timers are precise.
• 👍 Tests are faster than using "real" rAF (Cf. solution 1.1.)
• 👍 This solution is less fragile as it doesn't require the user to make sure that rAF is not faked.
• 🛑 This is not symmetric to production.

3. Introducing a Timer service

The abstraction + dependency injection approach in Angular has proven to be very powerful. Why not use it here too?

Angular could introduce a Timer service wrapping native timers (i.e. setTimeout, setInterval) + an alternative fake implementation for testing.

TestBed.configureTestingModule({
  providers: [provideTestingTimer()],
});
const fakeTimer = TestBed.inject(Timer);
await fakeTimer.advanceBy(1000);
await fixture.whenStable();

This could also be name something like TaskScheduler and also wrap other scheduling functions like requestIdleCallback, etc...

Pros & Cons

• 👍 Tests are straightforward.
• 👍 Timers are precise.
• 👍 Tests are faster than using "real" rAF (Cf. solution 1.1.)
• 👍 This solution is less fragile as it doesn't require the user to make sure that rAF is not faked.
• 👍 This solution is testing framework agnostic.
• 🛑 This can require major changes in existing code base and will not work with 3rd party code using setTimeout internally.

4. Avoiding fake timers using configurable delays

The most testing framework agnostic approach is to simply avoid using fake timers and instead use configurable delays.

The test could look something like this:

TestBed.configureTestingModule({
  providers: [{provide: MY_GREETINGS_MESSAGE_DELAY, useValue: 5}],
});
const fixture = TestBed.createComponent(GreetingsComponent);
await new Promise<void>(resolve => setTimeout(resolve, 5));
await fixture.whenStable();
expect(fixture.nativeElement.textContent).toBe('Hello!');

Comparison

Proposed Solution	Simplicity	Test Symmetry	Robustness	Exhaustiveness*	Speed	Testing Framework Agnosticity
3rd party fake timers + real rAF	⭐️	⭐️	-	⭐️	-	-
3rd party fake timers + fake rAF	-	⭐️	⭐️	⭐️	⭐️	-
Microtask Change Detection Scheduler	⭐️⭐️	-	⭐️⭐️	⭐️	⭐️	-
Introducing a Timer service	⭐️	⭐️⭐️	⭐️⭐️	-	⭐️	⭐️
Avoiding fake timers using configurable delays	⭐️	⭐️⭐️	⭐️⭐️	-️	⭐️	⭐️️

*Exhaustiveness: how many edge cases are covered by the solution.

Side notes

• It would be safer to use Promise.resolve().then(callback) instead of queueMicrotask(callback) for Angular internals to avoid fake timers interference.
• Solutions 2 & 3 can also help with SSR & stability in general.

❤️ Special thanks to @alxhub and @atscott for your patience, inspiration and ideas. 🙏

[atscott]

By the way, these problems generally go away if the fake timer library has a way to enable automatically advancing time (jasmine/jasmine#2042, jestjs/jest#15300)

[zargham-leanix]

@yjaaidi While migrating our existing jest tests to vitest we run into this problem, tick, flush inside fakeAsync(async =>...) does not work. What would be the recommended solution for the time being?

[yjaaidi]

Hi @zargham-leanix, what is the exact error you are getting?

Could you please check the following:

• Make sure @analogjs/vite-plugin-angular/setup-vitest is imported in test-setup.ts
• Make sure you use test and it globals instead of importing them from vitest. (the globals are those monkey patched by the angular plugin to work with zones)

[zargham-leanix]

@yjaaidi I was wrong, the issue had nothing to do with Angular, rather vite-plugin-angular and it is already fixed by analogjs/analog#1467

[yjaaidi]

Ah indeed! 3rd party libs' async/await usages are not downgraded.
I'll publish something about fakeAsync soon but meanwhile, I think it should be avoided as it relies totally on Zone.js + advancing microtasks synchronously is conceptually problematic.

I'd recommend using vi.useFakeTimers() + vi.advanceTimersByTimeAsync().