Jest should fail fast and exit early (change request for --bail) #6527
Comments
|
For those that want an immediate solution to this problem: #2867 |
|
The suggested solution works, as long as you're not using jest-circus, you have to still be using Jasmine under the hood. Is there any word on this functionality making it's way into jest-circus as an option? |
|
Hello, everyone, I'm looking forward to helping with this if needed 😊 My impression is that this is actually a bug rather than a feature request. It's also important to separate the two issues I see here:
I was able to reproduce (2) using describe("test bail", () => {
// These will run regardless of the success of `beforeEach` or any other hooks
beforeEach(() => { throw new Error("beforeEach hook fails!") });
test("first", () => expect(true).toBe(true));
test("second", () => expect(true).toBe(true));
test("third", () => expect(true).toBe(true));
});I think it's reasonable to expect that if For the sake of symmetry and coherence I think that the same should happen with I'd be happy to send a PR for solving (2) if the maintainers agree that should be the desired behaviour 💖 |
It might only make sense for I think my request is more of a feature request for circus. It'd be nice to be able to tell it to end a suite if a single test in it fails. My main use-case for this (which I'm using the workaround for with |
|
If We'll be removing Jasmine at some point (it'll stop being the default in the next major) so we're not really adding features to it, but happy to take a PR though if you wanna do the work :) And I agree |
|
@ALL @SimenB I'd love to try tackling this if possible 😊 If I understood correctly, as per your explanation here's a list of changes I intend to make if you agree they should be in:
A third one would be to make #2867 work in |
|
@lucasfcosta, given the above suggestions, would |
|
Just an update on this: As I was investigating today, it seems like @SimenB was correct in regards to the tests not running if the Running the following describe('a block', () => {
beforeEach(() => {
throw new Error('The beforeEach hook failed.');
});
test('skipped first test', () => {
throw new Error('First test should not have been run');
});
test('skipped second test', () => {
throw new Error('Second test should not have been run');
});
});With Given the explanation above, aborting tests in case the I haven't yet got to the item number 2 in this comment but I'll keep updates contained in this issue (preferably editing this post). PS: I'd rather report updates here since then I can link to the discussion in the PR and someone else can pick up the work if they need it, but if you think this is unnecessary noise in GH notifications just let me know and I'll avoid updates before the PR. |
Is this change to the |
|
I had a test case failing only when running after other test cases. I had to comment all following test cases to debug this because jest does not have an option to actually bail after 1 failed test case and not test suite. |
|
Running describe('a block', () => {
beforeEach(() => {
throw new Error('The beforeEach hook failed.');
});
test('skipped first test', () => {
console.log('XXX skipped first test');
// throw new Error('First test should not have been run');
expect(true).toBe(true);
});
test('skipped second test', () => {
console.log('XXX skipped second test');
// throw new Error('Second test should not have been run');
expect(true).toBe(true);
});
});Logs: This indicates to me that the tests are still run even though it says "skipped" |
|
It would be great to have a way to mark tests as dependent on previous step for e2e test as described here. I think the simplest implementation might be a |
|
Consider using "bail" config option: https://jestjs.io/docs/en/configuration#bail-number--boolean This works for cases when you'd want a fast fail in beforeAll / beforeEach setup blocks. |
|
Not the same as already explained several times in these Threads |
|
@seeruk I've got a
const JestEnvironmentNode = require("jest-environment-node");
class CustomEnvironment extends JestEnvironmentNode {
/**
* @param {import('jest-circus').Event} event
* @param {import('jest-circus').State} state
*/
async handleTestEvent(event, state) {
if (event.name === 'test_fn_failure' || event.name === 'hook_failure') {
// re-throw the error to exit immediately
throw new Error(event.error);
}
}
}
module.exports = CustomEnvironment;
module.exports = {
testRunner: 'jest-circus/runner',
testEnvironment: './CustomEnvironment.js',
…
}; |
|
Skip all after first failure Before: I run test with
const ParentEnvironment = require('jest-environment-node')
// or require('jest-environment-jsdom')
// or require('jest-playwright-preset/lib/PlaywrightEnvironment').default
class JestEnvironmentFailFast extends ParentEnvironment {
failedTest = false;
async handleTestEvent(event, state) {
if (event.name === 'hook_failure' || event.name === 'test_fn_failure') {
this.failedTest = true;
} else if (this.failedTest && event.name === 'test_start') {
event.test.mode = 'skip';
}
if (super.handleTestEvent) {
await super.handleTestEvent(event, state)
}
}
}
module.exports = JestEnvironmentFailFast
More events names in https://github.com/facebook/jest/blob/master/packages/jest-circus/src/run.ts, |
|
This |
|
@davidglezz Surely you mean await super.handleTestEvent(event, state)in your environment code, right? (Edit: Thanks for applying the fix!) |
|
Totally agree with #6527 (comment). Tests should stop running if |
|
For the solution posted by @davidglezz, it seems that [(for (edited as indicated to add clarification of context for the guidance) Footnotes
|
|
@vjjft in @davidglezz solution, it imports |
|
@zirkelc I was/am only working with |
|
In case someone using Jest with Typescript would like a TS native solution adapted from the #6527 (comment) by @davidglezz, this is working for me: import type { Circus, } from "@jest/types"
import { TestEnvironment, } from "jest-environment-node"
// import TestEnvironment from "jest-environment-jsdom"
class FailFastEnvironment extends TestEnvironment
{
failedTest = false
async handleTestEvent(
event: Circus.Event,
state: Circus.State,
)
{
if ( event.name === "hook_failure" || event.name === "test_fn_failure" )
{
this.failedTest = true
} else if ( this.failedTest && event.name === "test_start" )
{
event.test.mode = "skip"
}
// @ts-ignore
if ( super.handleTestEvent ) await super.handleTestEvent( event, state, )
}
}
export default FailFastEnvironment |
|
@vjjft it works perfectly ! clean and type-safe. thank you. |
|
This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days. |
|
Unstale |
|
Should there just be a |
|
Any updates on this? |
and others
🚀 Feature Proposal
Jest should not run subsequent hooks or tests associated with that hook if an exception is thrown in a hook.
Motivation
When setup/teardown is broken up into small chunks (open browser, navigate to url, login, etc) and one of those hooks fails, all subsequent hooks and tests may fail and the output is clogged.
Example
When using Jest with puppeteer, I need to do the following before running any tests:
If any of those steps fail, the subsequent steps and tests are likely to fail as well.
Pitch
I'm using jest because of the detailed reporter, especially when a test fails. But failure reports can become needlessly long when it's only the first item that needs to be fixed to prevent a long subsequent list of failures/errors
I do not believe this is considered a change to the default reporter, as --bail does not behave the way I expect it to (it does stop after the first failing test, but if the test has 5 hooks, it still tries to run subsequent hooks and then the test fails).
The text was updated successfully, but these errors were encountered: