Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_runner,cli: mark test isolation as stable #56298

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 18 additions & 16 deletions doc/api/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -1025,20 +1025,6 @@ generated as part of the test runner output. If no tests are run, a coverage
report is not generated. See the documentation on
[collecting code coverage from tests][] for more details.

### `--experimental-test-isolation=mode`

<!-- YAML
added: v22.8.0
-->

> Stability: 1.0 - Early development

Configures the type of test isolation used in the test runner. When `mode` is
`'process'`, each test file is run in a separate child process. When `mode` is
`'none'`, all test files run in the same process as the test runner. The default
isolation mode is `'process'`. This flag is ignored if the `--test` flag is not
present. See the [test runner execution model][] section for more information.

### `--experimental-test-module-mocks`

<!-- YAML
Expand Down Expand Up @@ -2238,8 +2224,8 @@ added:
-->

The maximum number of test files that the test runner CLI will execute
concurrently. If `--experimental-test-isolation` is set to `'none'`, this flag
is ignored and concurrency is one. Otherwise, concurrency defaults to
concurrently. If `--test-isolation` is set to `'none'`, this flag is ignored and
concurrency is one. Otherwise, concurrency defaults to
`os.availableParallelism() - 1`.

### `--test-coverage-branches=threshold`
Expand Down Expand Up @@ -2323,6 +2309,22 @@ added:
Configures the test runner to exit the process once all known tests have
finished executing even if the event loop would otherwise remain active.

### `--test-isolation=mode`

<!-- YAML
added: v22.8.0
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/56298
description: Test isolation configuration is now stable.
-->

Configures the type of test isolation used in the test runner. When `mode` is
`'process'`, each test file is run in a separate child process. When `mode` is
`'none'`, all test files run in the same process as the test runner. The default
isolation mode is `'process'`. This flag is ignored if the `--test` flag is not
present. See the [test runner execution model][] section for more information.

### `--test-name-pattern`

<!-- YAML
Expand Down
6 changes: 3 additions & 3 deletions doc/node.1
Original file line number Diff line number Diff line change
Expand Up @@ -180,9 +180,6 @@ Use this flag to enable ShadowRealm support.
.It Fl -experimental-test-coverage
Enable code coverage in the test runner.
.
.It Fl -experimental-test-isolation Ns = Ns Ar mode
Configures the type of test isolation used in the test runner.
.
.It Fl -experimental-test-module-mocks
Enable module mocking in the test runner.
.
Expand Down Expand Up @@ -455,6 +452,9 @@ Require a minimum threshold for line coverage (0 - 100).
Configures the test runner to exit the process once all known tests have
finished executing even if the event loop would otherwise remain active.
.
.It Fl -test-isolation Ns = Ns Ar mode
Configures the type of test isolation used in the test runner.
.
.It Fl -test-name-pattern
A regular expression that configures the test runner to only execute tests
whose name matches the provided pattern.
Expand Down
2 changes: 1 addition & 1 deletion lib/internal/test_runner/utils.js
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ function parseCommandLine() {
}

if (isTestRunner) {
isolation = getOptionValue('--experimental-test-isolation');
isolation = getOptionValue('--test-isolation');
timeout = getOptionValue('--test-timeout') || Infinity;

if (isolation === 'none') {
Expand Down
8 changes: 4 additions & 4 deletions src/node_options.cc
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ void EnvironmentOptions::CheckOptions(std::vector<std::string>* errors,
debug_options_.allow_attaching_debugger = true;
} else {
if (test_isolation != "process") {
errors->push_back("invalid value for --experimental-test-isolation");
errors->push_back("invalid value for --test-isolation");
}

#ifndef ALLOW_ATTACHING_DEBUGGER_IN_TEST_RUNNER
Expand Down Expand Up @@ -684,10 +684,10 @@ EnvironmentOptionsParser::EnvironmentOptionsParser() {
"the line coverage minimum threshold",
&EnvironmentOptions::test_coverage_lines,
kAllowedInEnvvar);

AddOption("--experimental-test-isolation",
AddOption("--test-isolation",
"configures the type of test isolation used in the test runner",
&EnvironmentOptions::test_isolation);
&EnvironmentOptions::test_isolation,
kAllowedInEnvvar);
AddOption("--experimental-test-module-mocks",
"enable module mocking in the test runner",
&EnvironmentOptions::test_runner_module_mocks);
Expand Down
4 changes: 2 additions & 2 deletions test/parallel/test-runner-cli-concurrency.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,14 @@ test('concurrency of two', async () => {
});

test('isolation=none uses a concurrency of one', async () => {
const args = ['--test', '--experimental-test-isolation=none'];
const args = ['--test', '--test-isolation=none'];
const cp = spawnSync(process.execPath, args, { cwd, env });
assert.match(cp.stderr.toString(), /concurrency: 1,/);
});

test('isolation=none overrides --test-concurrency', async () => {
const args = [
'--test', '--experimental-test-isolation=none', '--test-concurrency=2',
'--test', '--test-isolation=none', '--test-concurrency=2',
];
const cp = spawnSync(process.execPath, args, { cwd, env });
assert.match(cp.stderr.toString(), /concurrency: 1,/);
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-runner-cli-timeout.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ test('timeout of 10ms', async () => {

test('isolation=none uses the --test-timeout flag', async () => {
const args = [
'--test', '--experimental-test-isolation=none', '--test-timeout=10',
'--test', '--test-isolation=none', '--test-timeout=10',
];
const cp = spawnSync(process.execPath, args, { cwd, env });
assert.match(cp.stderr.toString(), /timeout: 10,/);
Expand Down
18 changes: 9 additions & 9 deletions test/parallel/test-runner-cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ for (const isolation of ['none', 'process']) {
// File not found.
const args = [
'--test',
`--experimental-test-isolation=${isolation}`,
`--test-isolation=${isolation}`,
'a-random-file-that-does-not-exist.js',
];
const child = spawnSync(process.execPath, args);
Expand All @@ -27,7 +27,7 @@ for (const isolation of ['none', 'process']) {
// Default behavior. node_modules is ignored. Files that don't match the
// pattern are ignored except in test/ directories.
const args = ['--test', '--test-reporter=tap',
`--experimental-test-isolation=${isolation}`];
`--test-isolation=${isolation}`];
const child = spawnSync(process.execPath, args, { cwd: join(testFixtures, 'default-behavior') });

assert.strictEqual(child.status, 1);
Expand All @@ -46,7 +46,7 @@ for (const isolation of ['none', 'process']) {
{
// Should match files with "-test.(c|m)js" suffix.
const args = ['--test', '--test-reporter=tap',
`--experimental-test-isolation=${isolation}`];
`--test-isolation=${isolation}`];
const child = spawnSync(process.execPath, args, { cwd: join(testFixtures, 'matching-patterns') });

assert.strictEqual(child.status, 0);
Expand All @@ -64,7 +64,7 @@ for (const isolation of ['none', 'process']) {
for (const type of ['strip', 'transform']) {
// Should match files with "-test.(c|m)(t|j)s" suffix when typescript support is enabled
const args = ['--test', '--test-reporter=tap', '--no-warnings',
`--experimental-${type}-types`, `--experimental-test-isolation=${isolation}`];
`--experimental-${type}-types`, `--test-isolation=${isolation}`];
const child = spawnSync(process.execPath, args, { cwd: join(testFixtures, 'matching-patterns') });

if (!process.config.variables.node_use_amaro) {
Expand All @@ -91,7 +91,7 @@ for (const isolation of ['none', 'process']) {
'--require', join(testFixtures, 'protoMutation.js'),
'--test',
'--test-reporter=tap',
`--experimental-test-isolation=${isolation}`,
`--test-isolation=${isolation}`,
];
const child = spawnSync(process.execPath, args, { cwd: join(testFixtures, 'default-behavior') });

Expand All @@ -112,7 +112,7 @@ for (const isolation of ['none', 'process']) {
const args = [
'--test',
'--test-reporter=tap',
`--experimental-test-isolation=${isolation}`,
`--test-isolation=${isolation}`,
join(testFixtures, 'index.js'),
];
const child = spawnSync(process.execPath, args, { cwd: testFixtures });
Expand All @@ -129,7 +129,7 @@ for (const isolation of ['none', 'process']) {
const args = [
'--test',
'--test-reporter=tap',
`--experimental-test-isolation=${isolation}`,
`--test-isolation=${isolation}`,
join(testFixtures, 'default-behavior/node_modules/*.js'),
];
const child = spawnSync(process.execPath, args);
Expand All @@ -143,7 +143,7 @@ for (const isolation of ['none', 'process']) {

{
// The current directory is used by default.
const args = ['--test', `--experimental-test-isolation=${isolation}`];
const args = ['--test', `--test-isolation=${isolation}`];
const options = { cwd: join(testFixtures, 'default-behavior') };
const child = spawnSync(process.execPath, args, options);

Expand All @@ -165,7 +165,7 @@ for (const isolation of ['none', 'process']) {
const args = [
'--test',
'--test-reporter=tap',
`--experimental-test-isolation=${isolation}`,
`--test-isolation=${isolation}`,
'test/fixtures/test-runner/default-behavior/index.test.js',
'test/fixtures/test-runner/nested.js',
'test/fixtures/test-runner/invalid-tap.js',
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-runner-coverage.js
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,7 @@ test.skip('coverage works with isolation=none', skipIfNoInspector, () => {
'--experimental-test-coverage',
'--test-reporter',
'tap',
'--experimental-test-isolation=none',
'--test-isolation=none',
];
const result = spawnSync(process.execPath, args, {
env: { ...process.env, NODE_TEST_TMPDIR: tmpdir.path },
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-runner-extraneous-async-activity.js
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ const { spawnSync } = require('child_process');
{
const child = spawnSync(process.execPath, [
'--test',
'--experimental-test-isolation=none',
'--test-isolation=none',
fixtures.path('test-runner', 'async-error-in-test-hook.mjs'),
]);
const stdout = child.stdout.toString();
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-runner-force-exit-failure.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ for (const isolation of ['none', 'process']) {
'--test',
'--test-reporter=spec',
'--test-force-exit',
`--experimental-test-isolation=${isolation}`,
`--test-isolation=${isolation}`,
fixture,
];
const r = spawnSync(process.execPath, args);
Expand Down
8 changes: 4 additions & 4 deletions test/parallel/test-runner-no-isolation-filtering.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ test('works with --test-only', () => {
const args = [
'--test',
'--test-reporter=tap',
'--experimental-test-isolation=none',
'--test-isolation=none',
'--test-only',
fixture1,
fixture2,
Expand All @@ -35,7 +35,7 @@ test('works without --test-only', () => {
const args = [
'--test',
'--test-reporter=tap',
'--experimental-test-isolation=none',
'--test-isolation=none',
fixture1,
fixture2,
];
Expand All @@ -57,7 +57,7 @@ test('works with --test-name-pattern', () => {
const args = [
'--test',
'--test-reporter=tap',
'--experimental-test-isolation=none',
'--test-isolation=none',
'--test-name-pattern=/test one/',
fixture1,
fixture2,
Expand All @@ -75,7 +75,7 @@ test('works with --test-skip-pattern', () => {
const args = [
'--test',
'--test-reporter=tap',
'--experimental-test-isolation=none',
'--test-isolation=none',
'--test-skip-pattern=/one/',
fixture1,
fixture2,
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-runner-no-isolation-hooks.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import { test } from 'node:test';

const testArguments = [
'--test',
'--experimental-test-isolation=none',
'--test-isolation=none',
];

const testFiles = [
Expand Down
6 changes: 3 additions & 3 deletions test/parallel/test-runner-snapshot-tests.js
Original file line number Diff line number Diff line change
Expand Up @@ -349,7 +349,7 @@ test('snapshots from multiple files (isolation=none)', async (t) => {
await t.test('fails prior to snapshot generation', async (t) => {
const args = [
'--test',
'--experimental-test-isolation=none',
'--test-isolation=none',
fixture,
fixture2,
];
Expand All @@ -370,7 +370,7 @@ test('snapshots from multiple files (isolation=none)', async (t) => {
await t.test('passes when regenerating snapshots', async (t) => {
const args = [
'--test',
'--experimental-test-isolation=none',
'--test-isolation=none',
'--test-update-snapshots',
fixture,
fixture2,
Expand All @@ -391,7 +391,7 @@ test('snapshots from multiple files (isolation=none)', async (t) => {
await t.test('passes when snapshots exist', async (t) => {
const args = [
'--test',
'--experimental-test-isolation=none',
'--test-isolation=none',
fixture,
fixture2,
];
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-runner-watch-mode.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ async function testWatch({
const ran2 = Promise.withResolvers();
const child = spawn(process.execPath,
['--watch', '--test', '--test-reporter=spec',
isolation ? `--experimental-test-isolation=${isolation}` : '',
isolation ? `--test-isolation=${isolation}` : '',
file ? fixturePaths[file] : undefined].filter(Boolean),
{ encoding: 'utf8', stdio: 'pipe', cwd: tmpdir.path });
let stdout = '';
Expand Down
Loading