Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_runner: emit test-only diagnostic warning #46540

Merged
merged 1 commit into from
Feb 21, 2023
Merged

test_runner: emit test-only diagnostic warning #46540

merged 1 commit into from
Feb 21, 2023

Conversation

richiemccoll
Copy link
Contributor

@richiemccoll richiemccoll commented Feb 7, 2023 •
edited
Loading

Fixes: #46448

This PR:

  • emits a test diagnostic when running tests with { only: true } or runOnlyTests(true) without the --test-only CLI flag.

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added dont-land-on-v14.x needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Feb 7, 2023
@richiemccoll richiemccoll marked this pull request as ready for review February 7, 2023 10:02
@richiemccoll richiemccoll force-pushed the test-runner/emit-only-warning branch from 3ee8044 to e9bf7c1 Compare February 7, 2023 15:10
cjihrig
cjihrig reviewed Feb 8, 2023
View reviewed changes
lib/internal/test_runner/test.js Show resolved Hide resolved
lib/internal/test_runner/test.js Outdated Show resolved Hide resolved
test/message/test_runner_output.out Outdated Show resolved Hide resolved
@richiemccoll richiemccoll force-pushed the test-runner/emit-only-warning branch 2 times, most recently from 8292f1b to 514fd0b Compare February 9, 2023 12:11
@richiemccoll richiemccoll force-pushed the test-runner/emit-only-warning branch from 514fd0b to 5669292 Compare February 9, 2023 12:28
MoLow
MoLow approved these changes Feb 9, 2023
View reviewed changes
Copy link
Member

@MoLow MoLow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, But I think we should emit a similar diagnostic when calling runOnly when no flag

@richiemccoll
Copy link
Contributor Author

richiemccoll commented Feb 10, 2023 •
edited
Loading

I think we should emit a similar diagnostic when calling runOnly when no flag

@MoLow Are you referring to this scenario with runOnly below or something else? I've tested this locally and the diagnostic is there.

// running without the --test-only flag
const { test } = require('node:test');

test('top level test', async (t) => {
  t.runOnly(true);
  await t.test('this subtest is run');
});
TAP version 13
# Subtest: top level test
    # Subtest: this subtest is run
    ok 1 - this subtest is run
      ---
      duration_ms: 4.77543
      ...
    # 'only' and 'runOnly' require the --test-only command-line option.
    1..1
ok 1 - top level test
  ---
  duration_ms: 6.194772
  ...
1..1
# tests 1
# pass 1

@cjihrig
Copy link
Contributor

cjihrig commented Feb 11, 2023

I think we should emit a similar diagnostic when calling runOnly when no flag

I think @MoLow is saying to add the diagnostic when runOnly() is called, even if no tests are started. If this change is made, please ensure we don't end up in a situation where the diagnostic is included multiple times for the same test.

@richiemccoll
Copy link
Contributor Author

@cjihrig @MoLow Is that something you'd like to see as part of this PR or can that be tackled in a follow up commit?

@MoLow
Copy link
Member

MoLow commented Feb 13, 2023

Is that something you'd like to see as part of this PR or can that be tackled in a follow up commit?

A follow-up PR is ok

@MoLow
Copy link
Member

MoLow commented Feb 18, 2023

@richiemccoll this PR requires a rebase

@richiemccoll richiemccoll force-pushed the test-runner/emit-only-warning branch from 5669292 to 7696218 Compare February 20, 2023 09:29
@MoLow MoLow added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 20, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 20, 2023
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/49781/

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/49790/

@github-actions github-actions bot mentioned this pull request Feb 21, 2023
Open
26 tasks
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/49814/

@MoLow MoLow added the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 21, 2023
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 21, 2023
@nodejs-github-bot nodejs-github-bot merged commit c90ea93 into nodejs:main Feb 21, 2023
@nodejs-github-bot
Copy link
Collaborator

Landed in c90ea93

@github-actions github-actions bot mentioned this pull request Feb 22, 2023
Open
34 tasks
MoLow pushed a commit to MoLow/node that referenced this pull request Feb 25, 2023
@richiemccoll @MoLow
test_runner: emit test-only diagnostic warning
3983267 
PR-URL: nodejs#46540
Fixes: nodejs#46448
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
MoLow pushed a commit to MoLow/node that referenced this pull request Feb 25, 2023
@richiemccoll @MoLow
test_runner: emit test-only diagnostic warning
0c8f40d 
PR-URL: nodejs#46540
Fixes: nodejs#46448
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@MoLow MoLow added the backport-open-v18.x Indicate that the PR has an open backport. label Feb 25, 2023
MoLow pushed a commit to MoLow/node that referenced this pull request Feb 25, 2023
@richiemccoll @MoLow
test_runner: emit test-only diagnostic warning
b50cfb3 
PR-URL: nodejs#46540
Fixes: nodejs#46448
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
juanarbol pushed a commit that referenced this pull request Mar 3, 2023
@richiemccoll @juanarbol
test_runner: emit test-only diagnostic warning
94c870b 
PR-URL: #46540
Backport-PR-URL: #46839
Fixes: #46448
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@juanarbol juanarbol mentioned this pull request Mar 3, 2023
Merged
juanarbol pushed a commit that referenced this pull request Mar 5, 2023
@richiemccoll @juanarbol
test_runner: emit test-only diagnostic warning
cd3aaa8 
PR-URL: #46540
Backport-PR-URL: #46839
Fixes: #46448
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
targos pushed a commit that referenced this pull request Mar 13, 2023
@richiemccoll @targos
test_runner: emit test-only diagnostic warning
62d0c0f 
PR-URL: #46540
Fixes: #46448
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
targos pushed a commit that referenced this pull request Mar 14, 2023
@richiemccoll @targos
test_runner: emit test-only diagnostic warning
ffa86f7 
PR-URL: #46540
Fixes: #46448
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@targos targos mentioned this pull request Mar 14, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MoLow MoLow MoLow approved these changes

@cjihrig cjihrig cjihrig approved these changes

@benjamingr benjamingr benjamingr approved these changes

@jasnell jasnell jasnell approved these changes

Assignees
No one assigned
Labels
backport-open-v18.x Indicate that the PR has an open backport. needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

test_runner: warn if only is used without properly enabling only tests
6 participants
@richiemccoll @nodejs-github-bot @cjihrig @MoLow @jasnell @benjamingr