Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

skip flaky tests #15689

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

skip flaky tests #15689

wants to merge 1 commit into from

Conversation

pfgithub
Copy link
Contributor

@pfgithub pfgithub commented Dec 10, 2024

probably not a bug:

  • test/cli/hot/hot.test.ts: delays added to allow for file watcher to catch up

likely a ci issue:

  • test/v8/v8.test.ts on musl: seems to be missing './build/Release/v8tests'

likely a real bug that is now hidden, but still needs to be fixed:

  • test/js/bun/util/filesink.test.ts: leaking fd on musl
  • test/js/node/fs/fs.test.ts: simple write stream finishes on windows
  • test/js/bun/util/sleep.test.ts: time limit increased to 2000ms, but it really shouldn't take that long to launch every
    fixtureThatShouldCompleteInstantly
  • test/js/deno/performance/performance.test.ts: "performanceMeasure" updated to allow setTimeout(, 100) to take >= 85ms
    rather than >= 100ms
    • deno harness is updated to display the message if the test fails
  • test/cli/run/require-cache.test.ts: "via require()" leaks memory on windows
  • test/integration/next-pages/test/next-build.test.ts: on linux x64 baseline, sometimes next build does not emit _error.js or index.js
  • test/integration/next-pages/test/next-build.test.ts: on windows, sometimes nodeBuild fails with exit code 1

TODO:

  • websocket-server.test.ts flaky between tests (server.upgrade but server is undefined occasionally)
  • the fix for hot.test.ts didn't work, undo it, skip on linux, and mark as a potential bug
  • v8.test.ts needs to be disabled on musl where the files are missing
  • test-timers-immediate-queue.js failure
  • bun/css/doesnt_crash.test.ts failure (timeout minifying tailwind.css. does it really take over 90 seconds or is it stuck?
  • test-zlib-dictionary.js failure on linux. delete it?
  • test-worker-process-exec-async-module.js failure on linux. delete?
  • bun-install-registry.test.ts
  • bun-link.test.ts
  • test-net-sync-cork..js
  • bundler_compile.ts
  • test-fs-readfile-flags.js
  • test-eventemitter-method-names.js
  • serve-body-leak.test.ts
  • test-fs-watch-file-enoent-after-deletion.js

@robobun
Copy link

robobun commented Dec 10, 2024

@pfgithub, your commit 35bc011 has 59 failures in #7719:

  • test/js/bun/websocket/websocket-server.test.ts - code 1 on 🐧 3.20 x64
  • test/js/bun/websocket/websocket-server.test.ts - code 1 on 🐧 3.20 x64-baseline
  • test/cli/hot/hot.test.ts - timeout on 🐧 12 x64-baseline
  • test/v8/v8.test.ts - 22 failing on 🐧 3.20 aarch64
  • test/v8/v8.test.ts - timeout on 🐧 20.04 aarch64
  • test/v8/v8.test.ts - 22 failing on 🐧 3.20 x64
  • test/v8/v8.test.ts - 22 failing on 🐧 3.20 x64-baseline
  • test/js/node/test/parallel/test-timers-immediate-queue.js - code 1 on 🐧 24.04 x64-baseline
  • test/js/node/test/parallel/test-assert-builtins-not-read-from-filesystem.js - code 1 on 🍎 13 aarch64
  • test/js/bun/css/doesnt_crash.test.ts - 1 failing on 🐧 12 aarch64
  • test/js/bun/css/doesnt_crash.test.ts - 1 failing on 🐧 3.20 aarch64
  • test/js/node/fs/promises.test.js - 3 failing on 🍎 13 aarch64
  • test/js/node/fs/cp.test.ts - 34 failing on 🍎 13 aarch64
  • test/js/node/test/parallel/test-zlib-dictionary.js - code 1 on 🐧 22.04 aarch64
  • test/js/node/test/parallel/test-zlib-dictionary.js - code 1 on 🐧 24.04 x64
  • test/js/node/test/parallel/test-zlib-dictionary.js - code 1 on 🐧 12 x64
  • test/js/node/test/parallel/test-zlib-dictionary.js - code 1 on 🐧 12 x64-baseline
  • test/js/node/test/parallel/test-zlib-dictionary.js - code 1 on 🐧 22.04 x64-baseline
  • test/js/node/test/parallel/test-zlib-dictionary.js - code 1 on 🐧 20.04 x64-baseline
  • test/js/node/test/parallel/test-zlib-dictionary.js - code 1 on 🐧 3.20 x64
  • test/js/node/test/parallel/test-zlib-dictionary.js - code 1 on 🐧 3.20 x64-baseline
  • test/js/node/test/parallel/test-worker-process-exit-async-module.js - code 1 on 🐧 22.04 aarch64
  • test/js/node/test/parallel/test-worker-process-exit-async-module.js - code 1 on 🐧 12 x64
  • test/js/node/child_process/child_process.test.ts - 2 failing on 🍎 13 aarch64
  • test/js/bun/util/which.test.ts - timeout on 🍎 13 aarch64
  • test/cli/install/registry/bun-install-registry.test.ts - 1 failing on 🐧 12 aarch64
  • test/cli/install/registry/bun-install-registry.test.ts - timeout on 🐧 24.04 aarch64
  • test/cli/install/registry/bun-install-registry.test.ts - timeout on 🐧 3.20 x64-baseline
  • test/cli/install/bun-link.test.ts - 1 failing on 🐧 22.04 x64
  • test/cli/install/bun-link.test.ts - 1 failing on 🐧 3.20 aarch64
  • test/cli/install/bun-link.test.ts - 1 failing on 🐧 3.20 x64-baseline
  • test/js/node/test/parallel/test-net-sync-cork.js - code 1 on 🐧 12 aarch64
  • test/js/node/test/parallel/test-net-sync-cork.js - code 1 on 🐧 12 x64
  • test/js/node/test/parallel/test-net-sync-cork.js - code 1 on 🐧 3.20 aarch64
  • test/js/node/test/parallel/test-net-sync-cork.js - code 1 on 🐧 12 x64-baseline
  • test/js/node/test/parallel/test-net-sync-cork.js - code 1 on 🐧 3.20 x64
  • test/js/node/test/parallel/test-net-sync-cork.js - code 1 on 🐧 3.20 x64-baseline
  • test/bundler/bundler_compile.test.ts - 1 failing on 🐧 12 aarch64
  • test/js/bun/util/filesink.test.ts - 21 failing on 🍎 13 aarch64
  • test/js/bun/util/bun-file-exists.test.js - 1 failing on 🍎 13 aarch64
  • test/js/bun/test/test-test.test.ts - 17 failing on 🍎 13 aarch64
  • test/js/bun/test/snapshot-tests/snapshots/snapshot.test.ts - timeout on 🍎 13 aarch64
  • test/js/node/test/parallel/test-fs-readfile-flags.js - code 1 on 🪟 2019 x64-baseline
  • test/js/node/test/parallel/test-fs-readfile-flags.js - code 1 on 🪟 2019 x64
  • test/js/node/test/parallel/test-event-emitter-method-names.js - code 1 on 🪟 2019 x64-baseline
  • test/js/node/test/parallel/test-event-emitter-method-names.js - code 1 on 🪟 2019 x64
  • test/js/bun/http/serve-body-leak.test.ts - 1 failing on 🪟 2019 x64
  • test/js/node/test/parallel/test-fs-watch-file-enoent-after-deletion.js - internal assertion failure on 🪟 2019 x64-baseline
  • test/js/node/test/parallel/test-fs-watch-file-enoent-after-deletion.js - internal assertion failure on 🪟 2019 x64
  • test/js/bun/test/snapshot-tests/new-snapshot.test.ts - 1 failing on 🍎 13 aarch64
  • test/js/bun/test/expect-assertions.test.ts - timeout on 🍎 13 aarch64
  • test/js/bun/spawn/spawn.test.ts - 6 failing on 🍎 13 aarch64
  • test/js/bun/spawn/bun-ipc-inherit.test.ts - 1 failing on 🍎 13 aarch64
  • test/js/bun/shell/leak.test.ts - 18 failing on 🍎 13 aarch64
  • test/js/bun/shell/exec.test.ts - code 1 on 🍎 13 aarch64
  • test/js/bun/shell/commands/rm.test.ts - 1 failing on 🍎 13 aarch64
  • test/js/bun/resolve/resolve-ts.test.ts - 26 failing on 🍎 13 aarch64
  • test/js/bun/resolve/non-english-import.test.js - 3 failing on 🍎 13 aarch64
  • test/js/bun/resolve/import-meta.test.js - 1 failing on 🍎 13 aarch64
  • test/js/bun/resolve/import-custom-condition.test.ts - 8 failing on 🍎 13 aarch64
  • test/js/bun/io/bun-write-leak.test.ts - 1 failing on 🍎 13 aarch64
  • test/js/bun/http/serve.test.ts - 3 failing on 🍎 13 aarch64
  • test/js/bun/http/proxy.test.js - 4 failing on 🍎 13 aarch64
  • test/js/bun/http/fetch-file-upload.test.ts - 1 failing on 🍎 13 aarch64
  • test/js/bun/http/bun-serve-propagate-errors.test.ts - 1 failing on 🍎 13 aarch64
  • test/js/bun/http/bun-listen-connect-args.test.ts - 7 failing on 🍎 13 aarch64
  • test/js/bun/glob/scan.test.ts - 1 failing on 🍎 13 aarch64
  • test/integration/next-pages/test/next-build.test.ts - 1 failing on 🍎 13 aarch64
  • test/integration/next-pages/test/dev-server-ssr-100.test.ts - 1 failing on 🍎 13 aarch64
  • test/integration/expo-app/expo.test.ts - 1 failing on 🍎 13 aarch64
  • test/cli/watch/watch.test.ts - 2 failing on 🍎 13 aarch64
  • test/cli/test/coverage.test.ts - 3 failing on 🍎 13 aarch64
  • test/cli/run/transpiler-cache.test.ts - 8 failing on 🍎 13 aarch64
  • test/cli/run/self-reference.test.ts - 1 failing on 🍎 13 aarch64
  • test/cli/run/run-unicode.test.ts - 1 failing on 🍎 13 aarch64
  • test/cli/run/run-quote.test.ts - 1 failing on 🍎 13 aarch64
  • test/cli/run/run-extensionless.test.ts - 2 failing on 🍎 13 aarch64
  • test/cli/run/require-cache.test.ts - 4 failing on 🍎 13 aarch64
  • test/cli/run/preload-test.test.js - 2 failing on 🍎 13 aarch64
  • test/cli/run/filter-workspace.test.ts - 48 failing on 🍎 13 aarch64
  • test/cli/run/env.test.ts - 76 failing on 🍎 13 aarch64
  • test/cli/install/redacted-config-logs.test.ts - 11 failing on 🍎 13 aarch64
  • test/cli/install/migration/migrate.test.ts - 4 failing on 🍎 13 aarch64
  • test/cli/install/bunx.test.ts - 2 failing on 🍎 13 aarch64
  • Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    None yet
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    2 participants