feat: test runner

[bartlomieju]

This PR closes #193

It provides simple test runner that searches recursively for *_test.js and *_test.ts files.

I'd prefer to land it ASAP in simplest version and then start bike-shedding based on our needs.

Waiting on #556 to land

[bartlomieju]

Alright so we need to settle on a couple of things for the MVP:

1. Default globs/file pattern. Currently there are only two default globs:

**/*_test.ts,
**/*_test.js

I agree with @hayd that we should also match:

**/test.ts,
**/test.js,

2. CLI
   There are at least a few options that we need to support for MVP, namely:

• --quiet - to capture console.logs
• --failfail - stop on first failure
• --exclude - exclude globs for file matching - in deno_std we need to exclude node_modules/ otherwise we'd be loading test files from there as well
• [globs...] - include globs for file matching

For now let's skip these options:

• parallel - not really working as expected with test runner, it just run all Promises simultaneously, flaky in my experience
• only/skip - run only tests matching/not matching specific name - I want to refactor testing framework a bit, including adding test.only and test.skip functions (see Discussion: run specific test - test.only #214), that would produce problems down the road

CC @ry @hayd

[ry]

Is //test.ts dead code now? Remove that.

[bartlomieju]

Is //test.ts dead code now? Remove that.

Yes. I removed all not needed anymore files. Although in test.ts we had this checkSourceFileChanges function which I'm not yet sure how to coordinate with runner.

I guess extracting it to separate file and passing datetime as arg could work.

[j-f1]

For reference, here are the equivalent options from Jest:

[j-f1]

--silent

Prevent tests from printing messages through the console.

[j-f1]

--bail

Alias: -b. Exit the test suite immediately upon n number of failing test suite. Defaults to 1.

[j-f1]

--testPathIgnorePatterns=[array]

An array of regexp pattern strings that is tested against all tests paths before executing the test. Contrary to --testPathPattern, it will only run those test with a path that does not match with the provided regexp expressions.

[j-f1]

Also, I’m wondering if -x would be a better short name for this option if you keep the exclude name.

[j-f1]

Although in test.ts we had this checkSourceFileChanges function which I'm not yet sure how to coordinate with runner.

I guess extracting it to separate file and passing datetime as arg could work.

Maybe the test runner should have that as a feature/option?

[bartlomieju]

Maybe the test runner should have that as a feature/option?

That's specific action to this repo. Test runner should be agnostic of such things

[ry]

Describe the glob behavior in a jsdoc comment

[bartlomieju]

CI now is considerably slower (2:10 on latest master and 3:47 on this branch)

[piscisaureus]

Yes, great!

[ry]

@bartlomieju @piscisaureus did you confirm that breaking some random test still breaks the runner?

[ry]

Slowness is due to the fact that the TS compiler worker is starting up and shutting down for each test file.

[ry]

LGTM - this is great to have. I confirmed that adding a failing test breaks it.

The test runner is quite slow now. This clearly shows that our TS compiler (cli/compiler/ts.rs) is suboptimal in some way. Let's fix it in core tho.