mochawesome, mocha-reporter-junit, third-party reporters do not work in parallel mode

[gurjeetbains]

Here is the screenshot of report in parallel mode

Here is the screenshot of same in serial mode

Here is the details of package.json files:-
{ "name": "playwrightmocha", "version": "1.0.0", "description": "", "main": "index.js", "scripts": { "test": "echo \"Error: no test specified\" && exit 1" }, "author": "", "license": "ISC", "dependencies": { "chai": "^4.2.0", "mocha": "^8.1.3", "mochawesome": "^6.1.1", "playwright": "^1.4.2", "playwright-cli": "^0.5.2" }, "mocha": { "parallel": true, "reporter": [ "spec", "json", "mochawesome" ], "timeout": 300000 } }

[boneskull]

you and/or the mochawesome maintainer(s) would need to find the root cause of the failure; once we have that, we can see if it's fixable. the data reporters get in parallel mode is not the same as serial mode; see #4403

going to leave this open, but it's not actionable until we know more about why.

(note: mochawesome is not maintained by the mocha team)

[gurjeetbains]

@boneskull I tried with the junit-reporter for mocha and got the below error. Mochawesome I don't have error
D:\My Practice Projects\playwrightmocha\node_modules\mocha\lib\nodejs\parallel-buffered-runner.js:269
throw err;
^

TypeError: Cannot read property 'testsuite' of undefined
at lastSuite (D:\My Practice Projects\playwrightmocha\node_modules\mocha-junit-reporter\index.js:209:46)
at MochaJUnitReporter. (D:\My Practice Projects\playwrightmocha\node_modules\mocha-junit-reporter\index.js:234:5)
at ParallelBufferedRunner.emit (events.js:327:22)
at ParallelBufferedRunner.Runner.fail (D:\My Practice Projects\playwrightmocha\node_modules\mocha\lib\runner.js:411:8)
at ParallelBufferedRunner.Runner._uncaught (D:\My Practice Projects\playwrightmocha\node_modules\mocha\lib\runner.js:939:12)
at D:\My Practice Projects\playwrightmocha\node_modules\mocha\lib\nodejs\parallel-buffered-runner.js:254:18
at Array.forEach ()
at D:\My Practice Projects\playwrightmocha\node_modules\mocha\lib\nodejs\parallel-buffered-runner.js:247:12
at processTicksAndRejections (internal/process/task_queues.js:97:5)

[boneskull]

These all fail for likely different reasons, but I do not have time to determine why.

For each of these, I need to know the root cause; it's likely one of these issues:

• A particular property is missing from the Test, Suite, or Hook data emitted from a Runner event
• A particular property is not strictly equal to an expected value in a Test, Suite, or Hook data emitted from a Runner event
• Test, Suite, or Hook data emitted from a Runner event is not strictly equal to an expected value

The best people to answer these questions would be maintainers-of or contributors-to the broken reporters. I encourage those interested in a fix to open an issue in their repo(s) and reference this one.

[sashikamw]

Hi @gurjeetbains, were you able to solve this issue? I'm using 'mocha-junit-reporter' with '--parallel' flag and getting the same error as in #4453 (comment)

[curtisman]

@boneskull I have run into some problem with the bundled xunit reporter as well. To repro, just create tests that fails the test or hook and run it in parallel mode with the xunit reporter. Some of root cause probably overlaps with the third-party reporters.

Here are what I have found looking at the code and debugging:
Repro case: run the following with --parallel --reporter xunit

describe("blah3", () => {
   beforeEach(() => {
       throw new Error();
   });
   it("test3", () => { });
});

---

First error:

TypeError: test.parent.fullTitle is not a function
    at XUnit.test (d:\temp\test\node_modules\mocha\lib\reporters\xunit.js:159:28)
    at d:\temp\test\node_modules\mocha\lib\reporters\xunit.js:107:12
    at Array.forEach (<anonymous>)
    at ParallelBufferedRunner.<anonymous> (d:\temp\test\node_modules\mocha\lib\reporters\xunit.js:106:11)
    at Object.onceWrapper (events.js:421:28)
    at ParallelBufferedRunner.emit (events.js:327:22)
    at d:\temp\test\node_modules\mocha\lib\nodejs\parallel-buffered-runner.js:347:14
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

Looking the code for Hook.prototype.serialize: the parent only get the mocha id, and nothing else (therefore fullTitle isn't there). A potential fix just call this.parent.serialize(). Applying that locally let me get thru this error.

---

Second error:

TypeError: test.isPending is not a function
    at XUnit.test (d:\temp\test\node_modules\mocha\lib\reporters\xunit.js:183:19)
    at d:\temp\test\node_modules\mocha\lib\reporters\xunit.js:107:12
    at Array.forEach (<anonymous>)
    at ParallelBufferedRunner.<anonymous> (d:\temp\test\node_modules\mocha\lib\reporters\xunit.js:106:11)
    at Object.onceWrapper (events.js:421:28)
    at ParallelBufferedRunner.emit (events.js:327:22)
    at d:\temp\test\node_modules\mocha\lib\nodejs\parallel-buffered-runner.js:347:14
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

Here, pending or isPending() can be undefined on Test, Suite, Hook. So during serialized, the property $$isPending would be assigned undefined and after IPC from the worker back to the main process, the property would be missing and the isPending() won't be create during deserialize. A solution would be forcing the value to be false but adding !!this.isPending()) or !!this.pending in serialize. Adding that to all of Test, Suite, Hook' s serialize function get me thru this error.

---

Third "error": This is not an error, but the output for xunit reporter is missing the "failure" information:
Expected:

<testsuite name="Mocha Tests" tests="0" failures="0" errors="1" skipped="-1" timestamp="Sat, 03 Apr 2021 22:45:47 GMT" time="0.003">
<testcase classname="blah3" name="&#x22;before each&#x22; hook for &#x22;test3&#x22;" time="0">**<failure>
Error
    at Context.&#x3C;anonymous&#x3E; (test3.spec.js:3:14)
    at processImmediate (internal/timers.js:461:21)</failure></testcase>**
</testsuite>

Actual:

<testsuite name="Mocha Tests" tests="0" failures="0" errors="1" skipped="-1" timestamp="Sat, 03 Apr 2021 22:44:57 GMT" time="0.252">
<testcase classname="blah3" name="&#x22;before each&#x22; hook for &#x22;test3&#x22;" time="0"/>
</testsuite>

The problem is that Hook's state is missing in serialize. Adding state: this.state to the object generated by Hook.prototype.serialize fixes it.

[juergba]

I'm closing this issue.

Mocha's own reporters xunit and json have been fixed.
Mochawesome has been adapted by their maintainers as well.
And there haven't been any additional input/requests by further third-party reporters or their maintainers.