Feature request: output alive test selectors at end of output

[backus]

Hello old friends. Long time no issue 😄

Often, I will run mutant against MyApp* (either locally or in CI) at the end of writing a feature. I have then done this workflow probably over a hundred times:

• I will select the entire output
• Copy it into my text editor
• Extract every line matching matching /^evil:(.+?):\// to get the unkilled test selectors
• Dedupe them

Then keep them in an unsaved file, rerun mutant with one of the specific test selectors, kill those mutants, remove it from the temp file, repeat until I have an empty file.

I don't want anything fancy really. It would just be nice if, at the very bottom of a mutant run, it said something like:

Uncovered mutations detected:

MyApp::User#valid?
MyApp::Signin#show
MyApp::Util#deep_fetch
MyApp::Country#supports_zone?

Exiting non-zero!

This would especially save time because, often, the CI output can be megabytes of text and doing this manual extraction is annoying and time consuming

[mbj]

I like this idea, a lot. Maybe we should "prefix compress" it so lets say your mutations from above would be reported like:

MyApp::
  User#Valid?
  Signing#show
...

This idea goes into an important direction for me to build the tooling to allow iterative use easier.

[dgollahon]

I also feel strong support for this idea/this direction for mutant!

The "prefix compression" idea is reasonable but I actually think it would probably be better to not do that so you can copy the whole expression if you want to re-run it or search through your top level subject describes quickly. What I think we should do is just sort them so common prefixes are grouped.

We could also consider outputting full mutant commands for them (like RSpec does) so they are easy to re-run but that might be too much noise.

[mbj]

@dgollahon We can abstract this problem space:

• We want to use information from previous runs to be available for next runs.
• Why have the user to transport this informatio in the first place via copy & paste?

TLDR: We need result presistence.

I made that multiple times, but not in the released version.

[dgollahon]

TLDR: We need result persistence.

That is clearly better than just outputting the human-readable copy/paste only, but having the simple output as well could be helpful (to just understand at a glance) and I was thinking of what I suggested as an intermediary step. I would love to have something more fully featured but if we can get something that is partly helpful in quicker, that's a win IMO.

So I see value in both a full persistence system and a helpful summary like @backus described just at the end of the CLI output.

[mbj]

@dgollahon Yes we want both, manual state carry forward and options for implicit state carry forward.

[mbj]

@dgollahon I'd even support a, per subject re-run command printout.