[WIP] Documentation revamp #1146
Conversation
Still to-do: * Big rework of docs/mutant-rspec.md * More jokes (or fewer jokes - depending on the main authors' preferences) * Get rid of the fancy quote marks & trailing whitespace * Continue reworking docs/nomenclature, starting with the TODO under Mutation Operators, and then carrying on from Insertion until the end of the file.
docs/nomenclature.md
Outdated
| Other subjects are possible (even project-specific subjects) but aren't implemented in the OSS version. Please get in touch with the authors. Some examples are: | ||
| * Constants | ||
| * Class bodies for DSLs | ||
|
|
||
| The more subjects that mutant can alter in your project, the more mutations it can create, and so the higher your confidence you can have that your tests cover the semantics of your application. Please get in touch if you require subjects beyond those implemented in Mutant already - support may be available in the commercial version. |
There was a problem hiding this comment.
This advises you to get in touch twice--feels slightly redundant.
There was a problem hiding this comment.
Yeah, I had meant to supplant the first one with the latter 😅
docs/nomenclature.md
Outdated
|
|
||
| The more subjects that mutant can alter in your project, the more mutations it can create, and so the higher your confidence you can have that your tests cover the semantics of your application. Please get in touch if you require subjects beyond those implemented in Mutant already - support may be available in the commercial version. | ||
|
|
||
| ## Mutation Operator |
There was a problem hiding this comment.
I don't know if this was discussed in the discord or elsewhere, but this terminology seems slightly confusing. What if this heading was just something like "Kinds of Mutations".
There was a problem hiding this comment.
I agree that it's confusing, and I think it's an unnecessary type of thing to have as its own heading. I am aware of some nomenclature-related discussions occurring in the discord but I've struggled to find any kind of conclusive decision-making.
I used it because it was used in the original document, but I think it may be better to have it as a sort-of "here are some types of mutations that can occur" sub-section in the "Mutations" section. What do you & @mbj think to that?
docs/nomenclature.md
Outdated
| We could argue[^1] that this category includes all of the following: | ||
|
|
||
| * Removing a particular expression, from the AST entirely | ||
| * Replacing an expression with `nil`, or `true`, or `false`, or other simple literals. |
There was a problem hiding this comment.
I don't tend to think of those as quite the same as "code removal" but maybe it belongs here
There was a problem hiding this comment.
Mm. That's true, it's more of an extreme semantic reduction, I suppose 😛
I think one thing that's missing from this whole section is a motivation for choosing to categorise things, as well as a motivation for these specific categories. I'm mostly using it as a way to structure the documentation to break it up into smaller more digestible pieces.
In my experience, when I run mutant on a subject and it reduces 2-3 lines of code to nil or self, I'm thinking "ah I just haven't specified those features in my tests", or "perhaps I could restructure that into its own method and test it individually". When I run it and it removes or reduces a smaller amount of code, I've found it's more likely that it's actually removing / reducing something that isn't necessary. This may just be a symptom of the fact my mutation testing has so far been limited to things I have built since setting up mutant.
I realise I am coming to this from a bit of a beginner's standpoint, which is perhaps not helping me to write this document in a universally applicable way. I'm happy to continue a discussion on this and to defer to experts!
There was a problem hiding this comment.
I've reworked this a bit. As well as combining the mutation and mutation operator sections, I've reworked this section a bit to add in some of that motivation. Let me know what you think of it now!
docs/nomenclature.md
Outdated
| ## Mutation Testing | ||
| The practice of systematically applying small changes one at a time to a codebase then re-running the (relevant) tests for each change. If the tests fail, then they cover the semantics of those changes. If the tests continue to pass with the changes in place, the tests do not cover the complete semantics of the changed parts of the codebase. | ||
|
|
||
| Each type of change (for example, “change `a && b` to `a`”) is known as a [Mutation Operator](#mutation-operator), and is applied to the pristine codebase - that is, the changes are not stacked up, but applied in isolation. The places in which the Mutation Operators can make their changes are called [Subjects](#subject). The changed Subject is then referred to as a [Mutation](#mutation). |
There was a problem hiding this comment.
"Each type of change (for example, “change a && b to a”) is known as a ..."
This sounds like the change itself is a "Mutation Operator" but I think that would be a Mutation. You mean to be talking about kinds of changes but I think the example confuses it a bit because it is an instance of a mutation.
| * `str` and `sym` are string and symbol literals that do not contain dynamic elements (`"I'm a dstring #{i_am_the_dynamic_element}"`) | ||
| * `dst` and `dsym` are the equivalents for strings and symbols which do | ||
| * `regexp` is a regular expression without any options (`/like this/`) | ||
| * `regopt` is a regular expression with options (`/like this/i`) | ||
| * a few come in regular and `-asgn` variants - the latter are the assignment versions. E.g., `||=` is the assignment version (`or_asgn.rb`) of `||` (`or.rb`) | ||
| * `gvar`, `cvar`, `ivar` and `lvar` refer to global (`$VERBOSE`), class (`@@debug`), instance (`@amount`) and local (`response`) variables, respectively. | ||
| * `csend` is the conditional send operator (`&.`) | ||
| * `cbase` is a top-level constant (`::Kernel` or `::Errno::ENOENT`), whereas `const` is a "normal" constant (`Kernel` or `Errno::ENOENT`) | ||
| * `casgn` is an assignment to a constant (`VERSION = "0.1.0"`) |
There was a problem hiding this comment.
This strikes me as very helpful to new contributors :)
There was a problem hiding this comment.
, or Things I Had To Learn When I Was Adding Metaclass Support 😁
|
Lots of improvements here 🎉 Glad to see the docs getting additional attention. I left a few thoughts I had as I read through. |
Thanks for the feedback - I haven't read any of it (yet) as it's midnight but I'll incorporate it into my TODOs up top tomorrow ❤️ |
Hi Markus and any other mutant contributors,
I've been reworking some of the documentation to try to reduce the feeling of absolute befuddlement any mutant novice experiences. This PR is still very much a to-do, though!
Please ignore the long lines, double-spaces and fancy quotes - they are not my preference. I've been trying out a new editor and it seemed like it was correctly automatically putting new-lines in, but it was all a front! I'll be going back over everything quickly at the end with vim, my one true love.
I'll be continuing to alter this PR over the next week or so - this push is just so you can see the current style and content and give any early feedback.
TODOS (probably not exhaustive)