docs(contributing): add info about skip and fail #1944
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rwoll
commented
Apr 23, 2020
| where `condition` is usually a certain browser like `FFOX` (for Firefox), | ||
| `WEBKIT` (for WebKit), and `CHROMIUM` (for Chromium). | ||
|
|
||
| For example, the [alt-click downloads test](https://github.com/microsoft/playwright/blob/471ccc72d3f0847caa36f629b394a028c7750d93/test/download.spec.js#L86) is marked |
There was a problem hiding this comment.
I think pointing to a concrete (permalinked) example is helpful, but this may throw people off about alt-click download once Playwright is fixed to support alt-click in WebKit and Chromium.
Consider adding a completely fiction example like:
it.fail(CHROMIUM)('should do exciting things', async({page, server} => {
const excitingThings = await page.excitingThings()
expect(excitingThings).toBeTruthy()
}))
There was a problem hiding this comment.
Yeah, that's a good point. On the other hand as long as it points to a particular revision of the file and not break over time I think it's ok. We can iterate on this later if it's unsatisfactory.
rwoll
commented
Apr 23, 2020
| @@ -198,6 +198,26 @@ HEADLESS=false SLOW_MO=500 npm run wtest | |||
| BROWSER=chromium node --inspect-brk test/test.js | |||
| ``` | |||
|
|
|||
| - When should a test be marked with `skip` or `fail`? | |||
There was a problem hiding this comment.
I did not comment about how these interact with CI/CD of an MR passing/failing. It may be helpful to add that, too.
yury-s
approved these changes
Apr 23, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When contributing some test cases, I was a bit confused about if I should use
skipandfail, the difference between the two, and how they interact with CI/CD tests passing or failing: #1939 (comment).This is my attempt at explaining the difference, but feel free to edit to make it more clear and concise (and correct any inaccuracies).