UI tests for example sites

[KyleAMathews]

This recently released library looks perfect for this. https://github.com/smooth-code/jest-puppeteer

We'll be using example sites as integration tests i.e. building them in Docker containers against PRs and checking there's no build errors. We should also write UI tests against the builds.

[jlengstorf]

Just to toss out a second option that I've been super impressed with: the cypress.io team and API are both great, and I had a conversation (that mostly went over my head) about how Cypress manages to avoid extra layers between the browser and the tests (e.g. fewer simulated events and proxies), and they're close to adding Firefox support with Edge in the pipeline.

And unless I'm mistaken, we can run Cypress as part of a Travis/Jenkins/CircleCI build without any extra setup.

I'm tagging @brian-mann so he can tell me all the ways I got that wrong. 😅

Whichever way we go, I'd be happy to contribute some tests.

[KyleAMathews]

Ah nice! Yeah, I hear good things about Cypress. That's definitely a great option too. We could start doing some limited UI testing sooner than later with Cypress.

[tsriram]

So I was planning to start writing UI tests for another example site after completing one in #4790 and I just realized that these tests always run against the latest version of gatsby. For example, if there's a PR with changes which would break gatsbygram, UI tests will still pass as the example site's dependency points to the latest version of gatsby and doesn't use the changes from PR. Appreciate any help with making example sites use current version of gatsby from the repo itself. Wonder if we've make them part of monorepo?

[KyleAMathews]

We're doing lots with Cypress so closing this for now.