Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expand performance tests setup #6515

Open
wants to merge 37 commits into
base: trunk
Choose a base branch
from

Conversation

swissspidy
Copy link
Member

@swissspidy swissspidy commented May 7, 2024

What this PR does:

  • Run tests against Multisite, now that the local Docker environment supports it (as of r58097)
  • Run tests on a single post page in addition to just the homepage
  • Try to improve cache flushes / resets between iterations
  • Try to improve workflow names
  • Make workflow job summaries more readable by using <details>
  • Collect information about object cache hits & misses, flushing cache between runs

Trac ticket: https://core.trac.wordpress.org/ticket/62725


This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

This comment was marked as resolved.

@swissspidy
Copy link
Member Author

Collect information about object cache hits & misses

That part doesn't seem to be working yet. Looks like stats are not reset between runs.

@swissspidy
Copy link
Member Author

@felixarntz @joemcgill This relates to WordPress/performance#1735

I previously opened this to test with multisite, but not sure how useful that is. Maybe we can just support it in the workflow for manual invoke but not actually run it for every commit/PR.

Second, this now tests a single post URL too, but that adds a lot of noise, as it increases the number of combinations. We might want to clean up the summary a little bit or so.

Example: https://github.com/WordPress/wordpress-develop/actions/runs/12373742576?pr=6515

Aside: The object cache stats still don't look correct and actually probably aren't useful, so I'll remove them.

Copy link
Member

@felixarntz felixarntz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@swissspidy This looks awesome! I hope we can land this soon.

Just a few questions, and some suggestions for consolidating duplicate code. Obviously we don't want to do any real refactoring as part of this, but there are a few quick and easy wins that I think are worth it.

.github/workflows/performance.yml Show resolved Hide resolved
.github/workflows/reusable-performance.yml Outdated Show resolved Hide resolved
tests/performance/specs/home.test.js Show resolved Hide resolved
tests/performance/specs/single-post.test.js Outdated Show resolved Hide resolved
tests/performance/utils.js Outdated Show resolved Hide resolved
tests/performance/specs/single-post.test.js Show resolved Hide resolved
tests/performance/wp-content/mu-plugins/server-timing.php Outdated Show resolved Hide resolved
@swissspidy swissspidy marked this pull request as ready for review December 18, 2024 16:27
Copy link

github-actions bot commented Dec 18, 2024

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props swissspidy, flixos90, desrosj, mukesh27.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

Copy link
Member

@felixarntz felixarntz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@swissspidy Looks great! Some suggestions, but nothing critical.

Can you open a Trac ticket for this?

.github/workflows/performance.yml Outdated Show resolved Hide resolved
.github/workflows/reusable-performance.yml Outdated Show resolved Hide resolved
.github/workflows/reusable-performance.yml Show resolved Hide resolved
.github/workflows/reusable-performance.yml Show resolved Hide resolved
tests/performance/compare-results.js Show resolved Hide resolved
@mukeshpanchal27
Copy link
Member

In https://github.com/WordPress/wordpress-develop/actions/runs/12413679906/job/34656106232?pr=6515#step:25:50, it downloads the Firefox browser. Do we actually need that? Additionally, there are warnings during the build of Playwright. Could these warnings indicate an issue on their side? If so, it might be worth opening a ticket in their repository.

@swissspidy
Copy link
Member Author

We don‘t need Firefox and in the step before we actually only install Chromium. But this download comes from wp-scripts, so this would need an enhancement ticket in Gutenberg.

Looks like these warnings are for missing audio and video codecs, nothing we‘d need for the tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants