Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a perfstar baseline for tracking the impact of generating binlogs #11093

Open
baronfel opened this issue Dec 5, 2024 · 1 comment
Open
Labels
Area: Debuggability Issues impacting the diagnosability of builds, including logging and clearer error messages. Area: Logging Area: PerfStar Issues related to our PerfStar tool. That measures MSBuild performance. performance Priority:2 Work that is important, but not critical for the release triaged

Comments

@baronfel
Copy link
Member

baronfel commented Dec 5, 2024

Summary

We want people to make binlogs all the time, but we should be clear-eyed about the cost of doing so. We should make a PerfStar baseline that tracks the cost of generating binlogs for a representative project (orchard core) on stable hardware to give users a way of thinking about this cost, as well as giving ourselves a target to minimize.

Background and Motivation

The latest orchard code commit on my machine has a ~25% overhead for binlog generation. I

  • cloned the repo
  • ran dotnet restore to pre-load the nuget caches
  • triggered the build without binlogs via dotnet build
  • run git clean -xdf to remove intermediates
  • ran dotnet build-server shutdown to kill the persistent build servers/nodes
  • triggered the build with binlogs via dotnet build -bl

and that gave me timings of 105.6s and 135.0s respectively on my machine (+- a tiny bit of jitter across iterations). This is on an AMD Ryzen 7 5700x3d, 64 GB of decently-fast RAM, and an NVMe SSD.

Proposed Feature

We should make a harness that does the above in a more repeatable way and tracks the ratio over time.

Alternative Designs

No response

@baronfel baronfel added Area: Debuggability Issues impacting the diagnosability of builds, including logging and clearer error messages. performance Area: Logging Area: PerfStar Issues related to our PerfStar tool. That measures MSBuild performance. labels Dec 5, 2024
@maridematte
Copy link
Contributor

This is one of the idea for the next perfStar improvements (you can find it in the one pager, which I'll link once the PR is completed).

@maridematte maridematte added Priority:2 Work that is important, but not critical for the release triaged labels Dec 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Debuggability Issues impacting the diagnosability of builds, including logging and clearer error messages. Area: Logging Area: PerfStar Issues related to our PerfStar tool. That measures MSBuild performance. performance Priority:2 Work that is important, but not critical for the release triaged
Projects
None yet
Development

No branches or pull requests

2 participants