-
Notifications
You must be signed in to change notification settings - Fork 2k
Benchmarking Getting Started
This is the getting started guide for benchmarking.
If you're interested in development please visit the getting started guide for development.
You will need three computers (or virtual machines) networked together to fill the three benchmark roles of database server, load generation server, and web framework server.
Make sure to secure the 3 computers / virtual machines to ensure the accuracy of the benchmarking results and your own network security.
- Disable password-based SSH and only use key-based authentication.
- Set up a private network amongst the machines and have them communicate with each other only via the private networking interfaces.
- Set up firewalls on all your machines (on the cloud service or on the machines themselves) to restrict any public access and only open necessary ports (e.g. only SSH port).
Note: If testing a pull request or doing development, it is usually adequate to only use one computer. In that case, your server, client, and database IPs will be 127.0.0.1.
Required Host Roles
- App (aka web framework) Server
- Runs the web framework containers
- Operating system:
Ubuntu Server 22.04 LTS 64-bit
- Database Server
- Runs database containers (e.g.
MySQL
,PostgreSQL
,MongoDB
, etc). - Operating system:
Ubuntu Server 22.04 LTS 64-bit
- Runs database containers (e.g.
- Client (aka load generation) Server
- Runs the benchmark client container
- Operating system:
Ubuntu Server 22.04 LTS 64-bit
Prerequisites
Before you get started, the following are the steps you must take on each machine (App, Database, Client) to run the benchmarking suite:
- Docker installed (you can verify this via
docker run hello-world
) -
/lib/systemd/system/docker.service
needsExecStart
to have an additional flag:-H [machine's ip]:2375
sudo systemctl daemon-reload
-
sudo service docker restart
(you can verify this viasudo service docker status
and see the newly added-H [machine's ip]:2375
flag)
Since the tests can run for several hours, it helps to set everything up so that once the tests are running, you can leave the machines unattended and don't need to be around to enter ssh or sudo passwords.
Running the Benchmarking Suite
Let us assume make some assumptions for the example:
- You cloned the repository into the home directory of user
techempower
- Your App machine has the IP
10.0.0.1
- Your Database machine has the IP
10.0.0.2
- Your Client machine has the IP
10.0.0.3
You can build the techempower/tfb by running:
docker build --build-arg GROUP_ID=$(id -g) --build-arg USER_ID=$(id -u) -t techempower/tfb .
The following command, given the prerequisites here, would run a benchmark for all the tests in the codebase.
docker run \
-e USER_ID=$(id -u) \
--network=host \
--mount type=bind,source=/home/techempower/FrameworkBenchmarks,target=/FrameworkBenchmarks \
techempower/tfb \
--server-host 10.0.0.1 \
--database-host 10.0.0.2 \
--client-host 10.0.0.3 \
--network-mode host \
--quiet
- Home
- Project Information
- Development
- Benchmarking
- Codebase
- About
- Support
- Work in progress