Benchmarking is incredibly useful, yet endlessly painful. This benchmark suite is the result of a group of Phd/post-docs/professors getting together and combining their workloads/frameworks/experiences/efforts. We hope this will save other people's time, and will provide an extensible platform, that can be grown in an open-source fashion.
OLTPBenchmark is a multi-threaded load generator. The framework is designed to be able to produce variable rate, variable mixture load against any JDBC-enabled relational database. The framework also provides data collection features, e.g., per-transaction-type latency and throughput logs.
Together with the framework we provide the following OLTP/Web benchmarks:
- TPC-C
- Wikipedia
- Synthetic Resource Stresser
- Epinions.com
- TATP
- AuctionMark
- SEATS ("Stonebraker Electronic Airline Ticketing System")
- YCSB
- JPAB (Hibernate)
- CH-benCHmark
- Voter (Japanese "American Idol")
- SIBench (Snapshot Isolation)
- SmallBank
- LinkBench
This framework is design to allow easy extension, we provide stub code that a contributor can use to include a new benchmark, leveraging all the system features (logging, controlled speed, controlled mixture, etc.)
- Java (+1.7)
- Apache Ant
See the on-line documentation on how to use OLTP-Bench.
A Dockerfile has been provided for running OLTPBench interactively without having to build the dependencies. To build the Docker image, run:
docker build -t oltpbench .This command builds the OLTPBench image with the tag oltpbench.
The Docker container will read the configuration file from STDIN. All other parameters must still be passed in through the container. For example, to use the example from the docs,
cat ./config/sample_tpcc_config.xml | docker run -i oltpbench -b tpcc --create=true --load=true --execute=true -s 5 -o outputfileThis will run the image created above using the tag we provided, passing in the configuration file sample_tpcc_config.xml found in the config directory.
If you are using this framework for your papers or for your work, please cite the paper:
OLTP-Bench: An extensible testbed for benchmarking relational databases D. E. Difallah, A. Pavlo, C. Curino, and P. Cudre-Mauroux. In VLDB 2014.
Also, let us know so we can add you to our list of publications.