Advanced Usage

Installation is completed using Maven using mvn package. The only steps required prior to installation is the creation of a service account, which is detailed in the README. You can then run custom load tests using java -jar target/driver.jar. You can see examples of usage in run.py which covers basic use cases.

--project
The Google Cloud Platform project name. This must be provided, since even if only using a Kafka client, we export metrics to Stackdriver and potentially set up our instances using Google Compute Engine, both require a Google Cloud Platform project.

--<client_type>_count
Each client type has a separate flag, the count determines how many of that type to start. By default a single Cloud Pub/Sub publisher and subscriber using the gcloud library are started.

--broker
The network address of the Kafka broker. Since Kafka brokers have many configuration options, we leave the setting up of the broker to the user. This can be provided to test your Kafka configuration against Cloud Pub/Sub. This is required when settings kafka_publisher_count or kafka_subscriber_count.

--message_size
The size of each user message. The size of each user message times the number of messages in each batch must not be larger than 10 MB for most client types, since this is the limitation set by the Cloud Pub/Sub API.

--publish_batch_size
The number of user messages to put in each batch message.

--cps_max_messages_per_pull
This sets the maximum number of messages you would like to receive in each Pull Request. This flag only has meaning when using a (non-streaming) Cloud Pub/Sub client.

--kafka_poll_duration
The length of time to poll for messages when using a Kafka subscriber. This is the Kafka equivalent of cps_max_messages_per_pull.

--cps_subscription_fanout
The number of subscriptions to fan out messages to. The number of Cloud Pub/Sub subscribers set should be a multiple of this flag, which defaults to 1. When set, each of the subscriptions will have the total number of Cloud Pub/Sub subscribers divided by this flag subscribers.

--request_rate
The maximum QPS to allow on each Client.

--max_outstanding_requests
The maximum number of outstanding requests to allow on each Client.

--burn_in_duration
The duration to allow for burn in time. During this interval, we will report metrics to Stackdriver but they will not be aggregated in the Driver. This is done in order to allow Cloud Pub/Sub to tune the topic, which will allow a test that yields more consistent results. This defaults to 2 minutes.

--loadtest_duration
The duration the load test should run. This is overridden by number_of_messages, which sets the number of messages to expect to publish and receive.

--number_of_messages
The number of messages to publish and to expect to receive. If this number is set, the test will run until this number of messages has been received. This flag overrides loadtest_seconds.

--max_publish_latency_test
In this test we will continuously run load tests with increasing request rates until we hit max_publish_latency_millis. You must only provide a single type of publisher to use this test. This uses the latency specified by max_publish_latency_percentile as the bound to check.

--max_subscriber_throughput_test
This test will continuously run load tests with greater publish request rate until the subscriber can no longer keep up, and will let you know the maximum throughput per subscribing client.

--zone
This controls the zone in Google Compute Engine.