Reactive spark streaming example

This is a fully dockerized Spark streaming example using a Kafka queue to explore the new back-pressure feature introduced in Spark 1.5.

Install docker-machine & docker-compose (require >= 1.5.1)
Create / start the virtual machine
docker-machine create rspark --driver virtualbox or
docker-machine start rspark

Update your environment

eval "$(docker-machine env rspark)"
export DOCKER_HOST_IP=`docker-machine ip rspark`

Run zookeeper, kafka, kafkamanager, spark master and worker

docker-compose up zookeeper kafka kafkamanager sparkmaster sparkworker

Configure kafkamanager and add the required topic
- Zookeeper: zk:2181
- Kafka version: 0.8.2.1
- Topic: numbers
Build and submit the spark streaming consumer
Unfortunately Spark expects jobs to be submitted to spark://master:7077 (the docker internal hostname), so just map it in your /etc/hosts.
```
sbt consumer/assembly consumer/sparkSubmit
```

Build and start the containerized producer

sbt producer/docker:publishLocal
docker-compose up producer

Tweak the producer settings in docker-compose.yml and observe how your processing time / processing delay changes.
Your Spark streaming UI should be here, but can always be found on the Spark master UI.
- PRODUCER_RATE: the event rate
- PRODUCER_MIN_DELAY: the min. processing weight of the produced messages
- PRODUCER_MAX_DELAY: the max. processing weight of the produced messages
```
docker-compose up producer
```
Tweak the Spark streaming settings in SparkConsumer.scala and resubmit the job.
```
sbt consumer/assembly consumer/sparkSubmit
```

Provide feedback