-
Notifications
You must be signed in to change notification settings - Fork 26
Using RabbitMQ for buffering on the Collector
This page describes how to configure RabbitMQ to create a resilient link between the collectors and mediators.
This is an optional (but recommended) feature added in OpenLI 1.0.6.
The OpenLI mediator also uses RabbitMQ internally to buffer intercept records before forwarding them on to the law enforcement agencies. This is a mandatory feature that has been added in OpenLI 1.1.0 and is described further on a separate wiki page.
Using RabbitMQs persistent link, acknowledgement and heart-beating features, OpenLI is able to weather connection loss between the mediator and collector. To enable this feature RabbitMQ server must be installed on each collector device, and each RabbitMQ server instance must be reachable by the OpenLI mediator(s).
The RabbitMQ instance needs to have an OpenLI vhost.
Accounts need to be created for the OpenLI components to login with, including username/password and permissions.
The username and password do not have to be unique for each component that is connecting.
rabbitmqctl add_vhost "OpenLI"
rabbitmqctl add_user "username" "password"
rabbitmqctl set_permissions -p "OpenLI" "username" ".*" ".*" ".*"
If you wish to use SSL Authentication for the collector RMQ instance, the SSL Auth plugin needs to be enabled before starting the RabbitMQ server.
rabbitmq-plugins enable rabbitmq_auth_mechanism_ssl
You'll also need to add the following configuration to your /etc/rabbitmq/rabbitmq.conf file -- you may have to create the file if it doesn't already exist:
listeners.ssl.default = 5671 # feel free to change this if you want RabbitMQ to listen on a different port
ssl_options.cacertfile = /etc/openli/openli-ca-crt.pem
ssl_options.certfile = /etc/openli/openli-collector-crt.pem
ssl_options.keyfile = /etc/openli/openli-collector-key.pem
ssl_options.verify = verify_peer
ssl_options.fail_if_no_peer_cert = true
ssl_cert_login_from = common_name
auth_mechanisms.1 = PLAIN
auth_mechanisms.2 = AMQPLAIN
auth_mechanisms.3 = EXTERNAL
IMPORTANT: make sure that the rabbitmq user has read permissions on all of your .pem files listed in the configuration file.
When creating a RabbitMQ account for an OpenLI mediator, the account name must be the common name of the certificate that the mediator is using. The password is not used by OpenLI when SSL Authentication is enabled, but this is still required by RabbitMQ.
To enable receiving messages from a RabbitMQ instance, the mediator must set the RMQenabled option to true.
The config file for the mediator must also contain the username/password to login, the frequency to send heartbeats to RMQ and a flag to indicate whether SSL Auth should be used to connect to the RabbitMQ server. If SSL Auth is enabled then the certificates will be the same ones used for other OpenLI TLS operations.
For example:
RMQname: username #username to login
RMQpass: password #password to login (Only needed if not using SSL Auth)
RMQheartbeatfreq: 30 #time between Heartbeats (2x this is the timeout for detecting a dropped connection)
RMQenabled: true #Set to true to enable RMQ
RMQSSL: false #Set to true to enable SSL Auth
tlscert: /etc/openli/openli-mediator-crt.pem
tlskey: /etc/openli/openli-mediator-key.pem
tlsca: /etc/openli/openli-ca-crt.pem
If RMQheartbeatfreq is not supplied or set to 0, then heartbeats will be disabled. This will mean connection loss to the RabbitMQ server will not be detected until the death of the TCP connection. Unacknowledged messages will still persist on the RabbitMQ server however.
The config file for each Collector needs to contain the RabbitMQ login information for the local RabbitMQ instance.
RMQenabled: true
RMQname: "username"
RMQpass: "password"
Because the collector will connect to RabbitMQ over localhost, we do not bother with the use of SSL to authenticate collector->RabbitMQ connections.
By default, an OpenLI mediator will assume that a collector's RabbitMQ instance is listening on port 5671 (for TLS) and port 5672 (for non-TLS) on the IP address that the collector itself connects to the mediator from.
If you need to override the port number (e.g. because you've configured RabbitMQ to listen on a different port), you may add the RMQport config option to both the collector and mediator configuration files. RMQhostname serves the same purpose if your RabbitMQ instance is listening on a different interface for some reason.
RMQport: 15671 # connect to RabbitMQ on port 15671 instead of 5671