RabbitMQ Clustering configurations

In this post we will see how to cluster rabbitMQ in 2 configurations.
one using same host, called a vertical cluster, and other using 2 differents hosts called the horizontal cluster.

Create Vertical clustering

1- Start RabbitMQ with differant node names and port numbers similar to below:

RABBITMQ_NODE_PORT=5672 RABBITMQ_SERVER_START_ARGS="-rabbitmq_management listener [{port,15672}]" RABBITMQ_NODENAME=rabbit ./rabbitmq-server -detached
RABBITMQ_NODE_PORT=5673 RABBITMQ_SERVER_START_ARGS="-rabbitmq_management listener [{port,15673}]" RABBITMQ_NODENAME=rabbit2 ./rabbitmq-server -detached
 
using the above env. variable form of starting up rabbitMQ vs using config file; enables us to use diffent ports and nodenames.
The config file should be empty in this case.
The above will start 2 RabbitMQ nodes on the same VM each one has its unique name and has its own uniq set of ports.
2- To enable the 2 nodes to be in a single cluster with one node as DISC and one as RAM do the below:

./rabbitmqctl -n rabbit2@beren stop_app -> stop the rabbitMQ application but node is still running on host beren
./rabbitmqctl -n rabbit2@beren join_cluster --ram rabbit@beren -> Add rabbit2 as RAM node to rabbit cluster, default is DISC
./rabbitmqctl -n rabbit2@beren start_app -> Start the rabbitMQ application
./rabbitmqctl -n rabbit@beren cluster_status -> check the cluster status should show something like this:

Cluster status of node rabbit@beren ...
[{nodes,[{disc,[rabbit@beren]},{ram,[rabbit2@beren]}]},
{running_nodes,[rabbit2@beren,rabbit@beren]},
{partitions,[]}]
...done.
 
3- to change node type from RAM to DISC do:

./rabbitmqctl -n rabbit2@beren stop_app
./rabbitmqctl -n rabbit2@beren change_cluster_node_type disc
./rabbitmqctl -n rabbit2@beren start_app
 
to change it back to RAM do:

./rabbitmqctl -n rabbit2@beren stop_app
./rabbitmqctl -n rabbit2@beren change_cluster_node_type ram
./rabbitmqctl -n rabbit2@beren start_app
Note that at least the cluster should have one disc node. Ram nodes don't offer faster response in queue performance but only offer better speed on queue creation, recommended to have at least 1 disc node on every VM to allow high availability and add ram nodes in a vertical cluster as decscribed above IF needed.

Create Horizontal clustering

To do this we need a second VM running RabbitMQ.
on the new 2nd node do the following.
1- copy the erlang cookie file from the old VM RabbitMQ user home directory to the new VM user home directory.
cookie file name is .erlang.cookie

2- go to RABBITMQ_HOME/sbin & start rabbitMQ

./rabbitmq-server -detached

3- Run the below list of commands:

./rabbitmqctl -n rabbit@vardamir stop_app
./rabbitmqctl -n rabbit@vardamir join_cluster rabbit@beren
./rabbitmqctl -n rabbit@vardamir start_app

All the queues and topics created will automatically be transfered to the new joining node.
This will create a 2 node on 2 VM cluster that will offer true HA for the rabbitMQ. queues can be configured to work in HA mode by creating a policy from the rabbitMQ admin console, the policy should containt ha-mode:all so that the queues are all working across the cluster in HA.
Also replication can be enabled in the same manner.

HAproxy config

HA proxy can be used to offer true load-balancing for a rabbitMQ cluster.
The cluster doesn't offer a common gateway by default so to benefit from it we need balancer sitting infornt of it.
In this case haproxy can help as it is very light weight and a performer.
The config for balancing 2 nodes should look like this:

# Simple configuration for an HTTP proxy listening on port 8080 on all
# interfaces and forwarding requests to a single backend "servers" with a
# single server "server1" listening on 127.0.0.1:8000

    global
    log 127.0.0.1 local0 info
    daemon
   
    defaults
    log     global
    mode     tcp
    option tcplog
    option dontlognull
    retries 3
    option redispatch
    maxconn    2000
    timeout connect 10s
    timeout client     120s
    timeout    server     120s
   
    listen rabbitmq_local_cluster 0.0.0.0:8080
   
    balance roundrobin
    server rabbitnode1    beren:5672 check inter 5000 rise 2 fall 3 #Disk Node
    server rabbitnode2 vardamir:5672 check inter 5000 rise 2 fall 3 #Disk Node
   
   
    listen private_monitoring  :8101
    mode http
    option httplog
    stats enable
    stats uri /stats
    stats refresh  30s

The above makes use of haproxy tcp mode since rabbitMQ uses amqp which is not http compatible.