RabbitMQ Cluster Setup

Posted on by admin

Install RabbitMQ

  1. [root@controller1 ~]# yum install erlang rabbitmq-server -y
  2. [root@controller2 ~]# yum install erlang rabbitmq-server -y
  3. [root@controller3 ~]# yum install erlang rabbitmq-server -y
  4.  
  5. [root@controller1 ~]# systemctl enable rabbitmq-server.service
  6. Created symlink from /etc/systemd/system/multi-user.target.wants/rabbitmq-server.service to /usr/lib/systemd/system/rabbitmq-server.service.
  7. [root@controller2 ~]# systemctl enable rabbitmq-server.service
  8. Created symlink from /etc/systemd/system/multi-user.target.wants/rabbitmq-server.service to /usr/lib/systemd/system/rabbitmq-server.service.
  9. [root@controller3 ~]# systemctl enable rabbitmq-server.service
  10. Created symlink from /etc/systemd/system/multi-user.target.wants/rabbitmq-server.service to /usr/lib/systemd/system/rabbitmq-server.service.
  11.  
  12. [root@controller1 ~]# systemctl start rabbitmq-server.service
  13.  
  14. [root@controller1 ~]# rabbitmq-plugins enable rabbitmq_management
  15. The following plugins have been enabled:
  16.   mochiweb
  17.   webmachine
  18.   rabbitmq_web_dispatch
  19.   amqp_client
  20.   rabbitmq_management_agent
  21.   rabbitmq_management
  22.  
  23. Applying plugin configuration to rabbit@controller1... started 6 plugins.
  24. [root@controller2 ~]# rabbitmq-plugins enable rabbitmq_management
  25. The following plugins have been enabled:
  26.   mochiweb
  27.   webmachine
  28.   rabbitmq_web_dispatch
  29.   amqp_client
  30.   rabbitmq_management_agent
  31.   rabbitmq_management
  32.  
  33. Applying plugin configuration to rabbit@controller2... started 6 plugins.
  34. [root@controller3 ~]# rabbitmq-plugins enable rabbitmq_management
  35. The following plugins have been enabled:
  36.   mochiweb
  37.   webmachine
  38.   rabbitmq_web_dispatch
  39.   amqp_client
  40.   rabbitmq_management_agent
  41.   rabbitmq_management
  42.  
  43. Applying plugin configuration to rabbit@controller3... started 6 plugins.

Configure RabbitMQ for HA queues

  1. [root@controller1 ~]# scp /var/lib/rabbitmq/.erlang.cookie [email protected]:/var/lib/rabbitmq/.erlang.cookie
  2. .erlang.cookie                                                             100%   20    19.8KB/s   00:00    
  3. [root@controller1 ~]# scp /var/lib/rabbitmq/.erlang.cookie [email protected]:/var/lib/rabbitmq/.erlang.cookie
  4. .erlang.cookie                                                             100%   20    34.2KB/s   00:00

On Each Nodes

  1. # chown rabbitmq:rabbitmq /var/lib/rabbitmq/.erlang.cookie
  2. # chmod 400 /var/lib/rabbitmq/.erlang.cookie
  3.  
  4. # systemctl enable rabbitmq-server.service
  5. # systemctl start rabbitmq-server.service
  6.  
  7. [root@controller1 ~]# rabbitmqctl cluster_status
  8. Cluster status of node rabbit@controller1 ...
  9. [{nodes,[{disc,[rabbit@controller1]}]},
  10.  {running_nodes,[rabbit@controller1]},
  11.  {cluster_name,<<"[email protected]">>},
  12.  {partitions,[]},
  13.  {alarms,[{rabbit@controller1,[]}]}]
  14. [root@controller2 ~]# rabbitmqctl cluster_status
  15. Cluster status of node rabbit@controller2 ...
  16. [{nodes,[{disc,[rabbit@controller2]}]},
  17.  {running_nodes,[rabbit@controller2]},
  18.  {cluster_name,<<"[email protected]">>},
  19.  {partitions,[]},
  20.  {alarms,[{rabbit@controller2,[]}]}]
  21. [root@controller3 ~]# rabbitmqctl cluster_status
  22. Cluster status of node rabbit@controller3 ...
  23. [{nodes,[{disc,[rabbit@controller3]}]},
  24.  {running_nodes,[rabbit@controller3]},
  25.  {cluster_name,<<"[email protected]">>},
  26.  {partitions,[]},
  27.  {alarms,[{rabbit@controller3,[]}]}]

On Other Nodes

  1. [root@controller2 ~]# rabbitmqctl stop_app
  2. Stopping node rabbit@controller2 ...
  3. [root@controller3 ~]# rabbitmqctl stop_app
  4. Stopping node rabbit@controller3 ...
  5.  
  6. [root@controller2 ~]# rabbitmqctl join_cluster --ram [email protected]
  7. Clustering node rabbit@controller2 with rabbit@controller1 ...
  8. [root@controller3 ~]# rabbitmqctl join_cluster --ram [email protected]
  9. Clustering node rabbit@controller3 with rabbit@controller1 ...
  10.  
  11. [root@controller2 ~]# rabbitmqctl start_app
  12. Starting node rabbit@controller2 ...
  13. [root@controller3 ~]# rabbitmqctl start_app
  14. Starting node rabbit@controller3 ...

Set the HA-Mode Policy

  1. [root@controller1 ~]# rabbitmqctl cluster_status
  2. Cluster status of node rabbit@controller1 ...
  3. [{nodes,[{disc,[rabbit@controller1]},
  4.          {ram,[rabbit@controller3,rabbit@controller2]}]},
  5.  {running_nodes,[rabbit@controller3,rabbit@controller2,rabbit@controller1]},
  6.  {cluster_name,<<"[email protected]">>},
  7.  {partitions,[]},
  8.  {alarms,[{rabbit@controller3,[]},
  9.           {rabbit@controller2,[]},
  10.           {rabbit@controller1,[]}]}]
  11. [root@controller2 ~]# rabbitmqctl cluster_status
  12. Cluster status of node rabbit@controller2 ...
  13. [{nodes,[{disc,[rabbit@controller1]},
  14.          {ram,[rabbit@controller3,rabbit@controller2]}]},
  15.  {running_nodes,[rabbit@controller3,rabbit@controller1,rabbit@controller2]},
  16.  {cluster_name,<<"[email protected]">>},
  17.  {partitions,[]},
  18.  {alarms,[{rabbit@controller3,[]},
  19.           {rabbit@controller1,[]},
  20.           {rabbit@controller2,[]}]}]
  21. [root@controller3 ~]# rabbitmqctl set_policy ha-all '^(?!amq\.).*' '{"ha-mode": "all"}'
  22. Setting policy "ha-all" for pattern "^(?!amq\\.).*" to "{\"ha-mode\": \"all\"}" with priority "0" ...
  23. [root@controller3 ~]# 
  24. [root@controller3 ~]# 
  25. [root@controller3 ~]# rabbitmqctl cluster_status
  26. Cluster status of node rabbit@controller3 ...
  27. [{nodes,[{disc,[rabbit@controller1]},
  28.          {ram,[rabbit@controller3,rabbit@controller2]}]},
  29.  {running_nodes,[rabbit@controller2,rabbit@controller1,rabbit@controller3]},
  30.  {cluster_name,<<"[email protected]">>},
  31.  {partitions,[]},
  32.  {alarms,[{rabbit@controller2,[]},
  33.           {rabbit@controller1,[]},
  34.           {rabbit@controller3,[]}]}]
  35.  
  36. [root@controller1 ~]# rabbitmqctl set_policy ha-all '^(?!amq\.).*' '{"ha-mode": "all"}'
  37. Setting policy "ha-all" for pattern "^(?!amq\\.).*" to "{\"ha-mode\": \"all\"}" with priority "0" ...
  38. [root@controller2 ~]# rabbitmqctl set_policy ha-all '^(?!amq\.).*' '{"ha-mode": "all"}'
  39. Setting policy "ha-all" for pattern "^(?!amq\\.).*" to "{\"ha-mode\": \"all\"}" with priority "0" ...
  40. [root@controller3 ~]# rabbitmqctl set_policy ha-all '^(?!amq\.).*' '{"ha-mode": "all"}'
  41. Setting policy "ha-all" for pattern "^(?!amq\\.).*" to "{\"ha-mode\": \"all\"}" with priority "0" ...

Now, Take the Browser

http://192.168.220.21:15672

username/password :

guest/guest

Configure OpenStack services to use RabbitMQ HA queues

  1. transport_url = rabbit://RABBIT_USER:RABBIT_PASS@rabbit1:5672,
  2. RABBIT_USER:RABBIT_PASS@rabbit2:5672,RABBIT_USER:RABBIT_PASS@rabbit3:5672
  3. rabbit_retry_interval=1
  4. rabbit_retry_backoff=2
  5. rabbit_max_retries=0
  6. rabbit_durable_queues=true
  7. rabbit_ha_queues=true

For the Old Set-Up

  1. # rabbitmqctl stop_app
  2. # rabbitmqctl reset
  3. # rabbitmqctl start_app

Create RabbitMQ Admin

  1. [root@controller1 ~]# rabbitmqctl  add_user admin admin
  2. Creating user "admin" ...
  3. [root@controller1 ~]# rabbitmqctl  set_user_tags admin administrator
  4. Setting tags for user "admin" to [administrator] ...
  5. [root@controller1 ~]# rabbitmqctl add_user openstack openstack
  6. Creating user "openstack" ...
  7. [root@controller1 ~]# rabbitmqctl set_permissions openstack ".*" ".*" ".*" 
  8. Setting permissions for user "openstack" in vhost "/" ...
  9. [root@controller1 ~]# rabbitmqctl  set_user_tags openstack administrator
  10. Setting tags for user "openstack" to [administrator] ...
  11. [root@controller1 ~]# systemctl restart rabbitmq-server.service