Highly available Nova API

Posted on by admin

Create Nova Service & Endpoint

  1. [root@controller1 ~]# openstack service create --name nova --description "OpenStack Compute" compute
  2. +-------------+----------------------------------+
  3. | Field       | Value                            |
  4. +-------------+----------------------------------+
  5. | description | OpenStack Compute                |
  6. | enabled     | True                             |
  7. | id          | 365c1378f8c641ba81f48efa7c62cd29 |
  8. | name        | nova                             |
  9. | type        | compute                          |
  10. +-------------+----------------------------------+
  11. [root@controller1 ~]# openstack endpoint create --region RegionOne compute public http://controller:8774/v2.1
  12. +--------------+----------------------------------+
  13. | Field        | Value                            |
  14. +--------------+----------------------------------+
  15. | enabled      | True                             |
  16. | id           | 2afb267d1bd34f9388f9568321313167 |
  17. | interface    | public                           |
  18. | region       | RegionOne                        |
  19. | region_id    | RegionOne                        |
  20. | service_id   | 365c1378f8c641ba81f48efa7c62cd29 |
  21. | service_name | nova                             |
  22. | service_type | compute                          |
  23. | url          | http://controller:8774/v2.1      |
  24. +--------------+----------------------------------+
  25. [root@controller1 ~]# openstack endpoint create --region RegionOne compute internal http://controller:8774/v2.1
  26. +--------------+----------------------------------+
  27. | Field        | Value                            |
  28. +--------------+----------------------------------+
  29. | enabled      | True                             |
  30. | id           | 92de622e904546cc9bd1ca4087529e98 |
  31. | interface    | internal                         |
  32. | region       | RegionOne                        |
  33. | region_id    | RegionOne                        |
  34. | service_id   | 365c1378f8c641ba81f48efa7c62cd29 |
  35. | service_name | nova                             |
  36. | service_type | compute                          |
  37. | url          | http://controller:8774/v2.1      |
  38. +--------------+----------------------------------+
  39. [root@controller1 ~]# openstack endpoint create --region RegionOne compute admin http://controller:8774/v2.1
  40. +--------------+----------------------------------+
  41. | Field        | Value                            |
  42. +--------------+----------------------------------+
  43. | enabled      | True                             |
  44. | id           | 9295b974edec4e06842ee4db2e2d2458 |
  45. | interface    | admin                            |
  46. | region       | RegionOne                        |
  47. | region_id    | RegionOne                        |
  48. | service_id   | 365c1378f8c641ba81f48efa7c62cd29 |
  49. | service_name | nova                             |
  50. | service_type | compute                          |
  51. | url          | http://controller:8774/v2.1      |
  52. +--------------+----------------------------------+

Create Placement & Endpoint

  1. [root@controller1 ~]# openstack user create --domain default --password=placement placement
  2. +---------------------+----------------------------------+
  3. | Field               | Value                            |
  4. +---------------------+----------------------------------+
  5. | domain_id           | default                          |
  6. | enabled             | True                             |
  7. | id                  | c18a3e3bf63c4114924fb6b31b3305fd |
  8. | name                | placement                        |
  9. | options             | {}                               |
  10. | password_expires_at | None                             |
  11. +---------------------+----------------------------------+
  12. [root@controller1 ~]# openstack role add --project service --user placement admin
  13. [root@controller1 ~]# openstack service create --name placement --description "Placement API" placement
  14. +-------------+----------------------------------+
  15. | Field       | Value                            |
  16. +-------------+----------------------------------+
  17. | description | Placement API                    |
  18. | enabled     | True                             |
  19. | id          | 99a48fd2c3bf496287fa1fcf82376c02 |
  20. | name        | placement                        |
  21. | type        | placement                        |
  22. +-------------+----------------------------------+
  23. [root@controller1 ~]# openstack endpoint create --region RegionOne placement public http://controller:8778
  24. +--------------+----------------------------------+
  25. | Field        | Value                            |
  26. +--------------+----------------------------------+
  27. | enabled      | True                             |
  28. | id           | c271472e882f49c3a77c518296d4045c |
  29. | interface    | public                           |
  30. | region       | RegionOne                        |
  31. | region_id    | RegionOne                        |
  32. | service_id   | 99a48fd2c3bf496287fa1fcf82376c02 |
  33. | service_name | placement                        |
  34. | service_type | placement                        |
  35. | url          | http://controller:8778           |
  36. +--------------+----------------------------------+
  37. [root@controller1 ~]# openstack endpoint create --region RegionOne placement internal http://controller:8778
  38. +--------------+----------------------------------+
  39. | Field        | Value                            |
  40. +--------------+----------------------------------+
  41. | enabled      | True                             |
  42. | id           | b04f59956eca4d8cab4b3a41a071fc6b |
  43. | interface    | internal                         |
  44. | region       | RegionOne                        |
  45. | region_id    | RegionOne                        |
  46. | service_id   | 99a48fd2c3bf496287fa1fcf82376c02 |
  47. | service_name | placement                        |
  48. | service_type | placement                        |
  49. | url          | http://controller:8778           |
  50. +--------------+----------------------------------+
  51. [root@controller1 ~]# openstack endpoint create --region RegionOne placement admin http://controller:8778
  52. +--------------+----------------------------------+
  53. | Field        | Value                            |
  54. +--------------+----------------------------------+
  55. | enabled      | True                             |
  56. | id           | 582bf2a041734838adbf92d7dd4b602e |
  57. | interface    | admin                            |
  58. | region       | RegionOne                        |
  59. | region_id    | RegionOne                        |
  60. | service_id   | 99a48fd2c3bf496287fa1fcf82376c02 |
  61. | service_name | placement                        |
  62. | service_type | placement                        |
  63. | url          | http://controller:8778           |
  64. +--------------+----------------------------------+

Install Nova

  1. [root@controller ~]# yum install openstack-nova-api openstack-nova-conductor openstack-nova-cert openstack-nova-console openstack-nova-novncproxy openstack-nova-scheduler openstack-nova-placement-api -y

Configure Nova

  1. [root@controller1 ~]# cp /etc/nova/nova.conf{,.bak}
  2.  
  3. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf DEFAULT my_ip controller1
  4. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf DEFAULT use_neutron True
  5. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf DEFAULT osapi_compute_listen controller1
  6. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf DEFAULT osapi_compute_listen_port 8774
  7. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf DEFAULT metadata_listen controller1
  8. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf DEFAULT metadata_listen_port 8775
  9. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf DEFAULT firewall_driver nova.virt.firewall.NoopFirewallDriver
  10. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf DEFAULT enabled_apis osapi_compute,metadata
  11. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf DEFAULT transport_url rabbit://openstack:[email protected]:5673
  12.  
  13. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf database connection mysql+pymysql://nova:[email protected]/nova
  14. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf api_database connection mysql+pymysql://nova:[email protected]/nova_api
  15.  
  16. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf api auth_strategy keystone
  17.  
  18. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf keystone_authtoken auth_uri http://controller:5000
  19. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf keystone_authtoken auth_url http://controller:35357
  20. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf keystone_authtoken memcached_servers controller1:11211,controller2:11211,controller3:11211
  21. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf keystone_authtoken auth_type password
  22. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf keystone_authtoken project_domain_name default
  23. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf keystone_authtoken user_domain_name default
  24. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf keystone_authtoken project_name service
  25. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf keystone_authtoken username nova
  26. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf keystone_authtoken password nova
  27.  
  28. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf vnc enabled true
  29. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf vnc vncserver_listen controller1
  30. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf vnc vncserver_proxyclient_address controller1
  31. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf vnc novncproxy_host controller1
  32. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf vnc novncproxy_port 6080
  33.  
  34. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf glance api_servers http://controller:9292
  35.  
  36. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf oslo_concurrency lock_path /var/lib/nova/tmp
  37.  
  38. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf placement os_region_name RegionOne
  39. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf placement project_domain_name Default
  40. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf placement project_name service
  41. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf placement auth_type password
  42. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf placement user_domain_name Default
  43. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf placement auth_url  http://controller:35357/v3
  44. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf placement username placement
  45. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf placement password placement
  46.  
  47. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf scheduler discover_hosts_in_cells_interval 300
  48.  
  49. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf cache enabled true
  50. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf cache backend oslo_cache.memcache_pool
  51. [root@controller1 ~]# openstack-config --set /etc/nova/nova.conf cache memcache_servers controller1:11211,controller2:11211,controller3:11211

Configure Nova-Placement

Refer to “CLOUD COMPUTING —> OpenStack Pike Installation —> 7.Nova”

  1. [root@controller1 ~]# echo "
  2. > #Placement API
  3. > <Directory /usr/bin>
  4. >    <IfVersion >= 2.4>
  5. >       Require all granted
  6. >    </IfVersion>
  7. >    <IfVersion < 2.4>
  8. >       Order allow,deny
  9. >       Allow from all
  10. >    </IfVersion>
  11. > </Directory>
  12. > ">>/etc/httpd/conf.d/00-nova-placement-api.conf
  13. [[email protected] ~]# "
  14. [root@controller1 ~]# systemctl restart httpd

Synchronize Database

  1. [root@controller1 ~]# su -s /bin/sh -c "nova-manage api_db sync" nova
  2. [root@controller1 ~]# su -s /bin/sh -c "nova-manage cell_v2 map_cell0" nova
  3. [root@controller1 ~]# su -s /bin/sh -c "nova-manage cell_v2 create_cell --name=cell1 --verbose" nova
  4. [root@controller1 ~]# su -s /bin/sh -c "nova-manage db sync" nova

List Nova Cell

  1. [root@controller1 ~]# nova-manage cell_v2 list_cells
  2. +-------+--------------------------------------+-----------------------------------------+-------------------------------------------------+
  3. |  Name |                 UUID                 |              Transport URL              |               Database Connection               |
  4. +-------+--------------------------------------+-----------------------------------------+-------------------------------------------------+
  5. | cell0 | 00000000-0000-0000-0000-000000000000 |                  none:/                 | mysql+pymysql://nova:****@controller/nova_cell0 |
  6. | cell1 | b55310bc-6d58-4c8d-9b7c-014b77cd48ac | rabbit://openstack:****@controller:5673 |    mysql+pymysql://nova:****@controller/nova    |
  7. +-------+--------------------------------------+-----------------------------------------+-------------------------------------------------+

Replace Default IP

  1. [root@controller1 ~]# sed -i 's/8778/9778/' /etc/httpd/conf.d/00-nova-placement-api.conf
  2. [root@controller1 ~]# systemctl restart httpd

Configure HA With HAProxy

  1. [root@controller1 ~]# echo '
  2. > ##nova_compute
  3. > listen nova_compute_api_cluster
  4. >   bind controller:8774
  5. >   balance source
  6. >   option tcpka
  7. >   option httpchk
  8. >   option tcplog
  9. >   server controller1 controller1:8774 check inter 2000 rise 2 fall 5
  10. >   server controller2 controller2:8774 check inter 2000 rise 2 fall 5
  11. >   server controller3 controller3:8774 check inter 2000 rise 2 fall 5
  12. > #Nova-api-metadata
  13. > listen Nova-api-metadata_cluster
  14. >   bind controller:8775
  15. >   balance source
  16. >   option tcpka
  17. >   option httpchk
  18. >   option tcplog
  19. >   server controller1 controller1:8775 check inter 2000 rise 2 fall 5
  20. >   server controller2 controller2:8775 check inter 2000 rise 2 fall 5
  21. >   server controller3 controller3:8775 check inter 2000 rise 2 fall 5
  22. > #nova_placement
  23. > listen nova_placement_cluster
  24. >   bind controller:8778
  25. >   balance source
  26. >   option tcpka
  27. >   option tcplog
  28. >   server controller1 controller1:9778 check inter 2000 rise 2 fall 5
  29. >   server controller2 controller2:9778 check inter 2000 rise 2 fall 5
  30. >  server controller3 controller3:9778 check inter 2000 rise 2 fall 5
  31. > '>>/etc/haproxy/haproxy.cfg
  32. [[email protected] ~]# '
  33. [root@controller1 ~]# systemctl restart haproxy.service
  34.  
  35. netstat -antp|grep haproxy
  36. netstat -antp|egrep '8774|8775|8778|6080'

Start Nova Service

  1. [root@controller ~]# systemctl enable openstack-nova-api.service openstack-nova-consoleauth.service openstack-nova-scheduler.service openstack-nova-conductor.service openstack-nova-novncproxy.service
  2. [root@controller ~]# systemctl restart openstack-nova-api.service openstack-nova-consoleauth.service openstack-nova-scheduler.service openstack-nova-conductor.service openstack-nova-novncproxy.service
  3. [root@controller ~]# systemctl status openstack-nova-api.service openstack-nova-consoleauth.service openstack-nova-scheduler.service openstack-nova-conductor.service openstack-nova-novncproxy.service

List Nova Status

  1. [root@controller1 ~]# openstack catalog list
  2. +-----------+-----------+-----------------------------------------+
  3. | Name      | Type      | Endpoints                               |
  4. +-----------+-----------+-----------------------------------------+
  5. | glance    | image     | RegionOne                               |
  6. |           |           |   internal: http://controller:9292      |
  7. |           |           | RegionOne                               |
  8. |           |           |   admin: http://controller:9292         |
  9. |           |           | RegionOne                               |
  10. |           |           |   public: http://controller:9292        |
  11. |           |           |                                         |
  12. | nova      | compute   | RegionOne                               |
  13. |           |           |   public: http://controller:8774/v2.1   |
  14. |           |           | RegionOne                               |
  15. |           |           |   admin: http://controller:8774/v2.1    |
  16. |           |           | RegionOne                               |
  17. |           |           |   internal: http://controller:8774/v2.1 |
  18. |           |           |                                         |
  19. | placement | placement | RegionOne                               |
  20. |           |           |   admin: http://controller:8778         |
  21. |           |           | RegionOne                               |
  22. |           |           |   internal: http://controller:8778      |
  23. |           |           | RegionOne                               |
  24. |           |           |   public: http://controller:8778        |
  25. |           |           |                                         |
  26. | neutron   | network   | RegionOne                               |
  27. |           |           |   admin: http://controller:9696         |
  28. |           |           | RegionOne                               |
  29. |           |           |   public: http://controller:9696        |
  30. |           |           | RegionOne                               |
  31. |           |           |   internal: http://controller:9696      |
  32. |           |           |                                         |
  33. | keystone  | identity  | RegionOne                               |
  34. |           |           |   admin: http://controller:35357/v3/    |
  35. |           |           | RegionOne                               |
  36. |           |           |   internal: http://controller:5000/v3/  |
  37. |           |           | RegionOne                               |
  38. |           |           |   public: http://controller:5000/v3/    |
  39. |           |           |                                         |
  40. +-----------+-----------+-----------------------------------------+
  41. [root@controller1 ~]# nova-status upgrade check
  42. +--------------------------------------------------------------------+
  43. | Upgrade Check Results                                              |
  44. +--------------------------------------------------------------------+
  45. | Check: Cells v2                                                    |
  46. | Result: Success                                                    |
  47. | Details: No host mappings or compute nodes were found. Remember to |
  48. |   run command 'nova-manage cell_v2 discover_hosts' when new        |
  49. |   compute hosts are deployed.                                      |
  50. +--------------------------------------------------------------------+
  51. | Check: Placement API                                               |
  52. | Result: Success                                                    |
  53. | Details: None                                                      |
  54. +--------------------------------------------------------------------+
  55. | Check: Resource Providers                                          |
  56. | Result: Success                                                    |
  57. | Details: There are no compute resource providers in the Placement  |
  58. |   service nor are there compute nodes in the database.             |
  59. |   Remember to configure new compute nodes to report into the       |
  60. |   Placement service. See                                           |
  61. |   http://docs.openstack.org/developer/nova/placement.html          |
  62. |   for more details.                                                |
  63. +--------------------------------------------------------------------+
  64. [root@controller1 ~]# openstack compute service list
  65. +----+------------------+-------------+----------+---------+-------+----------------------------+
  66. | ID | Binary           | Host        | Zone     | Status  | State | Updated At                 |
  67. +----+------------------+-------------+----------+---------+-------+----------------------------+
  68. | 15 | nova-conductor   | controller1 | internal | enabled | up    | 2017-12-19T09:02:58.000000 |
  69. | 18 | nova-scheduler   | controller1 | internal | enabled | up    | 2017-12-19T09:02:56.000000 |
  70. | 21 | nova-consoleauth | controller1 | internal | enabled | up    | 2017-12-19T09:02:59.000000 |
  71. | 27 | nova-consoleauth | controller2 | internal | enabled | up    | 2017-12-19T09:03:04.000000 |
  72. | 30 | nova-conductor   | controller2 | internal | enabled | up    | 2017-12-19T09:03:03.000000 |
  73. | 39 | nova-scheduler   | controller2 | internal | enabled | up    | 2017-12-19T09:02:57.000000 |
  74. | 42 | nova-consoleauth | controller3 | internal | enabled | up    | 2017-12-19T09:03:04.000000 |
  75. | 45 | nova-conductor   | controller3 | internal | enabled | up    | 2017-12-19T09:03:05.000000 |
  76. | 54 | nova-scheduler   | controller3 | internal | enabled | up    | 2017-12-19T09:02:59.000000 |
  77. | 57 | nova-compute     | compute01   | nova     | enabled | up    | 2017-12-19T09:02:59.000000 |
  78. +----+------------------+-------------+----------+---------+-------+----------------------------+

Install On Other Controller Nodes

  1. [root@controller2 ~]# yum install openstack-nova-api openstack-nova-conductor openstack-nova-console openstack-nova-novncproxy openstack-nova-scheduler openstack-nova-placement-api -y 
  2.  
  3. [root@controller2 ~]# rsync -avzP  -e 'ssh -p 22'  controller1:/etc/nova/*  /etc/nova/
  4. [root@controller2 ~]# rsync -avzP  -e 'ssh -p 22'  controller1:/etc/httpd/conf.d/00-nova-placement-api.conf /etc/httpd/conf.d/
  5. [root@controller2 ~]# rsync -avzP  -e 'ssh -p 22'  controller1:/etc/haproxy/* /etc/haproxy/
  6.  
  7. [root@controller2 ~]# sed -i '1,9s/controller1/controller2/'  /etc/nova/nova.conf
  8. [root@controller3 ~]# sed -i '1,9s/controller1/controller3/'  /etc/nova/nova.conf
  9.  
  10. [root@controller2 ~]# systemctl enable openstack-nova-api.service openstack-nova-consoleauth.service openstack-nova-scheduler.service openstack-nova-conductor.service openstack-nova-novncproxy.service
  11. [root@controller2 ~]# systemctl restart openstack-nova-api.service openstack-nova-consoleauth.service openstack-nova-scheduler.service openstack-nova-conductor.service openstack-nova-novncproxy.service
  12. [root@controller2 ~]# systemctl status openstack-nova-api.service openstack-nova-consoleauth.service openstack-nova-scheduler.service openstack-nova-conductor.service openstack-nova-novncproxy.service
  13.  
  14. [root@controller2 ~]# systemctl restart httpd haproxy

Discover New Compute Node

  1. [root@controller1 ~]# su -s /bin/sh -c "nova-manage cell_v2 discover_hosts --verbose" nova
  2. Found 2 cell mappings.
  3. Skipping cell0 since it does not contain hosts.
  4. Getting compute nodes from cell 'cell1': b55310bc-6d58-4c8d-9b7c-014b77cd48ac
  5. Found 0 unmapped computes in cell: b55310bc-6d58-4c8d-9b7c-014b77cd48ac
  6. [root@controller1 ~]# openstack compute service list
  7. +----+------------------+-------------+----------+---------+-------+----------------------------+
  8. | ID | Binary           | Host        | Zone     | Status  | State | Updated At                 |
  9. +----+------------------+-------------+----------+---------+-------+----------------------------+
  10. | 15 | nova-conductor   | controller1 | internal | enabled | up    | 2017-12-19T09:28:08.000000 |
  11. | 18 | nova-scheduler   | controller1 | internal | enabled | up    | 2017-12-19T09:28:16.000000 |
  12. | 21 | nova-consoleauth | controller1 | internal | enabled | up    | 2017-12-19T09:28:10.000000 |
  13. | 27 | nova-consoleauth | controller2 | internal | enabled | up    | 2017-12-19T09:28:15.000000 |
  14. | 30 | nova-conductor   | controller2 | internal | enabled | up    | 2017-12-19T09:28:13.000000 |
  15. | 39 | nova-scheduler   | controller2 | internal | enabled | up    | 2017-12-19T09:28:17.000000 |
  16. | 42 | nova-consoleauth | controller3 | internal | enabled | up    | 2017-12-19T09:28:15.000000 |
  17. | 45 | nova-conductor   | controller3 | internal | enabled | up    | 2017-12-19T09:28:15.000000 |
  18. | 54 | nova-scheduler   | controller3 | internal | enabled | up    | 2017-12-19T09:28:09.000000 |
  19. | 57 | nova-compute     | compute01   | nova     | enabled | up    | 2017-12-19T09:28:10.000000 |
  20. | 60 | nova-compute     | compute02   | nova     | enabled | up    | 2017-12-19T09:28:16.000000 |
  21. +----+------------------+-------------+----------+---------+-------+----------------------------+