Waiting For Test ,,,

Fdisk the 2nd Disk

[root@localhost ~]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p):
Using default response p
Partition number (1-4, default 1):
First sector (2048-125829119, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-125829119, default 125829119): +30G
Partition 1 of type Linux and of size 30 GiB is set

Command (m for help): n
Partition type:
p primary (1 primary, 0 extended, 3 free)
e extended
Select (default p):
Using default response p
Partition number (2-4, default 2):
First sector (62916608-125829119, default 62916608):
Using default value 62916608
Last sector, +sectors or +size{K,M,G} (62916608-125829119, default 125829119):
Using default value 125829119
Partition 2 of type Linux and of size 30 GiB is set

Command (m for help): p

Disk /dev/sdb: 64.4 GB, 64424509440 bytes, 125829120 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x285890cb

Device Boot Start End Blocks Id System
/dev/sdb1 2048 62916607 31457280 83 Linux
/dev/sdb2 62916608 125829119 31456256 83 Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[root@localhost ~]# ls -al /dev/sd*
brw-rw—-. 1 root disk 8, 0 Dec 26 2017 /dev/sda
brw-rw—-. 1 root disk 8, 1 Dec 26 2017 /dev/sda1
brw-rw—-. 1 root disk 8, 2 Dec 26 2017 /dev/sda2
brw-rw—-. 1 root disk 8, 16 Dec 26 01:11 /dev/sdb
brw-rw—-. 1 root disk 8, 17 Dec 26 01:11 /dev/sdb1
brw-rw—-. 1 root disk 8, 18 Dec 26 01:11 /dev/sdb2

Format 2nd Disk

[root@localhost ~]# mkfs.ext4 /dev/sdb1
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
1966080 inodes, 7864320 blocks
393216 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2155872256
240 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

[root@localhost ~]# mkfs.ext4 /dev/sdb2
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
1966080 inodes, 7864064 blocks
393203 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2155872256
240 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

Mount /dev/sdb1

[root@localhost ~]# mkdir -p /date
[root@localhost ~]# mount -t ext4 /dev/sdb1 /date
[root@localhost ~]# df -hgrep /dev/sdb1
/dev/sdb1 30G 45M 28G 1% /date
[root@localhost ~]# echo “mount -t ext4 /dev/sdb1 /date” >>/etc/rc.d/rc.local
[root@localhost ~]# tail -1 /etc/rc.d/rc.local
mount -t ext4 /dev/sdb1 /date
[root@localhost ~]# chmod +x /etc/rc.d/rc.local

Create Volume on /dev/sdb2

[root@localhost ~]# yum install lvm2 -y
[root@localhost ~]# systemctl enable lvm2-lvmetad.service
Created symlink from /etc/systemd/system/sysinit.target.wants/lvm2-lvmetad.service to /usr/lib/systemd/system/lvm2-lvmetad.service.
[root@localhost ~]# systemctl start lvm2-lvmetad.service

[root@localhost ~]# pvcreate /dev/sdb2
WARNING: ext4 signature detected on /dev/sdb2 at offset 1080. Wipe it? [y/n]: y
Wiping ext4 signature on /dev/sdb2.
Physical volume “/dev/sdb2” successfully created.
[root@localhost ~]# vgcreate cinder_lvm01 /dev/sdb2
Volume group “cinder_lvm01” successfully created
[root@localhost ~]# vgdisplay
— Volume group —
VG Name cinder_lvm01
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size <30.00 GiB
PE Size 4.00 MiB
Total PE 7679
Alloc PE / Size 0 / 0
Free PE / Size 7679 / <30.00 GiB
VG UUID jyb299-bo5k-E6Z3-Frho-e6Kz-d9Mu-yK0m6c

— Volume group —
VG Name centos
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 4
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 3
Open LV 3
Max PV 0
Cur PV 1
Act PV 1
VG Size <79.00 GiB
PE Size 4.00 MiB
Total PE 20223
Alloc PE / Size 20222 / 78.99 GiB
Free PE / Size 1 / 4.00 MiB
VG UUID NHtSF8-nozf-sbB4-vEBt-ogJo-WmuG-F8HYMQ

Install NFS

[root@localhost ~]# yum install nfs-utils rpcbind -y
[root@localhost ~]# mkdir -p /date/{cinder_nfs1,cinder_nfs2}
[root@localhost ~]# chmod -R 777 /date
[root@localhost ~]# echo “/date/cinder_nfs1 *(rw,root_squash,sync,anonuid=165,anongid=165)”>/etc/exports
[root@localhost ~]# exportfs -r
[root@localhost ~]# systemctl enable rpcbind nfs-server
Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.
[root@localhost ~]# systemctl restart rpcbind nfs-server
[root@localhost ~]# showmount -e localhost
Export list for localhost:
/date/cinder_nfs1 *

Install Cinder & Configure

[root@localhost ~]# yum install openstack-cinder targetcli python-keystone lvm2 -y
[root@localhost ~]# cp /etc/cinder/cinder.conf{,.bak}
[root@localhost ~]# cp /etc/lvm/lvm.conf{,.bak
[root@localhost ~]# sed -i ‘141a filter = [ “a/sdb2/“, “r/.*/“]‘ /etc/lvm/lvm.conf
[root@localhost ~]# echo ‘192.168.220.101:/date/cinder_nfs1’>/etc/cinder/nfs_shares
[root@localhost ~]# chmod 640 /etc/cinder/nfs_shares
[root@localhost ~]# chown root:cinder /etc/cinder/nfs_shares

Configure cinder.conf

[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf DEFAULT auth_strategy keystone
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf DEFAULT log_dir /var/log/cinder
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf DEFAULT state_path /var/lib/cinder
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf DEFAULT glance_api_servers http://controller:9292
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf DEFAULT transport_url rabbit://openstack:openstack@controller
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf DEFAULT enabled_backends lvm,nfs

[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf database connection mysql+pymysql://cinder:cinder@controller/cinder

[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf keystone_authtoken auth_uri http://controller:5000
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf keystone_authtoken auth_url http://controller:35357
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf keystone_authtoken memcached_servers controller1:11211,controller2:11211,controller3:11211
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf keystone_authtoken auth_type password
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf keystone_authtoken project_domain_name default
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf keystone_authtoken user_domain_name default
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf keystone_authtoken project_name service
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf keystone_authtoken username cinder
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf keystone_authtoken password cinder

[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf oslo_concurrency lock_path /var/lib/cinder/tmp

[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf lvm volume_driver cinder.volume.drivers.lvm.LVMVolumeDriver
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf lvm iscsi_helper lioadm
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf lvm iscsi_protocol iscsi
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf lvm volume_group cinder_lvm01
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf lvm iscsi_ip_address 192.168.220.101
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf lvm volumes_dir $state_path/volumes
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf lvm volume_backend_name lvm01

[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf nfs volume_driver = cinder.volume.drivers.nfs.NfsDriver
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf nfs nfs_shares_config = /etc/cinder/nfs_shares
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf nfs nfs_mount_point_base = $state_path/mnt
[root@localhost ~]# openstack-config –set /etc/cinder/cinder.conf nfs volume_backend_name = nfs01

Start Cinder Service

[root@localhost ~]# chmod 640 /etc/cinder/cinder.conf
[root@localhost ~]# chgrp cinder /etc/cinder/cinder.conf

[root@localhost ~]# systemctl enable openstack-cinder-volume.service target.service
Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-cinder-volume.service to /usr/lib/systemd/system/openstack-cinder-volume.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/target.service to /usr/lib/systemd/system/target.service.
[root@localhost ~]# systemctl start openstack-cinder-volume.service target.service

Add Cinder to Pacemaker

# pcs resource create openstack-cinder-api systemd:openstack-cinder-api –clone interleave=true

pcs resource create openstack-cinder-scheduler systemd:openstack-cinder-scheduler –clone interleave=true

pcs resource create openstack-cinder-volume systemd:openstack-cinder-volume

pcs constraint order start openstack-cinder-api-clone then openstack-cinder-scheduler-clone

pcs constraint colocation add openstack-cinder-scheduler-clone with openstack-cinder-api-clone

pcs constraint order start openstack-cinder-scheduler-clone then openstack-cinder-volume

pcs constraint colocation add openstack-cinder-volume with openstack-cinder-scheduler-clone

pcs constraint order start openstack-keystone-clone then openstack-cinder-api-clone

Configure Cinder service

# cp /etc/cinder/cinder.conf /etc/cinder/cinder.conf.bak

openstack-config –set /etc/cinder/cinder.conf DEFAULT host cinder-cluster-1

openstack-config –set /etc/cinder/cinder.conf DEFAULT osapi_volume_listen 10.0.0.11

openstack-config –set /etc/cinder/cinder.conf DEFAULT auth_strategy keystone

openstack-config –set /etc/cinder/cinder.conf DEFAULT control_exchange cinder

openstack-config –set /etc/cinder/cinder.conf DEFAULT volume_driver cinder.volume.drivers.nfs.NfsDriver

openstack-config –set /etc/cinder/cinder.conf DEFAULT nfs_shares_config /etc/cinder/nfs_exports

openstack-config –set /etc/cinder/cinder.conf DEFAULT nfs_sparsed_volumes true

openstack-config –set /etc/cinder/cinder.conf DEFAULT nfs_mount_options v3

openstack-config –set /etc/cinder/cinder.conf database connection mysql://cinder:password@10.0.0.11/cinder

openstack-config –set /etc/cinder/cinder.conf database max_retries -1

openstack-config –set /etc/cinder/cinder.conf keystone_authtoken identity_uri http://10.0.0.11:35357/

openstack-config –set /etc/cinder/cinder.conf keystone_authtoken auth_uri http://10.0.0.11:5000/

openstack-config –set /etc/cinder/cinder.conf keystone_authtoken admin_tenant_name service

openstack-config –set /etc/cinder/cinder.conf keystone_authtoken admin_user cinder

openstack-config –set /etc/cinder/cinder.conf keystone_authtoken admin_password CINDER_PASS

openstack-config –set /etc/cinder/cinder.conf rabbit_hosts 10.0.0.12,10.0.0.13,10.0.0.14

openstack-config –set /etc/cinder/cinder.conf rabbit_ha_queues True

openstack-config –set /etc/cinder/cinder.conf heartbeat_timeout_threshold 60

openstack-config –set /etc/cinder/cinder.conf heartbeat_rate 2

Configure HA Cinder API

$ openstack endpoint create volume –region $KEYSTONE_REGION \
–publicurl ‘http://10.0.0.11:8776/v1/%(tenant\_id)s' \
–adminurl ‘http://10.0.0.11:8776/v1/%(tenant\_id)s' \
–internalurl ‘http://10.0.0.11:8776/v1/%(tenant\_id)s'

Water's Home

Highly available Block Storage API

Config Network