Xen Virtualization on Linux and Solaris

Sunday, March 09, 2014

Setup Gluster 3.4.2 on Two Node Controller&Compute Neutron GRE+OVS Fedora 20 Cluster

*********************************************************************************
Update 08/14/2014 location of 3.5.2 RPMS http://download.gluster.org/pub/gluster/glusterfs/3.5/3.5.2/EPEL.repo/epel-7/x86_64/ for CentOS 7
*********************************************************************************
This post is an update for http://bderzhavets.blogspot.com/2014/01/setting-up-two-physical-node-openstack.html . It's focused on Gluster 3.4.2 implementation including tuning /etc/sysconfig/iptables files on Controller and Compute Nodes.
Copying ssh-key from master node to compute, step by step verification of gluster volume replica 2 functionality and switching RDO Havana cinder services to work with gluster volume created to store instances bootable cinders volumes for performance improvement. Of course creating gluster bricks under "/" is not recommended . It should be a separate mount point for "xfs" filesystem to store gluster bricks on each node. View also VNC Console in Dashboard on Two Node Neutron GRE+OVS+Gluster F20 Cluster
Typo in dhcp_agent.ini :- the reference for "dnsmasq.conf" has been fixed and line "dhcp-option=26,1454" added to dnsmasq.conf . Updated configuration files are critical for launching instance without "Customization script" and allow to work with usual ssh keypair. Actually , when updates are done instance gets created with MTU 1454. View http://bderzhavets.blogspot.com/2014/03/tuning-neutron-dhcp-agent-dnsmasq-on.html

Have to notice that http://kashyapc.fedorapeople.org/virt/openstack/neutron-configs-GRE-OVS-two-node.txt in meantime (03/26/2014) has dhcp_agent.ini and l3_agent.ini identical (i.e. the last one is missing rows

metadata_ip = 192.168.1.127
metadata_port = 8700

# and contains instead of them reference:-

dnsmasq_config_file = /etc/neutron/dnsmasq.conf

)

It is manual Setup

- Controller node: Nova, Keystone, Cinder, Glance, Neutron (using Open vSwitch plugin and GRE tunneling )

- Compute node: Nova (nova-compute), Neutron (openvswitch-agent)

Basic updated setup procedure

Final draft for Nova & Neutron Configuration files

dallas1.localdomain - Controller (192.168.1.130)
dallas2.localdomain - Compute   (192.168.1.140)

First step is tuning /etc/sysconfig/iptables for IPv4 iptables firewall (service firewalld should be disabled) :-

Update /etc/sysconfig/iptables on both nodes:-

-A INPUT -p tcp -m multiport --dport 24007:24047 -j ACCEPT
-A INPUT -p tcp --dport 111 -j ACCEPT
-A INPUT -p udp --dport 111 -j ACCEPT
-A INPUT -p tcp -m multiport --dport 38465:38485 -j ACCEPT

Comment out lines bellow , ignoring instruction from http://kashyapc.fedorapeople.org/virt/openstack/neutron-configs-GRE-OVS-two-node.txt . It's critical for Gluster functionality. Having them active you are supposed to work with thin LVM as cinder volumes. You won't be able even remote mount with "-t glusterfs" option. Gluster replications will be dead for ever.

# -A FORWARD -j REJECT --reject-with icmp-host-prohibited
# -A INPUT -j REJECT --reject-with icmp-host-prohibited

Restart service iptables on both nodes

Second step:-

On dallas1, run the following commands :

# ssh-keygen (Hit Enter to accept all of the defaults)
# ssh-copy-id -i ~/.ssh/id_rsa.pub root@dallas2

On both nodes run :-

# yum -y install glusterfs glusterfs-server glusterfs-fuse
# service glusterd start

On dallas1

#gluster peer probe dallas2.localdomain

Should return "success"

[root@dallas1 ~(keystone_admin)]$ gluster peer status
Number of Peers: 1

Hostname: dallas2.localdomain
Uuid: b3b1cf43-2fec-4904-82d4-b9be03f77c5f
State: Peer in Cluster (Connected)

On dallas2

[root@dallas2 ~]# gluster peer status
Number of Peers: 1

Hostname: 192.168.1.130
Uuid: a57433dd-4a1a-4442-a5ae-ba2f682e5c79
State: Peer in Cluster (Connected)

*************************************************************************
On Controller (192.168.1.130) and on Compute (192.168.1.140)
*************************************************************************

Verify ports availability:-

[root@dallas1 ~(keystone_admin)]$ netstat -lntp | grep gluster
tcp    0      0 0.0.0.0:655          0.0.0.0:*    LISTEN      2591/glusterfs
tcp    0      0 0.0.0.0:49152        0.0.0.0:*    LISTEN      2524/glusterfsd
tcp    0      0 0.0.0.0:2049          0.0.0.0:*    LISTEN      2591/glusterfs
tcp    0      0 0.0.0.0:38465        0.0.0.0:*    LISTEN      2591/glusterfs
tcp    0      0 0.0.0.0:38466        0.0.0.0:*    LISTEN      2591/glusterfs
tcp    0      0 0.0.0.0:49155        0.0.0.0:*    LISTEN      2525/glusterfsd
tcp    0      0 0.0.0.0:38468        0.0.0.0:*    LISTEN      2591/glusterfs
tcp    0      0 0.0.0.0:38469        0.0.0.0:*    LISTEN      2591/glusterfs
tcp    0      0 0.0.0.0:24007        0.0.0.0:*    LISTEN      2380/glusterd

************************************
Switching Cinder to Gluster volume
************************************

# gluster volume create cinder-volumes021 replica 2 ddallas1.localdomain:/FDR/Replicate dallas2.localdomain:/FDR/Replicate force

# gluster volume start cinder-volumes021

# gluster volume set cinder-volumes021 auth.allow 192.168.1.*

[root@dallas1 ~(keystone_admin)]$ gluster volume info cinder-volumes012

Volume Name: cinder-volumes012
Type: Replicate
Volume ID: 9ee31c6c-0ae3-4fee-9886-b9cb6a518f48
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: dallas1.localdomain:/FDR/Replicate
Brick2: dallas2.localdomain:/FDR/Replicate
Options Reconfigured:
storage.owner-gid: 165
storage.owner-uid: 165
auth.allow: 192.168.1.*

[root@dallas1 ~(keystone_admin)]$ gluster volume status cinder-volumes012
Status of volume: cinder-volumes012
Gluster process                          Port    Online    Pid
------------------------------------------------------------------------------
Brick dallas1.localdomain:/FDR/Replicate         49155    Y    2525
Brick dallas2.localdomain:/FDR/Replicate         49152    Y    1615
NFS Server on localhost                   2049    Y    2591
Self-heal Daemon on localhost                 N/A    Y    2596
NFS Server on dallas2.localdomain                       2049    Y    2202
Self-heal Daemon on dallas2.localdomain             N/A    Y    2197

# openstack-config --set /etc/cinder/cinder.conf DEFAULT volume_driver cinder.volume.drivers.glusterfs.GlusterfsDriver

# openstack-config --set /etc/cinder/cinder.conf DEFAULT glusterfs_shares_config /etc/cinder/shares.conf

# openstack-config --set /etc/cinder/cinder.conf DEFAULT glusterfs_mount_point_base /var/lib/cinder/volumes

# vi /etc/cinder/shares.conf
    192.168.1.130:/cinder-volumes021

:wq

Make sure all thin LVM have been deleted via `cinder list` , if no then delete them all.

[root@dallas1 ~(keystone_admin)]$ for i in api scheduler volume ; do service openstack-cinder-${i} restart ; done

It should add row to `df -h` output :

192.168.1.130:/cinder-volumes012 187G   32G 146G 18% /var/lib/cinder/volumes/1c9688348ab38662e3ac8fb121077d34

[root@dallas1 ~(keystone_admin)]$ openstack-status

== Nova services ==
openstack-nova-api:                    active
openstack-nova-cert:                   inactive (disabled on boot)
openstack-nova-compute:               inactive (disabled on boot)
openstack-nova-network:                inactive (disabled on boot)
openstack-nova-scheduler:             active
openstack-nova-volume:                  inactive (disabled on boot)
openstack-nova-conductor:             active
== Glance services ==
openstack-glance-api:                   active
openstack-glance-registry:           active
== Keystone service ==
openstack-keystone:                     active
== neutron services ==
neutron-server:                         active
neutron-dhcp-agent:                active
neutron-l3-agent:                     active
neutron-metadata-agent:        active
neutron-lbaas-agent:                    inactive (disabled on boot)
neutron-openvswitch-agent:       active
neutron-linuxbridge-agent:         inactive (disabled on boot)
neutron-ryu-agent:                      inactive (disabled on boot)
neutron-nec-agent:                      inactive (disabled on boot)
neutron-mlnx-agent:                   inactive (disabled on boot)
== Cinder services ==
openstack-cinder-api:                   active
openstack-cinder-scheduler:        active
openstack-cinder-volume:             active
== Support services ==
mysqld:                                 inactive (disabled on boot)
libvirtd:                               active
openvswitch:                            active
dbus:                                   active
tgtd:                                   active
qpidd:                                  active
== Keystone users ==
+----------------------------------+---------+---------+-------+
|                id                |   name | enabled | email |
+----------------------------------+---------+---------+-------+
| 871cf99617ff40e09039185aa7ab11f8 | admin |   True |       |
| df4a984ce2f24848a6b84aaa99e296f1 | boris |   True |       |
| 57fc5466230b497a9f206a20618dbe25 | cinder |   True |       |
| cdb2e5af7bae4c5486a1e3e2f42727f0 | glance |   True |       |
| adb14139a0874c74b14d61d2d4f22371 | neutron |   True |       |
| 2485122e3538409c8a6fa2ea4343cedf |   nova |   True |       |
+----------------------------------+---------+---------+-------+
== Glance images ==
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| ID                                   | Name                | Disk Format | Container Format | Size      | Status |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| 592faef8-308a-4438-867a-17adf685cde4 | CirrOS 31           | qcow2       | bare             | 13147648 | active |
| d0e90250-5814-4685-9b8d-65ec9daa7117 | Fedora 20 x86_64    | qcow2       | bare             | 214106112 | active |
| 3e6eea8e-32e6-4373-9eb1-e04b8a3167f9 | Ubuntu Server 13.10 | qcow2       | bare             | 244777472 | active |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
== Nova managed services ==
+----------------+---------------------+----------+---------+-------+----------------------------+-----------------+
| Binary         | Host                | Zone     | Status | State | Updated_at                 | Disabled Reason |
+----------------+---------------------+----------+---------+-------+----------------------------+-----------------+
| nova-scheduler | dallas1.localdomain | internal | enabled | up    | 2014-03-09T14:19:31.000000 | None            |
| nova-conductor | dallas1.localdomain | internal | enabled | up    | 2014-03-09T14:19:30.000000 | None            |
| nova-compute   | dallas2.localdomain | nova     | enabled | up    | 2014-03-09T14:19:33.000000 | None            |
+----------------+---------------------+----------+---------+-------+----------------------------+-----------------+
== Nova networks ==
+--------------------------------------+-------+------+
| ID                                   | Label | Cidr |
+--------------------------------------+-------+------+
| 0ed406bf-3552-4036-9006-440f3e69618e | ext   | None |
| 166d9651-d299-47df-a5a1-b368e87b612f | int   | None |
+--------------------------------------+-------+------+
== Nova instance flavors ==
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| ID | Name      | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| 1 | m1.tiny   | 512       | 1    | 0         |      | 1     | 1.0         | True      |
| 2 | m1.small | 2048      | 20   | 0         |      | 1     | 1.0         | True      |
| 3 | m1.medium | 4096      | 40   | 0         |      | 2     | 1.0         | True      |
| 4 | m1.large | 8192      | 80   | 0         |      | 4     | 1.0         | True      |
| 5 | m1.xlarge | 16384     | 160 | 0         |      | 8     | 1.0         | True      |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
== Nova instances ==
+----+------+--------+------------+-------------+----------+
| ID | Name | Status | Task State | Power State | Networks |
+----+------+--------+------------+-------------+----------+
+----+------+--------+------------+-------------+----------+

[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+-----------+--------+------------+-------------+-----------------------------+
| ID                                   | Name      | Status | Task State | Power State | Networks                    |
+--------------------------------------+-----------+--------+------------+-------------+-----------------------------+
| 9566adec-9406-4c3e-bce5-109ecb8bcf6b | VF20RS007 | ACTIVE | None       | Running     | int=10.0.0.2, 192.168.1.101 |
| 8142ee4c-ef56-4b61-8a0b-ecd82d21484f | VF20RS012 | ACTIVE | None       | Running     | int=10.0.0.4, 192.168.1.102 |
+--------------------------------------+-----------+--------+------------+-------------+-----------------------------+

[root@dallas1 ~(keystone_boris)]$ df -h
Filesystem                       Size Used Avail Use% Mounted on
/dev/mapper/fedora01-root        187G   32G 146G 18% /
devtmpfs                         3.9G     0 3.9G   0% /dev
tmpfs                            3.9G 184K 3.9G   1% /dev/shm
tmpfs                            3.9G 9.1M 3.9G   1% /run
tmpfs                            3.9G     0 3.9G   0% /sys/fs/cgroup
tmpfs                            3.9G 464K 3.9G   1% /tmp
/dev/sdb5                        477M 122M 327M 28% /boot
tmpfs                            3.9G 9.1M 3.9G   1% /run/netns
192.168.1.130:/cinder-volumes012 187G   32G 146G 18% /var/lib/cinder/volumes/1c9688348ab38662e3ac8fb121077d34

(neutron) agent-list
+--------------------------------------+--------------------+---------------------+-------+----------------+
| id                                   | agent_type         | host                | alive | admin_state_up |
+--------------------------------------+--------------------+---------------------+-------+----------------+
| 3ed1cd15-81af-4252-9d6f-e9bb140bf6cf | L3 agent           | dallas1.localdomain | :-)   | True           |
| a088a6df-633c-4959-a316-510c99f3876b | DHCP agent         | dallas1.localdomain | :-)   | True           |
| a3e5200c-b391-4930-b3ee-58c8d1b13c73 | Open vSwitch agent | dallas1.localdomain | :-)   | True           |
| b6da839a-0d93-44ad-9793-6d0919fbb547 | Open vSwitch agent | dallas2.localdomain | :-)   | True           |
+--------------------------------------+--------------------+---------------------+-------+----------------+

If Controller has been correctly set up:-

[root@dallas1 ~(keystone_admin)]$ netstat -lntp | grep python
tcp    0      0 0.0.0.0:8700         0.0.0.0:*     LISTEN      1160/python
tcp 0      0 0.0.0.0:35357       0.0.0.0:*     LISTEN      1163/python
tcp   0      0 0.0.0.0:9696      0.0.0.0:*      LISTEN      1165/python
tcp   0      0 0.0.0.0:8773          0.0.0.0:*      LISTEN      1160/python
tcp   0      0 0.0.0.0:8774          0.0.0.0:*      LISTEN      1160/python
tcp   0      0 0.0.0.0:9191          0.0.0.0:*      LISTEN      1173/python
tcp   0      0 0.0.0.0:8776          0.0.0.0:*      LISTEN      8169/python
tcp   0      0 0.0.0.0:5000          0.0.0.0:*      LISTEN      1163/python
tcp   0      0 0.0.0.0:9292          0.0.0.0:*      LISTEN      1168/python

**********************************************
Creating instance utilizing glusterfs volume
**********************************************

[root@dallas1 ~(keystone_boris)]$ glance image-list
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| ID                                   | Name                | Disk Format | Container Format | Size      | Status |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| 592faef8-308a-4438-867a-17adf685cde4 | CirrOS 31           | qcow2       | bare             | 13147648 | active |
| d0e90250-5814-4685-9b8d-65ec9daa7117 | Fedora 20 x86_64    | qcow2       | bare             | 214106112 | active |
| 3e6eea8e-32e6-4373-9eb1-e04b8a3167f9 | Ubuntu Server 13.10 | qcow2       | bare             | 244777472 | active |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+

   I have to notice that schema with `cinder create --image-id .. --display_name VOL_NAME SIZE` & `nova boot --flavor 2 --user-data=./myfile.txt --block_device_mapping vda=volume_id:::0 VM_NAME` doesn't work stable for me in meantime.

As of 03/11/2014 standard schema via `cinder create --image-id IMAGE_ID --display_name VOL_NAME SIZE `& ` nova boot --flavor 2 --user-data=./myfile.txt --block_device_mapping vda=VOLUME_ID:::0 INSTANCE_NAME` started to work fine. However, schema described bellow on the contrary stopped to work on glusterfs based cinder's volumes.Regarding last issue view recent post:-

Launching Instance via image and creating simultaneously bootable cinder volume (glusterfs) on Two Node GRE+OVS+Gluster F20 Cluster

http://bderzhavets.blogspot.com/2014/03/launching-instance-via-image-and.html

Create myfile.txt in cuurent directory like this :-

[root@dallas1 ~(keystone_boris)]$ cat ./myfile.txt
#cloud-config
password: mysecret
chpasswd: { expire: False }
ssh_pwauth: True

[root@dallas1 ~(keystone_boris)]$ nova boot --flavor 2 --user-data=./myfile.txt --block-device source=image,id=d0e90250-5814-4685-9b8d-65ec9daa7117,dest=volume,size=5,shutdown=preserve,bootindex=0 VF20RS012
+--------------------------------------+-------------------------------------------------+
| Property                             | Value                                           |
+--------------------------------------+-------------------------------------------------+
| status                               | BUILD                                           |
| updated                              | 2014-03-09T12:41:22Z                            |
| OS-EXT-STS:task_state                | scheduling                                      |
| key_name                             | None                                            |
| image                                | Attempt to boot from volume - no image supplied |
| hostId                               |                                                 |
| OS-EXT-STS:vm_state                  | building                                        |
| OS-SRV-USG:launched_at               | None                                            |
| flavor                               | m1.small                                        |
| id                                   | 8142ee4c-ef56-4b61-8a0b-ecd82d21484f            |
| security_groups                      | [{u'name': u'default'}]                         |
| OS-SRV-USG:terminated_at             | None                                            |
| user_id                              | df4a984ce2f24848a6b84aaa99e296f1                |
| name                                 | VF20RS012                                       |
| adminPass                            | eFDhC8ZSCFU2                                    |
| tenant_id                            | e896be65e94a4893b870bc29ba86d7eb                |
| created                              | 2014-03-09T12:41:22Z                            |
| OS-DCF:diskConfig                    | MANUAL                                          |
| metadata                             | {}                                              |
| os-extended-volumes:volumes_attached | []                                              |
| accessIPv4                           |                                                 |
| accessIPv6                           |                                                 |
| progress                             | 0                                               |
| OS-EXT-STS:power_state               | 0                                               |
| OS-EXT-AZ:availability_zone          | nova                                            |
| config_drive                         |                                                 |
+--------------------------------------+-------------------------------------------------+

[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+-----------+-----------+----------------------+-------------+-----------------------------+
| ID                                   | Name      | Status    | Task State           | Power State | Networks                    |
+--------------------------------------+-----------+-----------+----------------------+-------------+-----------------------------+
| 9566adec-9406-4c3e-bce5-109ecb8bcf6b | VF20RS007 | SUSPENDED | None                 | Shutdown    | int=10.0.0.2, 192.168.1.101 |
| 8142ee4c-ef56-4b61-8a0b-ecd82d21484f | VF20RS012 | BUILD     | block_device_mapping | NOSTATE     |                             |
+--------------------------------------+-----------+-----------+----------------------+-------------+-----------------------------+

WAIT ...

[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+-----------+-----------+------------+-------------+-----------------------------+
| ID                                   | Name      | Status    | Task State | Power State | Networks                    |
+--------------------------------------+-----------+-----------+------------+-------------+-----------------------------+
| 9566adec-9406-4c3e-bce5-109ecb8bcf6b | VF20RS007 | SUSPENDED | None       | Shutdown    | int=10.0.0.2, 192.168.1.101 |
| 8142ee4c-ef56-4b61-8a0b-ecd82d21484f | VF20RS012 | ACTIVE    | None       | Running     | int=10.0.0.4                |
+--------------------------------------+-----------+-----------+------------+-------------+-----------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron floatingip-create ext
Created a new floatingip:
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| fixed_ip_address    |                                      |
| floating_ip_address | 192.168.1.102                        |
| floating_network_id | 0ed406bf-3552-4036-9006-440f3e69618e |
| id                  | 5c74667d-9b22-4092-ae0a-70ff3a06e785 |
| port_id             |                                      |
| router_id           |                                      |
| tenant_id           | e896be65e94a4893b870bc29ba86d7eb     |
+---------------------+--------------------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron port-list --device-id 8142ee4c-ef56-4b61-8a0b-ecd82d21484f
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                       |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| dc60b5f4-739e-49bd-a004-3ef806e2b488 |      | fa:16:3e:70:56:cc | {"subnet_id": "2e838119-3e2e-46e8-b7cc-6d00975046f2", "ip_address": "10.0.0.4"} |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron floatingip-associate 5c74667d-9b22-4092-ae0a-70ff3a06e785 dc60b5f4-739e-49bd-a004-3ef806e2b488

Associated floatingip 5c74667d-9b22-4092-ae0a-70ff3a06e785

[root@dallas1 ~(keystone_boris)]$ ping 192.168.1.102
PING 192.168.1.102 (192.168.1.102) 56(84) bytes of data.
64 bytes from 192.168.1.102: icmp_seq=1 ttl=63 time=6.23 ms
64 bytes from 192.168.1.102: icmp_seq=2 ttl=63 time=0.702 ms
64 bytes from 192.168.1.102: icmp_seq=3 ttl=63 time=1.07 ms
64 bytes from 192.168.1.102: icmp_seq=4 ttl=63 time=0.693 ms
64 bytes from 192.168.1.102: icmp_seq=5 ttl=63 time=1.80 ms
64 bytes from 192.168.1.102: icmp_seq=6 ttl=63 time=0.750 ms
^C

[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+-----------+--------+------------+-------------+-----------------------------+
| ID                                   | Name      | Status | Task State | Power State | Networks                    |
+--------------------------------------+-----------+--------+------------+-------------+-----------------------------+
| 9566adec-9406-4c3e-bce5-109ecb8bcf6b | VF20RS007 | ACTIVE | None       | Running     | int=10.0.0.2, 192.168.1.101 |
| 8142ee4c-ef56-4b61-8a0b-ecd82d21484f | VF20RS012 | ACTIVE | None       | Running     | int=10.0.0.4, 192.168.1.102 |
+--------------------------------------+-----------+--------+------------+-------------+-----------------------------+

[root@dallas1 ~(keystone_boris)]$ cinder list
+--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+
|                  ID                  | Status | Display Name | Size | Volume Type | Bootable |             Attached to              |
+--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+
| 575be853-b104-458e-bc72-1785ef524416 | in-use |              | 5   |     None    |   true   | 8142ee4c-ef56-4b61-8a0b-ecd82d21484f |
| 9794bd45-8923-4f3e-a48f-fa1d62a964f8 | in-use |              | 5   |     None    |   true   | 9566adec-9406-4c3e-bce5-109ecb8bcf6b |
+--------------------------------------+--------+--------------+------+-------------+----------+------------------------------

Script to assign floating IP to new VM ($1 is the name of VM created) :-

#!/bin/bash -x
neutron floatingip-create ext > newip.log
NewIpId=`cat newip.log | grep ^"| id" | awk '{print $4}' ` ;
NewIp=`cat newip.log | grep "192.168.1"| awk '{print $4}' ` ;
NewVmId=`nova list | grep $1 | awk '{print $2}' ` ;
NewDevId=`neutron port-list --device-id $NewVmId | grep "subnet_id" | awk '{print $2}' ` ;
neutron floatingip-associate $NewIpId $NewDevId ;
echo "New floating Ip assigned : "$NewIp
sleep 10
ping $NewIp

[root@dallas1 ~(keystone_boris)]$ ./AssignIP.sh VF20RS012
+ neutron floatingip-create ext
++ grep '^| id'
++ cat newip.log
++ awk '{print $4}'
+ NewIpId=23b63cd5-3264-4861-abeb-ed27c053b93e
++ cat newip.log
++ awk '{print $4}'
++ grep 192.168.1
+ NewIp=192.168.1.110
++ nova list
++ awk '{print $2}'
++ grep VF20RS012
+ NewVmId=9a902538-6c7b-43eb-a0c5-cea801561c6a
++ neutron port-list --device-id 9a902538-6c7b-43eb-a0c5-cea801561c6a
++ grep subnet_id
++ awk '{print $2}'
+ NewDevId=78d0c401-731a-4ee0-80ae-a6afb4151241
+ neutron floatingip-associate 23b63cd5-3264-4861-abeb-ed27c053b93e 78d0c401-731a-4ee0-80ae-a6afb4151241
Associated floatingip 23b63cd5-3264-4861-abeb-ed27c053b93e
+ echo 'New floating Ip assigned : 192.168.1.110'
New floating Ip assigned : 192.168.1.110

. . . . .

64 bytes from 192.168.1.110: icmp_seq=21 ttl=63 time=8.65 ms
64 bytes from 192.168.1.110: icmp_seq=22 ttl=63 time=6.65 ms
64 bytes from 192.168.1.110: icmp_seq=23 ttl=63 time=1.19 ms
64 bytes from 192.168.1.110: icmp_seq=24 ttl=63 time=1.31 ms
64 bytes from 192.168.1.110: icmp_seq=25 ttl=63 time=1.42 ms

*******************
On Compute:-
*******************

[root@dallas1 ~]# ssh 192.168.1.140
Last login: Sun Mar 9 16:46:40 2014

[root@dallas2 ~]# df -h
Filesystem                       Size Used Avail Use% Mounted on
/dev/mapper/fedora01-root        187G   18G 160G 11% /
devtmpfs                         3.9G     0 3.9G   0% /dev
tmpfs                            3.9G 3.1M 3.9G   1% /dev/shm
tmpfs                            3.9G 9.4M 3.9G   1% /run
tmpfs                            3.9G     0 3.9G   0% /sys/fs/cgroup
tmpfs                            3.9G 115M 3.8G   3% /tmp
/dev/sdb5                        477M 122M 327M 28% /boot
192.168.1.130:/cinder-volumes012 187G   32G 146G 18% /var/lib/nova/mnt/1c9688348ab38662e3ac8fb121077d34

[root@dallas2 ~]# ps -ef| grep nova
nova      1548     1 0 16:29 ?        00:00:42 /usr/bin/python /usr/bin/nova-compute --logfile /var/log/nova/compute.log

root      3005     1 0 16:34 ?        00:00:38 /usr/sbin/glusterfs --volfile-id=cinder-volumes012 --volfile-server=192.168.1.130 /var/lib/nova/mnt/1c9688348ab38662e3ac8fb121077d34

qemu      4762     1 58 16:42 ?        00:52:17 /usr/bin/qemu-system-x86_64 -name instance-00000061 -S -machine pc-i440fx-1.6,accel=tcg,usb=off -cpu Penryn,+osxsave,+xsave,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 8142ee4c-ef56-4b61-8a0b-ecd82d21484f -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2013.2.2-1.fc20,serial=6050001e-8c00-00ac-818a-90e6ba2d11eb,uuid=8142ee4c-ef56-4b61-8a0b-ecd82d21484f -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000061.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/mnt/1c9688348ab38662e3ac8fb121077d34/volume-575be853-b104-458e-bc72-1785ef524416,if=none,id=drive-virtio-disk0,format=raw,serial=575be853-b104-458e-bc72-1785ef524416,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=24,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:70:56:cc,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/8142ee4c-ef56-4b61-8a0b-ecd82d21484f/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

qemu      6330     1 44 16:49 ?        00:36:02 /usr/bin/qemu-system-x86_64 -name instance-0000005f -S -machine pc-i440fx-1.6,accel=tcg,usb=off -cpu Penryn,+osxsave,+xsave,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 9566adec-9406-4c3e-bce5-109ecb8bcf6b -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2013.2.2-1.fc20,serial=6050001e-8c00-00ac-818a-90e6ba2d11eb,uuid=9566adec-9406-4c3e-bce5-109ecb8bcf6b -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-0000005f.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/mnt/1c9688348ab38662e3ac8fb121077d34/volume-9794bd45-8923-4f3e-a48f-fa1d62a964f8,if=none,id=drive-virtio-disk0,format=raw,serial=9794bd45-8923-4f3e-a48f-fa1d62a964f8,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=27,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:50:84:72,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/9566adec-9406-4c3e-bce5-109ecb8bcf6b/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:1 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -incoming fd:24 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
root     24713 24622 0 18:11 pts/4    00:00:00 grep --color=auto nova

[root@dallas2 ~]# ps -ef| grep neutron
neutron   1549     1 0 16:29 ?        00:00:53 /usr/bin/python /usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini --log-file /var/log/neutron/openvswitch-agent.log
root     24981 24622 0 18:12 pts/4    00:00:00 grep --color=auto neutron

Top at Compute node (192.168.1.140)

Runtime at Compute node ( dallas2 192.168.1.140)

******************************************************
Building Ubuntu 14.04 instance via cinder volume
******************************************************

[root@dallas1 ~(keystone_boris)]$ glance image-list
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| ID                                   | Name                | Disk Format | Container Format | Size      | Status |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| 592faef8-308a-4438-867a-17adf685cde4 | CirrOS 31           | qcow2       | bare             | 13147648 | active |
| d0e90250-5814-4685-9b8d-65ec9daa7117 | Fedora 20 x86_64    | qcow2       | bare             | 214106112 | active |
| c2b7c3ed-e25d-44c4-a5e7-4e013c4a8b00 | Ubuntu 14.04        | qcow2       | bare             | 264176128 | active |
| 3e6eea8e-32e6-4373-9eb1-e04b8a3167f9 | Ubuntu Server 13.10 | qcow2       | bare             | 244777472 | active |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+

[root@dallas1 ~(keystone_boris)]$ cinder create --image-id c2b7c3ed-e25d-44c4-a5e7-4e013c4a8b00 --display_name UbuntuTrusty 5
+---------------------+--------------------------------------+
|       Property      |                Value                 |
+---------------------+--------------------------------------+
|     attachments     |                  []                  |
| availability_zone |                 nova                 |
|       bootable      |                false                 |
|      created_at     |      2014-03-10T06:35:39.873978      |
| display_description |                 None                 |
|     display_name    |             UbuntuTrusty             |
|          id         | 8bcc02a7-b9ba-4cd6-a6b9-0574889bf8d2 |
|       image_id      | c2b7c3ed-e25d-44c4-a5e7-4e013c4a8b00 |
|       metadata      |                  {}                  |
|         size        |                  5                   |
|     snapshot_id     |                 None                 |
|     source_volid    |                 None                 |
|        status       |               creating               |
|     volume_type     |                 None                 |
+---------------------+--------------------------------------+

[root@dallas1 ~(keystone_boris)]$ cinder list
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+
|                  ID                  |   Status | Display Name | Size | Volume Type | Bootable |             Attached to              |
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+
| 56ceaaa8-c0ec-45f3-98a4-555c1231b34e |   in-use |              | 5   |     None    |   true   | e29606c5-582f-4766-ae1b-52043a698743 |
| 575be853-b104-458e-bc72-1785ef524416 |   in-use |              | 5   |     None    |   true   | 8142ee4c-ef56-4b61-8a0b-ecd82d21484f |
| 8bcc02a7-b9ba-4cd6-a6b9-0574889bf8d2 | available | UbuntuTrusty | 5   |     None    |   true   |                                      |
| 9794bd45-8923-4f3e-a48f-fa1d62a964f8 |   in-use |              | 5   |     None    |   true   | 9566adec-9406-4c3e-bce5-109ecb8bcf6b |
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+

[root@dallas1 ~(keystone_boris)]$ nova boot --flavor 2 --user-data=./myfile.txt --block_device_mapping vda=8bcc02a7-b9ba-4cd6-a6b9-0574889bf8d2:::0 UbuntuTR01
+--------------------------------------+----------------------------------------------------+
| Property                             | Value                                              |
+--------------------------------------+----------------------------------------------------+
| status                               | BUILD                                              |
| updated                              | 2014-03-10T06:40:14Z                               |
| OS-EXT-STS:task_state                | scheduling                                         |
| key_name                             | None                                               |
| image                                | Attempt to boot from volume - no image supplied    |
| hostId                               |                                                    |
| OS-EXT-STS:vm_state                  | building                                           |
| OS-SRV-USG:launched_at               | None                                               |
| flavor                               | m1.small                                           |
| id                                   | 0859e52d-c07b-4f56-ac79-2b37080d2843               |
| security_groups                      | [{u'name': u'default'}]                            |
| OS-SRV-USG:terminated_at             | None                                               |
| user_id                              | df4a984ce2f24848a6b84aaa99e296f1                   |
| name                                 | UbuntuTR01                                         |
| adminPass                            | L8VuhttJMbJf                                       |
| tenant_id                            | e896be65e94a4893b870bc29ba86d7eb                   |
| created                              | 2014-03-10T06:40:13Z                               |
| OS-DCF:diskConfig                    | MANUAL                                             |
| metadata                             | {}                                                 |
| os-extended-volumes:volumes_attached | [{u'id': u'8bcc02a7-b9ba-4cd6-a6b9-0574889bf8d2'}] |
| accessIPv4                           |                                                    |
| accessIPv6                           |                                                    |
| progress                             | 0                                                  |
| OS-EXT-STS:power_state               | 0                                                  |
| OS-EXT-AZ:availability_zone          | nova                                               |
| config_drive                         |                                                    |
+--------------------------------------+----------------------------------------------------+

[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+------------+-----------+------------+-------------+-----------------------------+
| ID                                   | Name       | Status    | Task State | Power State | Networks                    |
+--------------------------------------+------------+-----------+------------+-------------+-----------------------------+
| 0859e52d-c07b-4f56-ac79-2b37080d2843 | UbuntuTR01 | ACTIVE    | None       | Running     | int=10.0.0.6                |
| 9566adec-9406-4c3e-bce5-109ecb8bcf6b | VF20RS007 | SUSPENDED | None       | Shutdown    | int=10.0.0.2, 192.168.1.101 |
| 8142ee4c-ef56-4b61-8a0b-ecd82d21484f | VF20RS012 | SUSPENDED | None       | Shutdown    | int=10.0.0.4, 192.168.1.102 |
| e29606c5-582f-4766-ae1b-52043a698743 | VF20RS016 | ACTIVE    | None       | Running     | int=10.0.0.5, 192.168.1.103 |
+--------------------------------------+------------+-----------+------------+-------------+-----------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron floatingip-create ext
Created a new floatingip:
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| fixed_ip_address    |                                      |
| floating_ip_address | 192.168.1.104                        |
| floating_network_id | 0ed406bf-3552-4036-9006-440f3e69618e |
| id                  | 9498ac85-82b0-468a-b526-64a659080ab9 |
| port_id             |                                      |
| router_id           |                                      |
| tenant_id           | e896be65e94a4893b870bc29ba86d7eb     |
+---------------------+--------------------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron port-list --device-id 0859e52d-c07b-4f56-ac79-2b37080d2843
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                       |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| 1f02fe57-d844-4fd8-a325-646f27163c8b |      | fa:16:3e:3f:a3:d4 | {"subnet_id": "2e838119-3e2e-46e8-b7cc-6d00975046f2", "ip_address": "10.0.0.6"} |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron floatingip-associate 9498ac85-82b0-468a-b526-64a659080ab9 1f02fe57-d844-4fd8-a325-646f27163c8b

Associated floatingip 9498ac85-82b0-468a-b526-64a659080ab9
[root@dallas1 ~(keystone_boris)]$ ping 192.168.1.104
PING 192.168.1.104 (192.168.1.104) 56(84) bytes of data.
64 bytes from 192.168.1.104: icmp_seq=1 ttl=63 time=2.35 ms
64 bytes from 192.168.1.104: icmp_seq=2 ttl=63 time=2.56 ms
64 bytes from 192.168.1.104: icmp_seq=3 ttl=63 time=1.17 ms
64 bytes from 192.168.1.104: icmp_seq=4 ttl=63 time=4.08 ms
64 bytes from 192.168.1.104: icmp_seq=5 ttl=63 time=2.19 ms
^C

[boris@dallas1 ~]$ ssh root@192.168.1.140
root@192.168.1.140's password:
Last login: Mon Mar 10 11:32:02 2014 from 192.168.1.104

[root@dallas2 ~]# ps -ef | grep nova
nova      1564     1 0 08:42 ?        00:01:34 /usr/bin/python /usr/bin/nova-compute --logfile /var/log/nova/compute.log

root      2875     1 0 08:47 ?        00:00:40 /usr/sbin/glusterfs --volfile-id=cinder-volumes012 --volfile-server=192.168.1.130 /var/lib/nova/mnt/1c9688348ab38662e3ac8fb121077d34
root     18564 18530 0 12:29 pts/4    00:00:00 grep --color=auto nova

qemu     22753     1 37 10:09 ?        00:52:22 /usr/bin/qemu-system-x86_64 -name instance-00000063 -S -machine pc-i440fx-1.6,accel=tcg,usb=off -cpu Penryn,+osxsave,+xsave,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid e29606c5-582f-4766-ae1b-52043a698743 -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2013.2.2-1.fc20,serial=6050001e-8c00-00ac-818a-90e6ba2d11eb,uuid=e29606c5-582f-4766-ae1b-52043a698743 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000063.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/mnt/1c9688348ab38662e3ac8fb121077d34/volume-56ceaaa8-c0ec-45f3-98a4-555c1231b34e,if=none,id=drive-virtio-disk0,format=raw,serial=56ceaaa8-c0ec-45f3-98a4-555c1231b34e,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=25,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:f7:58:8c,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/e29606c5-582f-4766-ae1b-52043a698743/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

qemu     29140     1 25 10:40 ?        00:28:16 /usr/bin/qemu-system-x86_64 -name instance-00000066 -S -machine pc-i440fx-1.6,accel=tcg,usb=off -cpu Penryn,+osxsave,+xsave,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 0859e52d-c07b-4f56-ac79-2b37080d2843 -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2013.2.2-1.fc20,serial=6050001e-8c00-00ac-818a-90e6ba2d11eb,uuid=0859e52d-c07b-4f56-ac79-2b37080d2843 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000066.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/mnt/1c9688348ab38662e3ac8fb121077d34/volume-8bcc02a7-b9ba-4cd6-a6b9-0574889bf8d2,if=none,id=drive-virtio-disk0,format=raw,serial=8bcc02a7-b9ba-4cd6-a6b9-0574889bf8d2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=25,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:3f:a3:d4,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/0859e52d-c07b-4f56-ac79-2b37080d2843/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:1 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
[root@dallas2 ~]# cd /FDR/Replicate

[root@dallas2 Replicate]# ls -la
total 6470672
drwxr-xr-x. 3 165 165       4096 Mar 10 10:35 .
drwxr-xr-x. 3 root root       4096 Mar 6 00:50 ..
drw-------. 14 root root       4096 Mar 10 10:35 .glusterfs
-rw-rw-rw-. 2 qemu qemu 5368709120 Mar 10 12:29 volume-56ceaaa8-c0ec-45f3-98a4-555c1231b34e
-rw-rw-rw-. 2 root root 5368709120 Mar 10 09:59 volume-575be853-b104-458e-bc72-1785ef524416
-rw-rw-rw-. 2 qemu qemu 5368709120 Mar 10 12:30 volume-8bcc02a7-b9ba-4cd6-a6b9-0574889bf8d2
-rw-rw-rw-. 2 root root 5368709120 Mar 10 09:58 volume-9794bd45-8923-4f3e-a48f-fa1d62a964f8

Another Two node Controller&Compute Neutron GRE+OVS+Gluster 3.4.2 system :-

Processes running on Compute node (dfw01 192.168.1.137)

[root@dfw01 ~]# ps -ef | grep nova

root      2316     1 0 13:58 ?        00:00:00 /usr/sbin/glusterfsd -s dfw01.localdomain --volfile-id nova.dfw01.localdomain.data1-nova -p /var/lib/glusterd/vols/nova/run/dfw01.localdomain-data1-nova.pid -S /var/run/05a99291aadb7e65037a2d9c76e6eab8.socket --brick-name /data1/nova -l /var/log/glusterfs/bricks/data1-nova.log --xlator-option *-posix.glusterd-uuid=8b362d06-f504-45fd-9fa6-d2dfe46d9946 --brick-port 49152 --xlator-option nova-server.listen-port=49152

nova      3125     1 0 14:04 ?        00:01:12 /usr/bin/python /usr/bin/nova-compute --logfile /var/log/nova/compute.log

qemu      3687     1 7 14:06 ?        00:11:49 /usr/bin/qemu-system-x86_64 -name instance-00000089 -S -machine pc-i440fx-1.6,accel=tcg,usb=off -cpu Penryn,+osxsave,+xsave,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid b6c18662-3a61-45c2-9e8a-0e66824f1fee -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2013.2.2-1.fc20,serial=6050001e-8c00-00ac-818a-90e6ba2d11eb,uuid=b6c18662-3a61-45c2-9e8a-0e66824f1fee -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000089.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/instances/b6c18662-3a61-45c2-9e8a-0e66824f1fee/disk,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=24,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:d6:e2:97,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/b6c18662-3a61-45c2-9e8a-0e66824f1fee/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
root      4773     1 0 14:10 ?        00:01:22 /usr/sbin/glusterfs --volfile-id=cinder-volumes05 --volfile-server=192.168.1.127 /var/lib/nova/mnt/62f75cf6996a8a6bcc0d343be378c10a

qemu      8128     1 32 14:24 ?        00:46:12 /usr/bin/qemu-system-x86_64 -name instance-0000008b -S -machine pc-i440fx-1.6,accel=tcg,usb=off -cpu Penryn,+osxsave,+xsave,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid ca95fb8a-c434-4a83-9acb-7e32fafe8b31 -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2013.2.2-1.fc20,serial=6050001e-8c00-00ac-818a-90e6ba2d11eb,uuid=ca95fb8a-c434-4a83-9acb-7e32fafe8b31 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-0000008b.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/mnt/62f75cf6996a8a6bcc0d343be378c10a/volume-f64ae982-7d77-4a40-b697-61bffd8f30d1,if=none,id=drive-virtio-disk0,format=raw,serial=f64ae982-7d77-4a40-b697-61bffd8f30d1,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=27,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:a0:80:01,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/ca95fb8a-c434-4a83-9acb-7e32fafe8b31/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:2 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
root     10456 10308 0 16:44 pts/5    00:00:00 grep --color=auto nova

qemu     28838     1 68 15:47 ?        00:38:59 /usr/bin/qemu-system-x86_64 -name instance-0000008c -S -machine pc-i440fx-1.6,accel=tcg,usb=off -cpu Penryn,+osxsave,+xsave,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid ad8031df-e3c8-4ace-8657-c39ba6a8acf8 -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2013.2.2-1.fc20,serial=6050001e-8c00-00ac-818a-90e6ba2d11eb,uuid=ad8031df-e3c8-4ace-8657-c39ba6a8acf8 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-0000008c.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/mnt/62f75cf6996a8a6bcc0d343be378c10a/volume-b648e4dc-4521-48c5-962d-4948b9f12c76,if=none,id=drive-virtio-disk0,format=raw,serial=b648e4dc-4521-48c5-962d-4948b9f12c76,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=26,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:78:18:7f,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/ad8031df-e3c8-4ace-8657-c39ba6a8acf8/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:1 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

[root@dfw01 ~]# df -h
Filesystem                      Size Used Avail Use% Mounted on
/dev/mapper/fedora-root          96G   47G   45G 52% /
devtmpfs                        3.9G     0 3.9G   0% /dev
tmpfs                           3.9G 2.7M 3.9G   1% /dev/shm
tmpfs                           3.9G 9.5M 3.9G   1% /run
tmpfs                           3.9G     0 3.9G   0% /sys/fs/cgroup
tmpfs                           3.9G 117M 3.8G   3% /tmp
/dev/sda5                       477M 122M 326M 28% /boot
/dev/mapper/fedora-data1         77G 3.7G   70G   6% /data1
192.168.1.127:/cinder-volumes05   77G 3.7G   70G   6% /var/lib/nova/mnt/62f75cf6996a8a6bcc0d343be378c10a

[root@dfw01 ~]# cd /var/lib/nova/mnt/62f75cf6996a8a6bcc0d343be378c10a
[root@dfw01 62f75cf6996a8a6bcc0d343be378c10a]# ls -la
total 3152316
drwxr-xr-x. 3 cinder cinder       4096 Mar 11 15:08 .
drwxr-xr-x. 3 nova   nova         4096 Jan 25 07:51 ..
-rw-rw-rw-. 1 qemu   qemu   5368709120 Mar 11 17:06 volume-b648e4dc-4521-48c5-962d-4948b9f12c76
-rw-rw-rw-. 1 qemu   qemu   5368709120 Mar 11 17:06 volume-f64ae982-7d77-4a40-b697-61bffd8f30d1

References

1. http://bderzhavets.blogspot.com/2014/02/setup-light-weight-x-windows_2.html
2. http://bderzhavets.blogspot.com/2014/01/setting-up-two-physical-node-openstack.html

Monday, March 03, 2014

Up to date procedure of creating cinder's ThinLVM based cloud instances of F20,Ubuntu 13.10 on F20 Havana Compute Node.

Schema bellow allows to avoid two commands . First one creating cinder volume `cinder create --image-id Image-id --display_name VOL_NAME 7` && and second one booting instance via volume created - `nova boot --flavor 2 --user-data=./myfile.txt --block_device_mapping vda=cinder_id:::0 VM_NAME` It allows to create volume on F20 Controller node and boot instance on F20 Compute node simultaneously. This post follows up [ 1 ].

First stop and start tgtd service :-

[root@dallas1 ~(keystone_admin)]$ service tgtd stop

Redirecting to /bin/systemctl stop tgtd.service
[root@dallas1 ~(keystone_admin)]$ service tgtd status
Redirecting to /bin/systemctl status tgtd.service
tgtd.service - tgtd iSCSI target daemon
   Loaded: loaded (/usr/lib/systemd/system/tgtd.service; enabled)
   Active: inactive (dead) since Tue 2014-03-04 11:46:18 MSK; 8s ago
Process: 11978 ExecStop=/usr/sbin/tgtadm --op delete --mode system (code=exited, status=0/SUCCESS)
Process: 11974 ExecStop=/usr/sbin/tgt-admin --update ALL -c /dev/null (code=exited, status=0/SUCCESS)
Process: 11972 ExecStop=/usr/sbin/tgtadm --op update --mode sys --name State -v offline (code=exited, status=0/SUCCESS)
Process: 1797 ExecStartPost=/usr/sbin/tgtadm --op update --mode sys --name State -v ready (code=exited, status=0/SUCCESS)
Process: 1791 ExecStartPost=/usr/sbin/tgt-admin -e -c $TGTD_CONFIG (code=exited, status=0/SUCCESS)
Process: 1790 ExecStartPost=/usr/sbin/tgtadm --op update --mode sys --name State -v offline (code=exited, status=0/SUCCESS)
Process: 1173 ExecStartPost=/bin/sleep 5 (code=exited, status=0/SUCCESS)
Process: 1172 ExecStart=/usr/sbin/tgtd -f $TGTD_OPTS (code=exited, status=0/SUCCESS)
Main PID: 1172 (code=exited, status=0/SUCCESS)

Mar 04 11:14:04 dallas1.localdomain tgtd[1172]: tgtd: work_timer_start(146) use timer_fd based scheduler
Mar 04 11:14:04 dallas1.localdomain tgtd[1172]: tgtd: bs_init_signalfd(271) could not open backing-store module direct...store
Mar 04 11:14:04 dallas1.localdomain tgtd[1172]: tgtd: bs_init(390) use signalfd notification
Mar 04 11:14:09 dallas1.localdomain systemd[1]: Started tgtd iSCSI target daemon.
Mar 04 11:26:01 dallas1.localdomain tgtd[1172]: tgtd: device_mgmt(246) sz:69 params:path=/dev/cinder-volumes/volume-a0...2864d
Mar 04 11:26:01 dallas1.localdomain tgtd[1172]: tgtd: bs_thread_open(412) 16
Mar 04 11:33:32 dallas1.localdomain tgtd[1172]: tgtd: device_mgmt(246) sz:69 params:path=/dev/cinder-volumes/volume-01...f2969
Mar 04 11:33:32 dallas1.localdomain tgtd[1172]: tgtd: bs_thread_open(412) 16
Mar 04 11:46:18 dallas1.localdomain systemd[1]: Stopping tgtd iSCSI target daemon...
Mar 04 11:46:18 dallas1.localdomain systemd[1]: Stopped tgtd iSCSI target daemon.
Hint: Some lines were ellipsized, use -l to show in full.

[root@dallas1 ~(keystone_admin)]$ service tgtd start

Redirecting to /bin/systemctl start tgtd.service
[root@dallas1 ~(keystone_admin)]$ service tgtd status
Redirecting to /bin/systemctl status tgtd.service
tgtd.service - tgtd iSCSI target daemon
   Loaded: loaded (/usr/lib/systemd/system/tgtd.service; enabled)
   Active: active (running) since Tue 2014-03-04 19:31:15 MSK; 3s ago
Process: 7254 ExecStop=/usr/sbin/tgtadm --op delete --mode system (code=exited, status=22)
Process: 7251 ExecStop=/usr/sbin/tgt-admin --update ALL -c /dev/null (code=exited, status=0/SUCCESS)
Process: 7249 ExecStop=/usr/sbin/tgtadm --op update --mode sys --name State -v offline (code=exited, status=0/SUCCESS)
Process: 7656 ExecStartPost=/usr/sbin/tgtadm --op update --mode sys --name State -v ready (code=exited, status=0/SUCCESS)
Process: 7606 ExecStartPost=/usr/sbin/tgt-admin -e -c $TGTD_CONFIG (code=exited, status=0/SUCCESS)
Process: 7604 ExecStartPost=/usr/sbin/tgtadm --op update --mode sys --name State -v offline (code=exited, status=0/SUCCESS)
Process: 7583 ExecStartPost=/bin/sleep 5 (code=exited, status=0/SUCCESS)
Main PID: 7582 (tgtd)
   CGroup: /system.slice/tgtd.service
           └─7582 /usr/sbin/tgtd -f

Mar 04 19:31:09 dallas1.localdomain tgtd[7582]: tgtd: work_timer_start(146) use timer_fd based scheduler
Mar 04 19:31:09 dallas1.localdomain tgtd[7582]: tgtd: bs_init_signalfd(271) could not open backing-store module direct...store
Mar 04 19:31:09 dallas1.localdomain tgtd[7582]: tgtd: bs_init(390) use signalfd notification
Mar 04 19:31:14 dallas1.localdomain tgt-admin[7606]: File /etc/cinder/volumes/volume-0d7756dd-4d0d-4ea7-ae9f-dbccf5773...gain.
Mar 04 19:31:14 dallas1.localdomain tgt-admin[7606]: File /etc/cinder/volumes/volume-d69ca335-5448-4e48-8f14-79a2d8c82...gain.
Mar 04 19:31:14 dallas1.localdomain tgtd[7582]: tgtd: device_mgmt(246) sz:69 params:path=/dev/cinder-volumes/volume-0d...73f0a
Mar 04 19:31:14 dallas1.localdomain tgtd[7582]: tgtd: bs_thread_open(412) 16
Mar 04 19:31:15 dallas1.localdomain tgtd[7582]: tgtd: device_mgmt(246) sz:69 params:path=/dev/cinder-volumes/volume-d6...82728
Mar 04 19:31:15 dallas1.localdomain tgtd[7582]: tgtd: bs_thread_open(412) 16
Mar 04 19:31:15 dallas1.localdomain systemd[1]: Started tgtd iSCSI target daemon.
Hint: Some lines were ellipsized, use -l to show in full.

Restart cinder services :-

[root@dallas1 ~(keystone_admin)]$ for i in api scheduler volume ; do service openstack-cinder-${i} restart ;done

Redirecting to /bin/systemctl restart openstack-cinder-api.service
Redirecting to /bin/systemctl restart openstack-cinder-scheduler.service
Redirecting to /bin/systemctl restart openstack-cinder-volume.service

Now there is a chance to create instance via thinLVM volume

Create thin LVM via Nova with login option "fedora"&"mysecret" in one command

[root@dallas1 ~(keystone_boris)]$ glance image-list
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| ID                                   | Name                | Disk Format | Container Format | Size      | Status |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| 592faef8-308a-4438-867a-17adf685cde4 | CirrOS 31           | qcow2       | bare             | 13147648 | active |
| d0e90250-5814-4685-9b8d-65ec9daa7117 | Fedora 20 x86_64    | qcow2       | bare             | 214106112 | active |
| 3e6eea8e-32e6-4373-9eb1-e04b8a3167f9 | Ubuntu Server 13.10 | qcow2       | bare             | 244777472 | active |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+

[root@dallas1 ~(keystone_boris)]$ cat ./myfile.txt
#cloud-config
password: mysecret
chpasswd: { expire: False }
ssh_pwauth: True

[root@dallas1 ~(keystone_boris)]$ nova boot --flavor 2 --user-data=./myfile.txt --block-device source=image,id=d0e90250-5814-4685-9b8d-65ec9daa7117,dest=volume,size=5,shutdown=preserve,bootindex=0 VF20RS01

+--------------------------------------+-------------------------------------------------+
| Property                             | Value                                           |
+--------------------------------------+-------------------------------------------------+
| status                               | BUILD                                           |
| updated                              | 2014-03-07T05:50:18Z                            |
| OS-EXT-STS:task_state                | scheduling                                      |
| key_name                             | None                                            |
| image                                | Attempt to boot from volume - no image supplied |
| hostId                               |                                                 |
| OS-EXT-STS:vm_state                  | building                                        |
| OS-SRV-USG:launched_at               | None                                            |
| flavor                               | m1.small                                        |
| id                                   | 770e33f7-7aab-49f1-95ca-3cf343f744ef            |
| security_groups                      | [{u'name': u'default'}]                         |
| OS-SRV-USG:terminated_at             | None                                            |
| user_id                              | df4a984ce2f24848a6b84aaa99e296f1                |
| name                                 | VF20RS01                                        |
| adminPass                            | CqjGVUm9bbs9                                    |
| tenant_id                            | e896be65e94a4893b870bc29ba86d7eb                |
| created                              | 2014-03-07T05:50:18Z                            |
| OS-DCF:diskConfig                    | MANUAL                                          |
| metadata                             | {}                                              |
| os-extended-volumes:volumes_attached | []                                              |
| accessIPv4                           |                                                 |
| accessIPv6                           |                                                 |
| progress                             | 0                                               |
| OS-EXT-STS:power_state               | 0                                               |
| OS-EXT-AZ:availability_zone          | nova                                            |
| config_drive                         |                                                 |
+--------------------------------------+-------------------------------------------------+

[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+----------+--------+----------------------+-------------+----------+
| ID                                   | Name     | Status | Task State           | Power State | Networks |
+--------------------------------------+----------+--------+----------------------+-------------+----------+
| 770e33f7-7aab-49f1-95ca-3cf343f744ef | VF20RS01 | BUILD | block_device_mapping | NOSTATE     |          |
+--------------------------------------+----------+--------+----------------------+-------------+----------+

[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+----------+--------+----------------------+-------------+----------+
| ID                                   | Name     | Status | Task State           | Power State | Networks |
+--------------------------------------+----------+--------+----------------------+-------------+----------+
| 770e33f7-7aab-49f1-95ca-3cf343f744ef | VF20RS01 | BUILD | block_device_mapping | NOSTATE     |          |
+--------------------------------------+----------+--------+----------------------+-------------+----------+

[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+----------+--------+------------+-------------+--------------+
| ID                                   | Name     | Status | Task State | Power State | Networks     |
+--------------------------------------+----------+--------+------------+-------------+--------------+
| 770e33f7-7aab-49f1-95ca-3cf343f744ef | VF20RS01 | ACTIVE | None       | Running     | int=10.0.0.2 |
+--------------------------------------+----------+--------+------------+-------------+--------------+

[root@dallas1 ~(keystone_boris)]$ neutron floatingip-create ext
Created a new floatingip:
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| fixed_ip_address    |                                      |
| floating_ip_address | 192.168.1.101                        |
| floating_network_id | 0ed406bf-3552-4036-9006-440f3e69618e |
| id                  | f7d9cd3f-e544-4f23-821d-0307ed4eb852 |
| port_id             |                                      |
| router_id           |                                      |
| tenant_id           | e896be65e94a4893b870bc29ba86d7eb     |
+---------------------+--------------------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron port-list --device-id 770e33f7-7aab-49f1-95ca-3cf343f744ef
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                       |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| 8b5f142e-ce99-40e0-bbbe-620b201c0323 |      | fa:16:3e:0d:c4:e6 | {"subnet_id": "2e838119-3e2e-46e8-b7cc-6d00975046f2", "ip_address": "10.0.0.2"} |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron floatingip-associate f7d9cd3f-e544-4f23-821d-0307ed4eb852 8b5f142e-ce99-40e0-bbbe-620b201c0323
Associated floatingip f7d9cd3f-e544-4f23-821d-0307ed4eb852

[root@dallas1 ~(keystone_boris)]$ ping 192.168.1.101
PING 192.168.1.101 (192.168.1.101) 56(84) bytes of data.
64 bytes from 192.168.1.101: icmp_seq=1 ttl=63 time=7.75 ms
64 bytes from 192.168.1.101: icmp_seq=2 ttl=63 time=1.06 ms
64 bytes from 192.168.1.101: icmp_seq=3 ttl=63 time=1.27 ms
64 bytes from 192.168.1.101: icmp_seq=4 ttl=63 time=1.43 ms
64 bytes from 192.168.1.101: icmp_seq=5 ttl=63 time=1.80 ms
64 bytes from 192.168.1.101: icmp_seq=6 ttl=63 time=0.916 ms
64 bytes from 192.168.1.101: icmp_seq=7 ttl=63 time=0.919 ms
64 bytes from 192.168.1.101: icmp_seq=8 ttl=63 time=0.930 ms
64 bytes from 192.168.1.101: icmp_seq=9 ttl=63 time=0.977 ms
64 bytes from 192.168.1.101: icmp_seq=10 ttl=63 time=0.690 ms
^C
--- 192.168.1.101 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9008ms
rtt min/avg/max/mdev = 0.690/1.776/7.753/2.015 ms

[root@dallas1 ~(keystone_boris)]$ glance image-list
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| ID                                   | Name                | Disk Format | Container Format | Size      | Status |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| 592faef8-308a-4438-867a-17adf685cde4 | CirrOS 31           | qcow2       | bare             | 13147648 | active |
| d0e90250-5814-4685-9b8d-65ec9daa7117 | Fedora 20 x86_64    | qcow2       | bare             | 214106112 | active |
| 3e6eea8e-32e6-4373-9eb1-e04b8a3167f9 | Ubuntu Server 13.10 | qcow2       | bare             | 244777472 | active |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+

[root@dallas1 ~(keystone_boris)]$ nova boot --flavor 2 --user-data=./myfile.txt --block-device source=image,id=3e6eea8e-32e6-4373-9eb1-e04b8a3167f9,dest=volume,size=5,shutdown=preserve,bootindex=0 UbuntuRS01
+--------------------------------------+-------------------------------------------------+
| Property                             | Value                                           |
+--------------------------------------+-------------------------------------------------+
| status                               | BUILD                                           |
| updated                              | 2014-03-07T05:53:44Z                            |
| OS-EXT-STS:task_state                | scheduling                                      |
| key_name                             | None                                            |
| image                                | Attempt to boot from volume - no image supplied |
| hostId                               |                                                 |
| OS-EXT-STS:vm_state                  | building                                        |
| OS-SRV-USG:launched_at               | None                                            |
| flavor                               | m1.small                                        |
| id                                   | bfcb2120-942f-4d3f-a173-93f6076a4be8            |
| security_groups                      | [{u'name': u'default'}]                         |
| OS-SRV-USG:terminated_at             | None                                            |
| user_id                              | df4a984ce2f24848a6b84aaa99e296f1                |
| name                                 | UbuntuRS01                                      |
| adminPass                            | bXND2XTsvuA4                                    |
| tenant_id                            | e896be65e94a4893b870bc29ba86d7eb                |
| created                              | 2014-03-07T05:53:44Z                            |
| OS-DCF:diskConfig                    | MANUAL                                          |
| metadata                             | {}                                              |
| os-extended-volumes:volumes_attached | []                                              |
| accessIPv4                           |                                                 |
| accessIPv6                           |                                                 |
| progress                             | 0                                               |
| OS-EXT-STS:power_state               | 0                                               |
| OS-EXT-AZ:availability_zone          | nova                                            |
| config_drive                         |                                                 |
+--------------------------------------+-------------------------------------------------+
[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
| ID                                   | Name       | Status | Task State           | Power State | Networks                    |
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
| bfcb2120-942f-4d3f-a173-93f6076a4be8 | UbuntuRS01 | BUILD | block_device_mapping | NOSTATE     |                             |
| 770e33f7-7aab-49f1-95ca-3cf343f744ef | VF20RS01   | ACTIVE | None                 | Running     | int=10.0.0.2, 192.168.1.101 |
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+------------+--------+------------+-------------+-----------------------------+
| ID                                   | Name       | Status | Task State | Power State | Networks                    |
+--------------------------------------+------------+--------+------------+-------------+-----------------------------+
| bfcb2120-942f-4d3f-a173-93f6076a4be8 | UbuntuRS01 | ACTIVE | None       | Running     | int=10.0.0.4                |
| 770e33f7-7aab-49f1-95ca-3cf343f744ef | VF20RS01   | ACTIVE | None       | Running     | int=10.0.0.2, 192.168.1.101 |
+--------------------------------------+------------+--------+------------+-------------+-----------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron floatingip-create ext
Created a new floatingip:
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| fixed_ip_address    |                                      |
| floating_ip_address | 192.168.1.102                        |
| floating_network_id | 0ed406bf-3552-4036-9006-440f3e69618e |
| id                  | b3d3f262-5142-4a99-9b8d-431c231cb1d7 |
| port_id             |                                      |
| router_id           |                                      |
| tenant_id           | e896be65e94a4893b870bc29ba86d7eb     |
+---------------------+--------------------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron port-list --device-id bfcb2120-942f-4d3f-a173-93f6076a4be8
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                       |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| c81ca027-8f9b-49c3-af10-adc60f5d4d12 |      | fa:16:3e:ac:86:50 | {"subnet_id": "2e838119-3e2e-46e8-b7cc-6d00975046f2", "ip_address": "10.0.0.4"} |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron floatingip-associate b3d3f262-5142-4a99-9b8d-431c231cb1d7 c81ca027-8f9b-49c3-af10-adc60f5d4d12
Associated floatingip b3d3f262-5142-4a99-9b8d-431c231cb1d7
[root@dallas1 ~(keystone_boris)]$ ping 192.168.1.102
PING 192.168.1.102 (192.168.1.102) 56(84) bytes of data.
64 bytes from 192.168.1.102: icmp_seq=1 ttl=63 time=3.84 ms
64 bytes from 192.168.1.102: icmp_seq=2 ttl=63 time=3.06 ms
64 bytes from 192.168.1.102: icmp_seq=3 ttl=63 time=6.58 ms
64 bytes from 192.168.1.102: icmp_seq=4 ttl=63 time=7.98 ms
64 bytes from 192.168.1.102: icmp_seq=5 ttl=63 time=2.09 ms
64 bytes from 192.168.1.102: icmp_seq=6 ttl=63 time=1.06 ms
64 bytes from 192.168.1.102: icmp_seq=7 ttl=63 time=3.55 ms
64 bytes from 192.168.1.102: icmp_seq=8 ttl=63 time=2.01 ms
64 bytes from 192.168.1.102: icmp_seq=9 ttl=63 time=1.05 ms
64 bytes from 192.168.1.102: icmp_seq=10 ttl=63 time=3.45 ms
64 bytes from 192.168.1.102: icmp_seq=11 ttl=63 time=2.31 ms
64 bytes from 192.168.1.102: icmp_seq=12 ttl=63 time=0.977 ms
^C
--- 192.168.1.102 ping statistics ---
12 packets transmitted, 12 received, 0% packet loss, time 11014ms
rtt min/avg/max/mdev = 0.977/3.168/7.985/2.091 ms

[root@dallas1 ~(keystone_boris)]$ nova boot --flavor 2 --user-data=./myfile.txt --block-device source=image,id=d0e90250-5814-4685-9b8d-65ec9daa7117,dest=volume,size=5,shutdown=preserve,bootindex=0 VF20GLX
+--------------------------------------+-------------------------------------------------+
| Property                             | Value                                           |
+--------------------------------------+-------------------------------------------------+
| status                               | BUILD                                           |
| updated                              | 2014-03-07T05:58:40Z                            |
| OS-EXT-STS:task_state                | scheduling                                      |
| key_name                             | None                                            |
| image                                | Attempt to boot from volume - no image supplied |
| hostId                               |                                                 |
| OS-EXT-STS:vm_state                  | building                                        |
| OS-SRV-USG:launched_at               | None                                            |
| flavor                               | m1.small                                        |
| id                                   | 62ff1641-2c96-470f-9147-9272d68d2e5c            |
| security_groups                      | [{u'name': u'default'}]                         |
| OS-SRV-USG:terminated_at             | None                                            |
| user_id                              | df4a984ce2f24848a6b84aaa99e296f1                |
| name                                 | VF20GLX                                         |
| adminPass                            | E9KXeLp8fWig                                    |
| tenant_id                            | e896be65e94a4893b870bc29ba86d7eb                |
| created                              | 2014-03-07T05:58:40Z                            |
| OS-DCF:diskConfig                    | MANUAL                                          |
| metadata                             | {}                                              |
| os-extended-volumes:volumes_attached | []                                              |
| accessIPv4                           |                                                 |
| accessIPv6                           |                                                 |
| progress                             | 0                                               |
| OS-EXT-STS:power_state               | 0                                               |
| OS-EXT-AZ:availability_zone          | nova                                            |
| config_drive                         |                                                 |
+--------------------------------------+-------------------------------------------------+
[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
| ID                                   | Name       | Status | Task State           | Power State | Networks                    |
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
| bfcb2120-942f-4d3f-a173-93f6076a4be8 | UbuntuRS01 | ACTIVE | None                 | Running     | int=10.0.0.4, 192.168.1.102 |
| 62ff1641-2c96-470f-9147-9272d68d2e5c | VF20GLX    | BUILD | block_device_mapping | NOSTATE     |                             |
| 770e33f7-7aab-49f1-95ca-3cf343f744ef | VF20RS01   | ACTIVE | None                 | Running     | int=10.0.0.2, 192.168.1.101 |
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
| ID                                   | Name       | Status | Task State           | Power State | Networks                    |
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
| bfcb2120-942f-4d3f-a173-93f6076a4be8 | UbuntuRS01 | ACTIVE | None                 | Running     | int=10.0.0.4, 192.168.1.102 |
| 62ff1641-2c96-470f-9147-9272d68d2e5c | VF20GLX    | BUILD | block_device_mapping | NOSTATE     |                             |
| 770e33f7-7aab-49f1-95ca-3cf343f744ef | VF20RS01   | ACTIVE | None                 | Running     | int=10.0.0.2, 192.168.1.101 |
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
| ID                                   | Name       | Status | Task State           | Power State | Networks                    |
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
| bfcb2120-942f-4d3f-a173-93f6076a4be8 | UbuntuRS01 | ACTIVE | None                 | Running     | int=10.0.0.4, 192.168.1.102 |
| 62ff1641-2c96-470f-9147-9272d68d2e5c | VF20GLX    | BUILD | block_device_mapping | NOSTATE     |                             |
| 770e33f7-7aab-49f1-95ca-3cf343f744ef | VF20RS01   | ACTIVE | None                 | Running     | int=10.0.0.2, 192.168.1.101 |
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
| ID                                   | Name       | Status | Task State           | Power State | Networks                    |
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
| bfcb2120-942f-4d3f-a173-93f6076a4be8 | UbuntuRS01 | ACTIVE | None                 | Running     | int=10.0.0.4, 192.168.1.102 |
| 62ff1641-2c96-470f-9147-9272d68d2e5c | VF20GLX    | BUILD | block_device_mapping | NOSTATE     |                             |
| 770e33f7-7aab-49f1-95ca-3cf343f744ef | VF20RS01   | ACTIVE | None                 | Running     | int=10.0.0.2, 192.168.1.101 |
+--------------------------------------+------------+--------+----------------------+-------------+-----------------------------+
[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+------------+--------+------------+-------------+-----------------------------+
| ID                                   | Name       | Status | Task State | Power State | Networks                    |
+--------------------------------------+------------+--------+------------+-------------+-----------------------------+
| bfcb2120-942f-4d3f-a173-93f6076a4be8 | UbuntuRS01 | ACTIVE | None       | Running     | int=10.0.0.4, 192.168.1.102 |
| 62ff1641-2c96-470f-9147-9272d68d2e5c | VF20GLX    | ACTIVE | None       | Running     | int=10.0.0.5                |
| 770e33f7-7aab-49f1-95ca-3cf343f744ef | VF20RS01   | ACTIVE | None       | Running     | int=10.0.0.2, 192.168.1.101 |
+--------------------------------------+------------+--------+------------+-------------+-----------------------------+
[root@dallas1 ~(keystone_boris)]$ neutron floatingip-create extCreated a new floatingip:
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| fixed_ip_address    |                                      |
| floating_ip_address | 192.168.1.103                        |
| floating_network_id | 0ed406bf-3552-4036-9006-440f3e69618e |
| id                  | 3fb87bb2-f485-4f1c-b2b7-7c5d90588d27 |
| port_id             |                                      |
| router_id           |                                      |
| tenant_id           | e896be65e94a4893b870bc29ba86d7eb     |
+---------------------+--------------------------------------+
[root@dallas1 ~(keystone_boris)]$ neutron port-list --device-id 62ff1641-2c96-470f-9147-9272d68d2e5c
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                       |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| 0845ad30-4d2c-487d-8847-2b6e3e8b9b9d |      | fa:16:3e:2c:84:62 | {"subnet_id": "2e838119-3e2e-46e8-b7cc-6d00975046f2", "ip_address": "10.0.0.5"} |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
[root@dallas1 ~(keystone_boris)]$ neutron floatingip-associate 3fb87bb2-f485-4f1c-b2b7-7c5d90588d27 0845ad30-4d2c-487d-8847-2b6e3e8b9b9d
Associated floatingip 3fb87bb2-f485-4f1c-b2b7-7c5d90588d27
[root@dallas1 ~(keystone_boris)]$ ping 192.168.1.103
PING 192.168.1.103 (192.168.1.103) 56(84) bytes of data.
64 bytes from 192.168.1.103: icmp_seq=1 ttl=63 time=4.08 ms
64 bytes from 192.168.1.103: icmp_seq=2 ttl=63 time=1.59 ms
64 bytes from 192.168.1.103: icmp_seq=3 ttl=63 time=1.22 ms
64 bytes from 192.168.1.103: icmp_seq=4 ttl=63 time=1.49 ms
64 bytes from 192.168.1.103: icmp_seq=5 ttl=63 time=1.11 ms
64 bytes from 192.168.1.103: icmp_seq=6 ttl=63 time=0.980 ms
64 bytes from 192.168.1.103: icmp_seq=7 ttl=63 time=6.71 ms
^C
--- 192.168.1.103 ping statistics ---
7 packets transmitted, 7 received, 0% packet loss, time 6007ms
rtt min/avg/max/mdev = 0.980/2.458/6.711/1.996 ms

[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+------------+--------+------------+-------------+-----------------------------+
| ID                                   | Name       | Status | Task State | Power State | Networks                    |
+--------------------------------------+------------+--------+------------+-------------+-----------------------------+
| bfcb2120-942f-4d3f-a173-93f6076a4be8 | UbuntuRS01 | ACTIVE | None       | Running     | int=10.0.0.4, 192.168.1.102 |
| 62ff1641-2c96-470f-9147-9272d68d2e5c | VF20GLX    | ACTIVE | None       | Running     | int=10.0.0.5, 192.168.1.103 |
| 770e33f7-7aab-49f1-95ca-3cf343f744ef | VF20RS01   | ACTIVE | None       | Running     | int=10.0.0.2, 192.168.1.101 |
+--------------------------------------+------------+--------+------------+-------------+-----------------------------+

[root@dallas1 ~(keystone_admin)]$ vgdisplay
....

--- Volume group ---
VG Name               cinder-volumes
System ID
Format                lvm2
Metadata Areas        1
Metadata Sequence No 66
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                3
Open LV               3
Max PV                0
Cur PV                1
Act PV                1
VG Size               20.00 GiB
PE Size               4.00 MiB
Total PE              5119
Alloc PE / Size       3840 / 15.00 GiB
Free PE / Size       1279 / 5.00 GiB
VG UUID               M11ikP-i6sd-ftwG-3XIH-F9wt-cSHe-m9kCtU

....

Three volumes have been created each one 5 GB

[root@dallas1 ~(keystone_admin)]$ losetup -a

/dev/loop0: [64768]:14 (/cinder-volumes)

Same messages in log , but now it works - just one time "Try number :0" no 1,2,.., no python trace back which means failure to boot from volume

2014-03-04 19:40:24.836 1579 WARNING nova.virt.libvirt.driver [req-ee27c779-22f6-468d-b5e0-74cab84228ff df4a984ce2f24848a6b84aaa99e296f1 e896be65e94a4893b870bc29ba86d7eb] [instance: e41bec90-16ac-4d60-9b33-e466d3e91369] File injection into a boot from volume instance is not supported
2014-03-04 19:40:27.379 1579 WARNING nova.virt.libvirt.volume [req-ee27c779-22f6-468d-b5e0-74cab84228ff df4a984ce2f24848a6b84aaa99e296f1 e896be65e94a4893b870bc29ba86d7eb] ISCSI volume not yet found at: vda. Will rescan & retry. Try number: 0
2014-03-04 19:40:27.458 1579 WARNING nova.virt.libvirt.vif [req-ee27c779-22f6-468d-b5e0-74cab84228ff df4a984ce2f24848a6b84aaa99e296f1 e896be65e94a4893b870bc29ba86d7eb] Deprecated: The LibvirtHybridOVSBridgeDriver VIF driver is now deprecated and will be removed in the next release. Please use the LibvirtGenericVIFDriver VIF driver, together with a network plugin that reports the 'vif_type' attribute
2014-03-04 19:42:38.963 1579 WARNING nova.virt.libvirt.driver [req-fdd0396e-7003-4a44-9a56-23de90ce6dc3 df4a984ce2f24848a6b84aaa99e296f1 e896be65e94a4893b870bc29ba86d7eb] [instance: 4f2515ad-e879-4c48-9258-e65687a0cf07] File injection into a boot from volume instance is not supported
2014-03-04 19:42:40.812 1579 WARNING nova.virt.libvirt.volume [req-fdd0396e-7003-4a44-9a56-23de90ce6dc3 df4a984ce2f24848a6b84aaa99e296f1 e896be65e94a4893b870bc29ba86d7eb] ISCSI volume not yet found at: vda. Will rescan & retry. Try number: 0
2014-03-04 19:42:40.898 1579 WARNING nova.virt.libvirt.vif [req-fdd0396e-7003-4a44-9a56-23de90ce6dc3 df4a984ce2f24848a6b84aaa99e296f1 e896be65e94a4893b870bc29ba86d7eb] Deprecated: The LibvirtHybridOVSBridgeDriver VIF driver is now deprecated and will be removed in the next release. Please use the LibvirtGenericVIFDriver VIF driver, together with a network plugin that reports the 'vif_type' attribute
2014-03-04 19:48:54.628 1579 WARNING nova.compute.manager [-] Found 3 in the database and 0 on the hypervisor.
2014-03-04 19:49:39.588 1579 WARNING nova.virt.libvirt.driver [req-c05e5c30-b4e3-4034-8f70-60e0251caba5 df4a984ce2f24848a6b84aaa99e296f1 e896be65e94a4893b870bc29ba86d7eb] [instance: 1ffff3bf-f811-4f33-b563-6dd70be3191b] File injection into a boot from volume instance is not supported
2014-03-04 19:49:41.076 1579 WARNING nova.virt.libvirt.volume [req-c05e5c30-b4e3-4034-8f70-60e0251caba5 df4a984ce2f24848a6b84aaa99e296f1 e896be65e94a4893b870bc29ba86d7eb] ISCSI volume not yet found at: vda. Will rescan & retry. Try number: 0
2014-03-04 19:49:41.159 1579 WARNING nova.virt.libvirt.vif [req-c05e5c30-b4e3-4034-8f70-60e0251caba5 df4a984ce2f24848a6b84aaa99e296f1 e896be65e94a4893b870bc29ba86d7eb] Deprecated: The LibvirtHybridOVSBridgeDriver VIF driver is now deprecated and will be removed in the next release. Please use the LibvirtGenericVIFDriver VIF driver, together with a network plugin that reports the 'vif_type' attribute

References
1. http://bderzhavets.blogspot.com/2014/01/setting-up-two-physical-node-openstack.html

Xen Virtualization on Linux and Solaris

Sunday, March 09, 2014

Setup Gluster 3.4.2 on Two Node Controller&Compute Neutron GRE+OVS Fedora 20 Cluster

Monday, March 03, 2014

Up to date procedure of creating cinder's ThinLVM based cloud instances of F20,Ubuntu 13.10 on F20 Havana Compute Node.

About Me

Blog Archive