Sunday, March 09, 2014

Setup Gluster 3.4.2 on Two Node Controller&Compute Neutron GRE+OVS Fedora 20 Cluster

This post is an update for http://bderzhavets.blogspot.com/2014/01/setting-up-two-physical-node-openstack.html  . It's focused on Gluster 3.4.2  implementation including tuning /etc/sysconfig/iptables files on Controller and Compute Nodes.
Copying ssh-key from master node to compute, step by step verification of gluster volume replica 2  functionality and switching RDO Havana cinder services to work with gluster volume created  to store instances bootable cinders volumes for performance improvement. Of course creating gluster bricks under "/"  is not recommended . It should be a separate mount point for "xfs" filesystem to store gluster bricks on each node. View also VNC Console in Dashboard on Two Node Neutron GRE+OVS+Gluster F20 Cluster 
   Typo in dhcp_agent.ini :- the reference for "dnsmasq.conf"  has been fixed and line "dhcp-option=26,1454" added to dnsmasq.conf . Updated configuration files are critical for launching instance without "Customization script" and allow to work with usual ssh keypair.  Actually , when updates are done instance gets created with MTU 1454. View  http://bderzhavets.blogspot.com/2014/03/tuning-neutron-dhcp-agent-dnsmasq-on.html

Have to notice that http://kashyapc.fedorapeople.org/virt/openstack/neutron-configs-GRE-OVS-two-node.txt in meantime (03/26/2014) has dhcp_agent.ini and l3_agent.ini  identical (i.e. the last one is missing rows


metadata_ip = 192.168.1.127
metadata_port = 8700
 
# and contains instead of them reference:-
 
dnsmasq_config_file = /etc/neutron/dnsmasq.conf

)

It is manual Setup 
- Controller node: Nova, Keystone, Cinder, Glance, Neutron (using Open vSwitch plugin and GRE tunneling )
 - Compute node: Nova (nova-compute), Neutron (openvswitch-agent)

Basic updated setup procedure

Final draft for Nova & Neutron Configuration files
 
dallas1.localdomain   -  Controller (192.168.1.130)
dallas2.localdomain   -  Compute   (192.168.1.140)


First step is tuning /etc/sysconfig/iptables for IPv4 iptables firewall (service firewalld should be disabled) :-

Update /etc/sysconfig/iptables on both nodes:-

-A INPUT -p tcp -m multiport --dport 24007:24047 -j ACCEPT
-A INPUT -p tcp --dport 111 -j ACCEPT
-A INPUT -p udp --dport 111 -j ACCEPT
-A INPUT -p tcp -m multiport --dport 38465:38485 -j ACCEPT

Comment out lines bellow , ignoring instruction from http://kashyapc.fedorapeople.org/virt/openstack/neutron-configs-GRE-OVS-two-node.txt  . It's critical for Gluster functionality. Having them active you are supposed to work with thin LVM as cinder volumes. You won't be able even remote mount with "-t glusterfs" option. Gluster replications will be dead for ever.


# -A FORWARD -j REJECT --reject-with icmp-host-prohibited
# -A INPUT -j REJECT --reject-with icmp-host-prohibited
 
 Restart service iptables on both nodes

Second step:-


On dallas1, run the following commands :

# ssh-keygen (Hit Enter to accept all of the defaults)
# ssh-copy-id -i ~/.ssh/id_rsa.pub root@dallas2

On both nodes run :-

# yum  -y install glusterfs glusterfs-server glusterfs-fuse
# service glusterd start

On dallas1

#gluster peer probe dallas2.localdomain

Should return "success"


[root@dallas1 ~(keystone_admin)]$ gluster peer status
Number of Peers: 1

Hostname: dallas2.localdomain
Uuid: b3b1cf43-2fec-4904-82d4-b9be03f77c5f
State: Peer in Cluster (Connected)

On dallas2

[root@dallas2 ~]# gluster peer status
Number of Peers: 1

Hostname: 192.168.1.130
Uuid: a57433dd-4a1a-4442-a5ae-ba2f682e5c79
State: Peer in Cluster (Connected)


*************************************************************************
On Controller (192.168.1.130) and on Compute (192.168.1.140)
*************************************************************************

Verify ports availability:-

[root@dallas1 ~(keystone_admin)]$ netstat -lntp | grep gluster
tcp    0      0 0.0.0.0:655            0.0.0.0:*    LISTEN      2591/glusterfs     
tcp    0      0 0.0.0.0:49152        0.0.0.0:*    LISTEN      2524/glusterfsd    
tcp    0      0 0.0.0.0:2049          0.0.0.0:*    LISTEN      2591/glusterfs     
tcp    0      0 0.0.0.0:38465        0.0.0.0:*    LISTEN      2591/glusterfs     
tcp    0      0 0.0.0.0:38466        0.0.0.0:*    LISTEN      2591/glusterfs     
tcp    0      0 0.0.0.0:49155        0.0.0.0:*    LISTEN      2525/glusterfsd    
tcp    0      0 0.0.0.0:38468        0.0.0.0:*    LISTEN      2591/glusterfs     
tcp    0      0 0.0.0.0:38469        0.0.0.0:*    LISTEN      2591/glusterfs     
tcp    0      0 0.0.0.0:24007        0.0.0.0:*    LISTEN      2380/glusterd 


************************************
Switching Cinder to Gluster volume
************************************

# gluster volume create cinder-volumes021  replica 2 ddallas1.localdomain:/FDR/Replicate   dallas2.localdomain:/FDR/Replicate force

# gluster volume start cinder-volumes021

# gluster volume set cinder-volumes021  auth.allow 192.168.1.*

[root@dallas1 ~(keystone_admin)]$ gluster volume info cinder-volumes012

Volume Name: cinder-volumes012
Type: Replicate
Volume ID: 9ee31c6c-0ae3-4fee-9886-b9cb6a518f48
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: dallas1.localdomain:/FDR/Replicate
Brick2: dallas2.localdomain:/FDR/Replicate
Options Reconfigured:
storage.owner-gid: 165
storage.owner-uid: 165
auth.allow: 192.168.1.*

[root@dallas1 ~(keystone_admin)]$ gluster volume status cinder-volumes012
Status of volume: cinder-volumes012
Gluster process                                                    Port    Online    Pid
------------------------------------------------------------------------------
Brick dallas1.localdomain:/FDR/Replicate         49155    Y    2525
Brick dallas2.localdomain:/FDR/Replicate         49152    Y    1615
NFS Server on localhost                                         2049    Y    2591
Self-heal Daemon on localhost                               N/A    Y    2596
NFS Server on dallas2.localdomain                       2049    Y    2202
Self-heal Daemon on dallas2.localdomain             N/A    Y    2197



# openstack-config --set /etc/cinder/cinder.conf DEFAULT volume_driver cinder.volume.drivers.glusterfs.GlusterfsDriver

# openstack-config --set /etc/cinder/cinder.conf DEFAULT glusterfs_shares_config /etc/cinder/shares.conf

# openstack-config --set /etc/cinder/cinder.conf DEFAULT glusterfs_mount_point_base /var/lib/cinder/volumes

# vi /etc/cinder/shares.conf
    192.168.1.130:/cinder-volumes021

:wq

Make sure all thin LVM have been deleted via `cinder list` , if no then delete them all.

[root@dallas1 ~(keystone_admin)]$ for i in api scheduler volume ; do service openstack-cinder-${i} restart ; done

 It should add row to `df -h` output :

192.168.1.130:/cinder-volumes012  187G   32G  146G  18% /var/lib/cinder/volumes/1c9688348ab38662e3ac8fb121077d34



[root@dallas1 ~(keystone_admin)]$ openstack-status

== Nova services ==
openstack-nova-api:                        active
openstack-nova-cert:                       inactive  (disabled on boot)
openstack-nova-compute:               inactive  (disabled on boot)
openstack-nova-network:                inactive  (disabled on boot)
openstack-nova-scheduler:             active
openstack-nova-volume:                  inactive  (disabled on boot)
openstack-nova-conductor:             active
== Glance services ==
openstack-glance-api:                   active
openstack-glance-registry:           active
== Keystone service ==
openstack-keystone:                     active
== neutron services ==
neutron-server:                         active
neutron-dhcp-agent:                active
neutron-l3-agent:                     active
neutron-metadata-agent:        active
neutron-lbaas-agent:                    inactive  (disabled on boot)
neutron-openvswitch-agent:       active
neutron-linuxbridge-agent:         inactive  (disabled on boot)
neutron-ryu-agent:                      inactive  (disabled on boot)
neutron-nec-agent:                      inactive  (disabled on boot)
neutron-mlnx-agent:                   inactive  (disabled on boot)
== Cinder services ==
openstack-cinder-api:                   active
openstack-cinder-scheduler:        active
openstack-cinder-volume:             active
== Support services ==
mysqld:                                 inactive  (disabled on boot)
libvirtd:                               active
openvswitch:                            active
dbus:                                   active
tgtd:                                   active
qpidd:                                  active
== Keystone users ==
+----------------------------------+---------+---------+-------+
|                id                |   name  | enabled | email |
+----------------------------------+---------+---------+-------+
| 871cf99617ff40e09039185aa7ab11f8 |  admin  |   True  |       |
| df4a984ce2f24848a6b84aaa99e296f1 |  boris  |   True  |       |
| 57fc5466230b497a9f206a20618dbe25 |  cinder |   True  |       |
| cdb2e5af7bae4c5486a1e3e2f42727f0 |  glance |   True  |       |
| adb14139a0874c74b14d61d2d4f22371 | neutron |   True  |       |
| 2485122e3538409c8a6fa2ea4343cedf |   nova  |   True  |       |
+----------------------------------+---------+---------+-------+
== Glance images ==
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| ID                                   | Name                | Disk Format | Container Format | Size      | Status |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| 592faef8-308a-4438-867a-17adf685cde4 | CirrOS 31           | qcow2       | bare             | 13147648  | active |
| d0e90250-5814-4685-9b8d-65ec9daa7117 | Fedora 20 x86_64    | qcow2       | bare             | 214106112 | active |
| 3e6eea8e-32e6-4373-9eb1-e04b8a3167f9 | Ubuntu Server 13.10 | qcow2       | bare             | 244777472 | active |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
== Nova managed services ==
+----------------+---------------------+----------+---------+-------+----------------------------+-----------------+
| Binary         | Host                | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+----------------+---------------------+----------+---------+-------+----------------------------+-----------------+
| nova-scheduler | dallas1.localdomain | internal | enabled | up    | 2014-03-09T14:19:31.000000 | None            |
| nova-conductor | dallas1.localdomain | internal | enabled | up    | 2014-03-09T14:19:30.000000 | None            |
| nova-compute   | dallas2.localdomain | nova     | enabled | up    | 2014-03-09T14:19:33.000000 | None            |
+----------------+---------------------+----------+---------+-------+----------------------------+-----------------+
== Nova networks ==
+--------------------------------------+-------+------+
| ID                                   | Label | Cidr |
+--------------------------------------+-------+------+
| 0ed406bf-3552-4036-9006-440f3e69618e | ext   | None |
| 166d9651-d299-47df-a5a1-b368e87b612f | int   | None |
+--------------------------------------+-------+------+
== Nova instance flavors ==
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| ID | Name      | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| 1  | m1.tiny   | 512       | 1    | 0         |      | 1     | 1.0         | True      |
| 2  | m1.small  | 2048      | 20   | 0         |      | 1     | 1.0         | True      |
| 3  | m1.medium | 4096      | 40   | 0         |      | 2     | 1.0         | True      |
| 4  | m1.large  | 8192      | 80   | 0         |      | 4     | 1.0         | True      |
| 5  | m1.xlarge | 16384     | 160  | 0         |      | 8     | 1.0         | True      |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
== Nova instances ==
+----+------+--------+------------+-------------+----------+
| ID | Name | Status | Task State | Power State | Networks |
+----+------+--------+------------+-------------+----------+
+----+------+--------+------------+-------------+----------+


 [root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+-----------+--------+------------+-------------+-----------------------------+
| ID                                   | Name      | Status | Task State | Power State | Networks                    |
+--------------------------------------+-----------+--------+------------+-------------+-----------------------------+
| 9566adec-9406-4c3e-bce5-109ecb8bcf6b | VF20RS007 | ACTIVE | None       | Running     | int=10.0.0.2, 192.168.1.101 |
| 8142ee4c-ef56-4b61-8a0b-ecd82d21484f | VF20RS012 | ACTIVE | None       | Running     | int=10.0.0.4, 192.168.1.102 |
+--------------------------------------+-----------+--------+------------+-------------+-----------------------------+

[root@dallas1 ~(keystone_boris)]$ df -h
Filesystem                       Size  Used Avail Use% Mounted on
/dev/mapper/fedora01-root        187G   32G  146G  18% /
devtmpfs                         3.9G     0  3.9G   0% /dev
tmpfs                            3.9G  184K  3.9G   1% /dev/shm
tmpfs                            3.9G  9.1M  3.9G   1% /run
tmpfs                            3.9G     0  3.9G   0% /sys/fs/cgroup
tmpfs                            3.9G  464K  3.9G   1% /tmp
/dev/sdb5                        477M  122M  327M  28% /boot
tmpfs                            3.9G  9.1M  3.9G   1% /run/netns
192.168.1.130:/cinder-volumes012  187G   32G  146G  18% /var/lib/cinder/volumes/1c9688348ab38662e3ac8fb121077d34

(neutron) agent-list
+--------------------------------------+--------------------+---------------------+-------+----------------+
| id                                   | agent_type         | host                | alive | admin_state_up |
+--------------------------------------+--------------------+---------------------+-------+----------------+
| 3ed1cd15-81af-4252-9d6f-e9bb140bf6cf | L3 agent           | dallas1.localdomain | :-)   | True           |
| a088a6df-633c-4959-a316-510c99f3876b | DHCP agent         | dallas1.localdomain | :-)   | True           |
| a3e5200c-b391-4930-b3ee-58c8d1b13c73 | Open vSwitch agent | dallas1.localdomain | :-)   | True           |
| b6da839a-0d93-44ad-9793-6d0919fbb547 | Open vSwitch agent | dallas2.localdomain | :-)   | True           |
+--------------------------------------+--------------------+---------------------+-------+----------------+

If Controller has been correctly set up:-

[root@dallas1 ~(keystone_admin)]$ netstat -lntp | grep python
tcp    0      0 0.0.0.0:8700         0.0.0.0:*     LISTEN      1160/python        
tcp    0      0 0.0.0.0:35357       0.0.0.0:*     LISTEN      1163/python        
tcp   0      0 0.0.0.0:9696          0.0.0.0:*      LISTEN      1165/python        
tcp   0      0 0.0.0.0:8773          0.0.0.0:*      LISTEN      1160/python        
tcp   0      0 0.0.0.0:8774          0.0.0.0:*      LISTEN      1160/python        
tcp   0      0 0.0.0.0:9191          0.0.0.0:*      LISTEN      1173/python        
tcp   0      0 0.0.0.0:8776          0.0.0.0:*      LISTEN      8169/python        
tcp   0      0 0.0.0.0:5000          0.0.0.0:*      LISTEN      1163/python        
tcp   0      0 0.0.0.0:9292          0.0.0.0:*      LISTEN      1168/python 



**********************************************
Creating instance utilizing glusterfs volume
**********************************************

[root@dallas1 ~(keystone_boris)]$ glance image-list
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| ID                                   | Name                | Disk Format | Container Format | Size      | Status |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| 592faef8-308a-4438-867a-17adf685cde4 | CirrOS 31           | qcow2       | bare             | 13147648  | active |
| d0e90250-5814-4685-9b8d-65ec9daa7117 | Fedora 20 x86_64    | qcow2       | bare             | 214106112 | active |
| 3e6eea8e-32e6-4373-9eb1-e04b8a3167f9 | Ubuntu Server 13.10 | qcow2       | bare             | 244777472 | active |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+

   I have to notice that schema with `cinder create --image-id  .. --display_name VOL_NAME SIZE` & `nova boot --flavor 2 --user-data=./myfile.txt --block_device_mapping vda=volume_id:::0 VM_NAME`  doesn't work stable  for me in meantime.

As of 03/11/2014  standard schema via `cinder create --image-id IMAGE_ID --display_name VOL_NAME SIZE `& ` nova boot --flavor 2 --user-data=./myfile.txt --block_device_mapping vda=VOLUME_ID:::0  INSTANCE_NAME`  started to work fine. However, schema described bellow on the contrary stopped to work on glusterfs based cinder's volumes.Regarding last issue view recent post:-
Launching Instance via image and creating simultaneously bootable cinder volume (glusterfs) on Two Node GRE+OVS+Gluster F20 Cluster
http://bderzhavets.blogspot.com/2014/03/launching-instance-via-image-and.html

Create myfile.txt in cuurent directory like this :-

[root@dallas1 ~(keystone_boris)]$  cat ./myfile.txt
#cloud-config
password: mysecret
chpasswd: { expire: False }
ssh_pwauth: True

 
[root@dallas1 ~(keystone_boris)]$ nova boot --flavor 2 --user-data=./myfile.txt --block-device source=image,id=d0e90250-5814-4685-9b8d-65ec9daa7117,dest=volume,size=5,shutdown=preserve,bootindex=0 VF20RS012
+--------------------------------------+-------------------------------------------------+
| Property                             | Value                                           |
+--------------------------------------+-------------------------------------------------+
| status                               | BUILD                                           |
| updated                              | 2014-03-09T12:41:22Z                            |
| OS-EXT-STS:task_state                | scheduling                                      |
| key_name                             | None                                            |
| image                                | Attempt to boot from volume - no image supplied |
| hostId                               |                                                 |
| OS-EXT-STS:vm_state                  | building                                        |
| OS-SRV-USG:launched_at               | None                                            |
| flavor                               | m1.small                                        |
| id                                   | 8142ee4c-ef56-4b61-8a0b-ecd82d21484f            |
| security_groups                      | [{u'name': u'default'}]                         |
| OS-SRV-USG:terminated_at             | None                                            |
| user_id                              | df4a984ce2f24848a6b84aaa99e296f1                |
| name                                 | VF20RS012                                       |
| adminPass                            | eFDhC8ZSCFU2                                    |
| tenant_id                            | e896be65e94a4893b870bc29ba86d7eb                |
| created                              | 2014-03-09T12:41:22Z                            |
| OS-DCF:diskConfig                    | MANUAL                                          |
| metadata                             | {}                                              |
| os-extended-volumes:volumes_attached | []                                              |
| accessIPv4                           |                                                 |
| accessIPv6                           |                                                 |
| progress                             | 0                                               |
| OS-EXT-STS:power_state               | 0                                               |
| OS-EXT-AZ:availability_zone          | nova                                            |
| config_drive                         |                                                 |
+--------------------------------------+-------------------------------------------------+

[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+-----------+-----------+----------------------+-------------+-----------------------------+
| ID                                   | Name      | Status    | Task State           | Power State | Networks                    |
+--------------------------------------+-----------+-----------+----------------------+-------------+-----------------------------+
| 9566adec-9406-4c3e-bce5-109ecb8bcf6b | VF20RS007 | SUSPENDED | None                 | Shutdown    | int=10.0.0.2, 192.168.1.101 |
| 8142ee4c-ef56-4b61-8a0b-ecd82d21484f | VF20RS012 | BUILD     | block_device_mapping | NOSTATE     |                             |
+--------------------------------------+-----------+-----------+----------------------+-------------+-----------------------------+


WAIT ...

[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+-----------+-----------+------------+-------------+-----------------------------+
| ID                                   | Name      | Status    | Task State | Power State | Networks                    |
+--------------------------------------+-----------+-----------+------------+-------------+-----------------------------+
| 9566adec-9406-4c3e-bce5-109ecb8bcf6b | VF20RS007 | SUSPENDED | None       | Shutdown    | int=10.0.0.2, 192.168.1.101 |
| 8142ee4c-ef56-4b61-8a0b-ecd82d21484f | VF20RS012 | ACTIVE    | None       | Running     | int=10.0.0.4                |
+--------------------------------------+-----------+-----------+------------+-------------+-----------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron floatingip-create ext
Created a new floatingip:
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| fixed_ip_address    |                                      |
| floating_ip_address | 192.168.1.102                        |
| floating_network_id | 0ed406bf-3552-4036-9006-440f3e69618e |
| id                  | 5c74667d-9b22-4092-ae0a-70ff3a06e785 |
| port_id             |                                      |
| router_id           |                                      |
| tenant_id           | e896be65e94a4893b870bc29ba86d7eb     |
+---------------------+--------------------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron port-list --device-id 8142ee4c-ef56-4b61-8a0b-ecd82d21484f
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                       |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| dc60b5f4-739e-49bd-a004-3ef806e2b488 |      | fa:16:3e:70:56:cc | {"subnet_id": "2e838119-3e2e-46e8-b7cc-6d00975046f2", "ip_address": "10.0.0.4"} |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron floatingip-associate 5c74667d-9b22-4092-ae0a-70ff3a06e785 dc60b5f4-739e-49bd-a004-3ef806e2b488

Associated floatingip 5c74667d-9b22-4092-ae0a-70ff3a06e785

[root@dallas1 ~(keystone_boris)]$ ping 192.168.1.102
PING 192.168.1.102 (192.168.1.102) 56(84) bytes of data.
64 bytes from 192.168.1.102: icmp_seq=1 ttl=63 time=6.23 ms
64 bytes from 192.168.1.102: icmp_seq=2 ttl=63 time=0.702 ms
64 bytes from 192.168.1.102: icmp_seq=3 ttl=63 time=1.07 ms
64 bytes from 192.168.1.102: icmp_seq=4 ttl=63 time=0.693 ms
64 bytes from 192.168.1.102: icmp_seq=5 ttl=63 time=1.80 ms
64 bytes from 192.168.1.102: icmp_seq=6 ttl=63 time=0.750 ms
^C

[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+-----------+--------+------------+-------------+-----------------------------+
| ID                                   | Name      | Status | Task State | Power State | Networks                    |
+--------------------------------------+-----------+--------+------------+-------------+-----------------------------+
| 9566adec-9406-4c3e-bce5-109ecb8bcf6b | VF20RS007 | ACTIVE | None       | Running     | int=10.0.0.2, 192.168.1.101 |
| 8142ee4c-ef56-4b61-8a0b-ecd82d21484f | VF20RS012 | ACTIVE | None       | Running     | int=10.0.0.4, 192.168.1.102 |
+--------------------------------------+-----------+--------+------------+-------------+-----------------------------+

[root@dallas1 ~(keystone_boris)]$ cinder list
+--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+
|                  ID                  | Status | Display Name | Size | Volume Type | Bootable |             Attached to              |
+--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+
| 575be853-b104-458e-bc72-1785ef524416 | in-use |              |  5   |     None    |   true   | 8142ee4c-ef56-4b61-8a0b-ecd82d21484f |
| 9794bd45-8923-4f3e-a48f-fa1d62a964f8  | in-use |              |  5   |     None    |   true   | 9566adec-9406-4c3e-bce5-109ecb8bcf6b |
+--------------------------------------+--------+--------------+------+-------------+----------+------------------------------

Script to assign floating IP to new VM ($1 is the name of VM created) :-

#!/bin/bash -x
neutron floatingip-create ext > newip.log
NewIpId=`cat newip.log | grep ^"| id" | awk '{print $4}' ` ;
NewIp=`cat newip.log | grep "192.168.1"| awk '{print $4}' ` ;
NewVmId=`nova list | grep $1 | awk '{print $2}' ` ;
NewDevId=`neutron port-list --device-id $NewVmId | grep "subnet_id" | awk '{print $2}' ` ;
neutron floatingip-associate $NewIpId $NewDevId ;
echo "New floating Ip assigned : "$NewIp
sleep 10
ping $NewIp


[root@dallas1 ~(keystone_boris)]$ ./AssignIP.sh  VF20RS012
+ neutron floatingip-create ext
++ grep '^| id'
++ cat newip.log
++ awk '{print $4}'
+ NewIpId=23b63cd5-3264-4861-abeb-ed27c053b93e
++ cat newip.log
++ awk '{print $4}'
++ grep 192.168.1
+ NewIp=192.168.1.110
++ nova list
++ awk '{print $2}'
++ grep VF20RS012
+ NewVmId=9a902538-6c7b-43eb-a0c5-cea801561c6a
++ neutron port-list --device-id 9a902538-6c7b-43eb-a0c5-cea801561c6a
++ grep subnet_id
++ awk '{print $2}'
+ NewDevId=78d0c401-731a-4ee0-80ae-a6afb4151241
+ neutron floatingip-associate 23b63cd5-3264-4861-abeb-ed27c053b93e 78d0c401-731a-4ee0-80ae-a6afb4151241
Associated floatingip 23b63cd5-3264-4861-abeb-ed27c053b93e
+ echo 'New floating Ip assigned : 192.168.1.110'
New floating Ip assigned : 192.168.1.110

. . . . .

64 bytes from 192.168.1.110: icmp_seq=21 ttl=63 time=8.65 ms
64 bytes from 192.168.1.110: icmp_seq=22 ttl=63 time=6.65 ms
64 bytes from 192.168.1.110: icmp_seq=23 ttl=63 time=1.19 ms
64 bytes from 192.168.1.110: icmp_seq=24 ttl=63 time=1.31 ms
64 bytes from 192.168.1.110: icmp_seq=25 ttl=63 time=1.42 ms


*******************
On Compute:-
*******************

[root@dallas1 ~]# ssh 192.168.1.140
Last login: Sun Mar  9 16:46:40 2014

[root@dallas2 ~]# df -h
Filesystem                       Size  Used Avail Use% Mounted on
/dev/mapper/fedora01-root        187G   18G  160G  11% /
devtmpfs                         3.9G     0  3.9G   0% /dev
tmpfs                            3.9G  3.1M  3.9G   1% /dev/shm
tmpfs                            3.9G  9.4M  3.9G   1% /run
tmpfs                            3.9G     0  3.9G   0% /sys/fs/cgroup
tmpfs                            3.9G  115M  3.8G   3% /tmp
/dev/sdb5                        477M  122M  327M  28% /boot
192.168.1.130:/cinder-volumes012  187G   32G  146G  18% /var/lib/nova/mnt/1c9688348ab38662e3ac8fb121077d34


[root@dallas2 ~]# ps -ef| grep nova
nova      1548     1  0 16:29 ?        00:00:42 /usr/bin/python /usr/bin/nova-compute --logfile /var/log/nova/compute.log

root      3005     1  0 16:34 ?        00:00:38 /usr/sbin/glusterfs --volfile-id=cinder-volumes012 --volfile-server=192.168.1.130 /var/lib/nova/mnt/1c9688348ab38662e3ac8fb121077d34

qemu      4762     1 58 16:42 ?        00:52:17 /usr/bin/qemu-system-x86_64 -name instance-00000061 -S -machine pc-i440fx-1.6,accel=tcg,usb=off -cpu Penryn,+osxsave,+xsave,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 8142ee4c-ef56-4b61-8a0b-ecd82d21484f -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2013.2.2-1.fc20,serial=6050001e-8c00-00ac-818a-90e6ba2d11eb,uuid=8142ee4c-ef56-4b61-8a0b-ecd82d21484f -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000061.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/mnt/1c9688348ab38662e3ac8fb121077d34/volume-575be853-b104-458e-bc72-1785ef524416,if=none,id=drive-virtio-disk0,format=raw,serial=575be853-b104-458e-bc72-1785ef524416,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=24,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:70:56:cc,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/8142ee4c-ef56-4b61-8a0b-ecd82d21484f/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

qemu      6330     1 44 16:49 ?        00:36:02 /usr/bin/qemu-system-x86_64 -name instance-0000005f -S -machine pc-i440fx-1.6,accel=tcg,usb=off -cpu Penryn,+osxsave,+xsave,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 9566adec-9406-4c3e-bce5-109ecb8bcf6b -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2013.2.2-1.fc20,serial=6050001e-8c00-00ac-818a-90e6ba2d11eb,uuid=9566adec-9406-4c3e-bce5-109ecb8bcf6b -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-0000005f.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/mnt/1c9688348ab38662e3ac8fb121077d34/volume-9794bd45-8923-4f3e-a48f-fa1d62a964f8,if=none,id=drive-virtio-disk0,format=raw,serial=9794bd45-8923-4f3e-a48f-fa1d62a964f8,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=27,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:50:84:72,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/9566adec-9406-4c3e-bce5-109ecb8bcf6b/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:1 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -incoming fd:24 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
root     24713 24622  0 18:11 pts/4    00:00:00 grep --color=auto nova

[root@dallas2 ~]# ps -ef| grep neutron
neutron   1549     1  0 16:29 ?        00:00:53 /usr/bin/python /usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini --log-file /var/log/neutron/openvswitch-agent.log
root     24981 24622  0 18:12 pts/4    00:00:00 grep --color=auto neutron




  Top at Compute node (192.168.1.140)


      Runtime at Compute node ( dallas2 192.168.1.140)

 ******************************************************
 Building Ubuntu 14.04 instance via cinder volume
 ******************************************************

[root@dallas1 ~(keystone_boris)]$ glance image-list
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| ID                                   | Name                | Disk Format | Container Format | Size      | Status |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+
| 592faef8-308a-4438-867a-17adf685cde4 | CirrOS 31           | qcow2       | bare             | 13147648  | active |
| d0e90250-5814-4685-9b8d-65ec9daa7117 | Fedora 20 x86_64    | qcow2       | bare             | 214106112 | active |
| c2b7c3ed-e25d-44c4-a5e7-4e013c4a8b00 | Ubuntu 14.04        | qcow2       | bare             | 264176128 | active |
| 3e6eea8e-32e6-4373-9eb1-e04b8a3167f9 | Ubuntu Server 13.10 | qcow2       | bare             | 244777472 | active |
+--------------------------------------+---------------------+-------------+------------------+-----------+--------+

[root@dallas1 ~(keystone_boris)]$ cinder create --image-id c2b7c3ed-e25d-44c4-a5e7-4e013c4a8b00 --display_name UbuntuTrusty 5
+---------------------+--------------------------------------+
|       Property      |                Value                 |
+---------------------+--------------------------------------+
|     attachments     |                  []                  |
|  availability_zone  |                 nova                 |
|       bootable      |                false                 |
|      created_at     |      2014-03-10T06:35:39.873978      |
| display_description |                 None                 |
|     display_name    |             UbuntuTrusty             |
|          id         | 8bcc02a7-b9ba-4cd6-a6b9-0574889bf8d2 |
|       image_id      | c2b7c3ed-e25d-44c4-a5e7-4e013c4a8b00 |
|       metadata      |                  {}                  |
|         size        |                  5                   |
|     snapshot_id     |                 None                 |
|     source_volid    |                 None                 |
|        status       |               creating               |
|     volume_type     |                 None                 |
+---------------------+--------------------------------------+


[root@dallas1 ~(keystone_boris)]$ cinder list
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+
|                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable |             Attached to              |
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+
| 56ceaaa8-c0ec-45f3-98a4-555c1231b34e |   in-use  |              |  5   |     None    |   true   | e29606c5-582f-4766-ae1b-52043a698743 |
| 575be853-b104-458e-bc72-1785ef524416 |   in-use  |              |  5   |     None    |   true   | 8142ee4c-ef56-4b61-8a0b-ecd82d21484f |
| 8bcc02a7-b9ba-4cd6-a6b9-0574889bf8d2 | available | UbuntuTrusty |  5   |     None    |   true   |                                      |
| 9794bd45-8923-4f3e-a48f-fa1d62a964f8 |   in-use  |              |  5   |     None    |   true   | 9566adec-9406-4c3e-bce5-109ecb8bcf6b |
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+

[root@dallas1 ~(keystone_boris)]$  nova boot --flavor 2 --user-data=./myfile.txt --block_device_mapping vda=8bcc02a7-b9ba-4cd6-a6b9-0574889bf8d2:::0 UbuntuTR01
+--------------------------------------+----------------------------------------------------+
| Property                             | Value                                              |
+--------------------------------------+----------------------------------------------------+
| status                               | BUILD                                              |
| updated                              | 2014-03-10T06:40:14Z                               |
| OS-EXT-STS:task_state                | scheduling                                         |
| key_name                             | None                                               |
| image                                | Attempt to boot from volume - no image supplied    |
| hostId                               |                                                    |
| OS-EXT-STS:vm_state                  | building                                           |
| OS-SRV-USG:launched_at               | None                                               |
| flavor                               | m1.small                                           |
| id                                   | 0859e52d-c07b-4f56-ac79-2b37080d2843               |
| security_groups                      | [{u'name': u'default'}]                            |
| OS-SRV-USG:terminated_at             | None                                               |
| user_id                              | df4a984ce2f24848a6b84aaa99e296f1                   |
| name                                 | UbuntuTR01                                         |
| adminPass                            | L8VuhttJMbJf                                       |
| tenant_id                            | e896be65e94a4893b870bc29ba86d7eb                   |
| created                              | 2014-03-10T06:40:13Z                               |
| OS-DCF:diskConfig                    | MANUAL                                             |
| metadata                             | {}                                                 |
| os-extended-volumes:volumes_attached | [{u'id': u'8bcc02a7-b9ba-4cd6-a6b9-0574889bf8d2'}] |
| accessIPv4                           |                                                    |
| accessIPv6                           |                                                    |
| progress                             | 0                                                  |
| OS-EXT-STS:power_state               | 0                                                  |
| OS-EXT-AZ:availability_zone          | nova                                               |
| config_drive                         |                                                    |
+--------------------------------------+----------------------------------------------------+

[root@dallas1 ~(keystone_boris)]$ nova list
+--------------------------------------+------------+-----------+------------+-------------+-----------------------------+
| ID                                   | Name       | Status    | Task State | Power State | Networks                    |
+--------------------------------------+------------+-----------+------------+-------------+-----------------------------+
| 0859e52d-c07b-4f56-ac79-2b37080d2843 | UbuntuTR01 | ACTIVE    | None       | Running     | int=10.0.0.6                |
| 9566adec-9406-4c3e-bce5-109ecb8bcf6b | VF20RS007  | SUSPENDED | None       | Shutdown    | int=10.0.0.2, 192.168.1.101 |
| 8142ee4c-ef56-4b61-8a0b-ecd82d21484f | VF20RS012  | SUSPENDED | None       | Shutdown    | int=10.0.0.4, 192.168.1.102 |
| e29606c5-582f-4766-ae1b-52043a698743 | VF20RS016  | ACTIVE    | None       | Running     | int=10.0.0.5, 192.168.1.103 |
+--------------------------------------+------------+-----------+------------+-------------+-----------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron floatingip-create ext
Created a new floatingip:
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| fixed_ip_address    |                                      |
| floating_ip_address | 192.168.1.104                        |
| floating_network_id | 0ed406bf-3552-4036-9006-440f3e69618e |
| id                  | 9498ac85-82b0-468a-b526-64a659080ab9 |
| port_id             |                                      |
| router_id           |                                      |
| tenant_id           | e896be65e94a4893b870bc29ba86d7eb     |
+---------------------+--------------------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron port-list --device-id 0859e52d-c07b-4f56-ac79-2b37080d2843
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                       |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| 1f02fe57-d844-4fd8-a325-646f27163c8b |      | fa:16:3e:3f:a3:d4 | {"subnet_id": "2e838119-3e2e-46e8-b7cc-6d00975046f2", "ip_address": "10.0.0.6"} |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+

[root@dallas1 ~(keystone_boris)]$ neutron floatingip-associate  9498ac85-82b0-468a-b526-64a659080ab9 1f02fe57-d844-4fd8-a325-646f27163c8b

Associated floatingip 9498ac85-82b0-468a-b526-64a659080ab9
[root@dallas1 ~(keystone_boris)]$ ping 192.168.1.104
PING 192.168.1.104 (192.168.1.104) 56(84) bytes of data.
64 bytes from 192.168.1.104: icmp_seq=1 ttl=63 time=2.35 ms
64 bytes from 192.168.1.104: icmp_seq=2 ttl=63 time=2.56 ms
64 bytes from 192.168.1.104: icmp_seq=3 ttl=63 time=1.17 ms
64 bytes from 192.168.1.104: icmp_seq=4 ttl=63 time=4.08 ms
64 bytes from 192.168.1.104: icmp_seq=5 ttl=63 time=2.19 ms
^C


[boris@dallas1 ~]$ ssh root@192.168.1.140
root@192.168.1.140's password:
Last login: Mon Mar 10 11:32:02 2014 from 192.168.1.104

[root@dallas2 ~]# ps -ef | grep nova
nova      1564     1  0 08:42 ?        00:01:34 /usr/bin/python /usr/bin/nova-compute --logfile /var/log/nova/compute.log

root      2875     1  0 08:47 ?        00:00:40 /usr/sbin/glusterfs --volfile-id=cinder-volumes012 --volfile-server=192.168.1.130 /var/lib/nova/mnt/1c9688348ab38662e3ac8fb121077d34
root     18564 18530  0 12:29 pts/4    00:00:00 grep --color=auto nova

qemu     22753     1 37 10:09 ?        00:52:22 /usr/bin/qemu-system-x86_64 -name instance-00000063 -S -machine pc-i440fx-1.6,accel=tcg,usb=off -cpu Penryn,+osxsave,+xsave,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid e29606c5-582f-4766-ae1b-52043a698743 -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2013.2.2-1.fc20,serial=6050001e-8c00-00ac-818a-90e6ba2d11eb,uuid=e29606c5-582f-4766-ae1b-52043a698743 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000063.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/mnt/1c9688348ab38662e3ac8fb121077d34/volume-56ceaaa8-c0ec-45f3-98a4-555c1231b34e,if=none,id=drive-virtio-disk0,format=raw,serial=56ceaaa8-c0ec-45f3-98a4-555c1231b34e,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=25,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:f7:58:8c,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/e29606c5-582f-4766-ae1b-52043a698743/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

qemu     29140     1 25 10:40 ?        00:28:16 /usr/bin/qemu-system-x86_64 -name instance-00000066 -S -machine pc-i440fx-1.6,accel=tcg,usb=off -cpu Penryn,+osxsave,+xsave,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 0859e52d-c07b-4f56-ac79-2b37080d2843 -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2013.2.2-1.fc20,serial=6050001e-8c00-00ac-818a-90e6ba2d11eb,uuid=0859e52d-c07b-4f56-ac79-2b37080d2843 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000066.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/mnt/1c9688348ab38662e3ac8fb121077d34/volume-8bcc02a7-b9ba-4cd6-a6b9-0574889bf8d2,if=none,id=drive-virtio-disk0,format=raw,serial=8bcc02a7-b9ba-4cd6-a6b9-0574889bf8d2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=25,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:3f:a3:d4,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/0859e52d-c07b-4f56-ac79-2b37080d2843/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:1 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
[root@dallas2 ~]# cd /FDR/Replicate

[root@dallas2 Replicate]# ls -la
total 6470672
drwxr-xr-x.  3  165  165       4096 Mar 10 10:35 .
drwxr-xr-x.  3 root root       4096 Mar  6 00:50 ..
drw-------. 14 root root       4096 Mar 10 10:35 .glusterfs
-rw-rw-rw-.  2 qemu qemu 5368709120 Mar 10 12:29 volume-56ceaaa8-c0ec-45f3-98a4-555c1231b34e
-rw-rw-rw-.  2 root root 5368709120 Mar 10 09:59 volume-575be853-b104-458e-bc72-1785ef524416
-rw-rw-rw-.  2 qemu qemu 5368709120 Mar 10 12:30 volume-8bcc02a7-b9ba-4cd6-a6b9-0574889bf8d2
-rw-rw-rw-.  2 root root 5368709120 Mar 10 09:58 volume-9794bd45-8923-4f3e-a48f-fa1d62a964f8


 

   Another Two node Controller&Compute Neutron GRE+OVS+Gluster 3.4.2 system :-
  

Processes running on Compute node (dfw01 192.168.1.137)

 [root@dfw01 ~]# ps -ef | grep nova

root      2316     1  0 13:58 ?        00:00:00 /usr/sbin/glusterfsd -s dfw01.localdomain --volfile-id nova.dfw01.localdomain.data1-nova -p /var/lib/glusterd/vols/nova/run/dfw01.localdomain-data1-nova.pid -S /var/run/05a99291aadb7e65037a2d9c76e6eab8.socket --brick-name /data1/nova -l /var/log/glusterfs/bricks/data1-nova.log --xlator-option *-posix.glusterd-uuid=8b362d06-f504-45fd-9fa6-d2dfe46d9946 --brick-port 49152 --xlator-option nova-server.listen-port=49152

nova      3125     1  0 14:04 ?        00:01:12 /usr/bin/python /usr/bin/nova-compute --logfile /var/log/nova/compute.log

qemu      3687     1  7 14:06 ?        00:11:49 /usr/bin/qemu-system-x86_64 -name instance-00000089 -S -machine pc-i440fx-1.6,accel=tcg,usb=off -cpu Penryn,+osxsave,+xsave,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid b6c18662-3a61-45c2-9e8a-0e66824f1fee -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2013.2.2-1.fc20,serial=6050001e-8c00-00ac-818a-90e6ba2d11eb,uuid=b6c18662-3a61-45c2-9e8a-0e66824f1fee -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000089.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/instances/b6c18662-3a61-45c2-9e8a-0e66824f1fee/disk,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=24,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:d6:e2:97,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/b6c18662-3a61-45c2-9e8a-0e66824f1fee/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
root      4773     1  0 14:10 ?        00:01:22 /usr/sbin/glusterfs --volfile-id=cinder-volumes05 --volfile-server=192.168.1.127 /var/lib/nova/mnt/62f75cf6996a8a6bcc0d343be378c10a

qemu      8128     1 32 14:24 ?        00:46:12 /usr/bin/qemu-system-x86_64 -name instance-0000008b -S -machine pc-i440fx-1.6,accel=tcg,usb=off -cpu Penryn,+osxsave,+xsave,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid ca95fb8a-c434-4a83-9acb-7e32fafe8b31 -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2013.2.2-1.fc20,serial=6050001e-8c00-00ac-818a-90e6ba2d11eb,uuid=ca95fb8a-c434-4a83-9acb-7e32fafe8b31 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-0000008b.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/mnt/62f75cf6996a8a6bcc0d343be378c10a/volume-f64ae982-7d77-4a40-b697-61bffd8f30d1,if=none,id=drive-virtio-disk0,format=raw,serial=f64ae982-7d77-4a40-b697-61bffd8f30d1,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=27,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:a0:80:01,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/ca95fb8a-c434-4a83-9acb-7e32fafe8b31/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:2 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
root     10456 10308  0 16:44 pts/5    00:00:00 grep --color=auto nova

qemu     28838     1 68 15:47 ?        00:38:59 /usr/bin/qemu-system-x86_64 -name instance-0000008c -S -machine pc-i440fx-1.6,accel=tcg,usb=off -cpu Penryn,+osxsave,+xsave,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid ad8031df-e3c8-4ace-8657-c39ba6a8acf8 -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2013.2.2-1.fc20,serial=6050001e-8c00-00ac-818a-90e6ba2d11eb,uuid=ad8031df-e3c8-4ace-8657-c39ba6a8acf8 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-0000008c.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/mnt/62f75cf6996a8a6bcc0d343be378c10a/volume-b648e4dc-4521-48c5-962d-4948b9f12c76,if=none,id=drive-virtio-disk0,format=raw,serial=b648e4dc-4521-48c5-962d-4948b9f12c76,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=26,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:78:18:7f,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/ad8031df-e3c8-4ace-8657-c39ba6a8acf8/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:1 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

[root@dfw01 ~]# df -h
Filesystem                      Size  Used Avail Use% Mounted on
/dev/mapper/fedora-root          96G   47G   45G  52% /
devtmpfs                        3.9G     0  3.9G   0% /dev
tmpfs                           3.9G  2.7M  3.9G   1% /dev/shm
tmpfs                           3.9G  9.5M  3.9G   1% /run
tmpfs                           3.9G     0  3.9G   0% /sys/fs/cgroup
tmpfs                           3.9G  117M  3.8G   3% /tmp
/dev/sda5                       477M  122M  326M  28% /boot
/dev/mapper/fedora-data1         77G  3.7G   70G   6% /data1
192.168.1.127:/cinder-volumes05   77G  3.7G   70G   6% /var/lib/nova/mnt/62f75cf6996a8a6bcc0d343be378c10a

[root@dfw01 ~]# cd /var/lib/nova/mnt/62f75cf6996a8a6bcc0d343be378c10a
[root@dfw01 62f75cf6996a8a6bcc0d343be378c10a]# ls -la
total 3152316
drwxr-xr-x. 3 cinder cinder       4096 Mar 11 15:08 .
drwxr-xr-x. 3 nova   nova         4096 Jan 25 07:51 ..
-rw-rw-rw-. 1 qemu   qemu   5368709120 Mar 11 17:06 volume-b648e4dc-4521-48c5-962d-4948b9f12c76
-rw-rw-rw-. 1 qemu   qemu   5368709120 Mar 11 17:06 volume-f64ae982-7d77-4a40-b697-61bffd8f30d1

References

1. http://bderzhavets.blogspot.com/2014/02/setup-light-weight-x-windows_2.html 
2. http://bderzhavets.blogspot.com/2014/01/setting-up-two-physical-node-openstack.html