Thursday, May 22, 2014

Two Node (Controller+Compute) IceHouse Neutron OVS&VLAN Cluster on Fedora 20

Two KVMs have been created , each one having 2 virtual NICs (eth0,eth1) for
Controller && Compute Nodes setup. Before running `packstack --answer-file=twoNode-answer.txt` SELINUX set to permissive on both nodes.  Interfaces eth1 on both nodes set to promiscuous mode (e.g. HWADDRESS was commented out). Testing VMs network performance on Compute Node , I had to disable checksum offloading on eth1 on Compute :-

# /sbin/ethtool --offload eth1 tx off

In case of further problems with network,  switch to setup on non-default Libvirt's subnet ( for instance 192.169.142.0/24).

Creating  non-default Libvirt subnet per
http://kashyapc.fedorapeople.org/virt/create-a-new-libvirt-bridge.txt

1. Create a new libvirt network (other than your default 198.162.x.x) file:

$ cat openstackvms.xml 
 <network>
   <name>openstackvms</name>
   <uuid>d0e9964a-f91a-40c0-b769-a609aee41bf2</uuid>
   <forward mode='nat'>
     <nat>
       <port start='1024' end='65535'/>
     </nat>
   </forward>
   <bridge name='virbr1' stp='on' delay='0' />
   <mac address='52:54:00:60:f8:6e'/>
   <ip address='192.169.142.1' netmask='255.255.255.0'>
     <dhcp>
       <range start='192.169.142.2' end='192.169.142.254' />
     </dhcp>
   </ip>
 </network>
      
2. Define the above network:

  $ virsh net-define openstackvms.xml


3. Start the network and enable it for "autostart"

  $ virsh net-start openstackvms
  $ virsh net-autostart openstackvms


4. List your libvirt networks to see if it reflects:

  $ virsh net-list
  Name                 State      Autostart     Persistent
  ----------------------------------------------------------
  default              active     yes           yes
  openstackvms         active     yes           yes


5. Optionally, list your bridge devices:

  $ brctl show
  bridge name     bridge id               STP enabled     interfaces
  virbr0          8000.5254003339b3       yes             virbr0-nic
  virbr1          8000.52540060f86e       yes             virbr1-nic


Link for download is here Answer-file

After  `packstack --answer-file=twoNode-answer.txt` successful run.

OVS bridge br-ex && OVS port eth0 created manually

[root@ip-192-169-142-127 ~]# cat ifcfg-br-ex
DEVICE="br-ex"
BOOTPROTO="static"
IPADDR="192.169.142.127"
NETMASK="255.255.255.0"
DNS1="83.221.202.254"
BROADCAST="192.169.142.255"
GATEWAY="192.169.142.1"
NM_CONTROLLED="no"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="yes"
IPV6INIT=no
ONBOOT="yes"
TYPE="OVSBridge"
DEVICETYPE="ovs"

[root@ip-192-169-142-127 ~]# cat ifcfg-eth0
NAME="eth0"
ONBOOT="yes"
TYPE="OVSPort"
DEVICETYPE="ovs"
OVS_BRIDGE=br-ex
NM_CONTROLLED=no
IPV6INIT=no


[root@ip-192-169-142-127 ~]# ovs-vsctl add-port br-ex eth0

Service NetworkManager disabled, service network enabled.
System restarted with /etc/rc.d/rc.local :-

ifdown br-ex;
ifup br-ex ;

 [root@ip-192-169-142-127 ~(keystone_admin)]# openstack-status
== Nova services ==
openstack-nova-api:                     active
openstack-nova-cert:                    active
openstack-nova-compute:                 active
openstack-nova-network:                 inactive  (disabled on boot)
openstack-nova-scheduler:               active
openstack-nova-volume:                  inactive  (disabled on boot)
openstack-nova-conductor:               active
== Glance services ==
openstack-glance-api:                   active
openstack-glance-registry:              active
== Keystone service ==
openstack-keystone:                     active
== Horizon service ==
openstack-dashboard:                    active
== neutron services ==
neutron-server:                         active
neutron-dhcp-agent:                     active
neutron-l3-agent:                       active
neutron-metadata-agent:                 active
neutron-lbaas-agent:                    inactive  (disabled on boot)
neutron-openvswitch-agent:              active
neutron-linuxbridge-agent:              inactive  (disabled on boot)
neutron-ryu-agent:                      inactive  (disabled on boot)
neutron-nec-agent:                      inactive  (disabled on boot)
neutron-mlnx-agent:                     inactive  (disabled on boot)
== Swift services ==
openstack-swift-proxy:                  active
openstack-swift-account:                active
openstack-swift-container:              active
openstack-swift-object:                 active
== Cinder services ==
openstack-cinder-api:                   active
openstack-cinder-scheduler:             active
openstack-cinder-volume:                active
openstack-cinder-backup:                active
== Ceilometer services ==
openstack-ceilometer-api:               active
openstack-ceilometer-central:           active
openstack-ceilometer-compute:           active
openstack-ceilometer-collector:         active
openstack-ceilometer-alarm-notifier:    active
openstack-ceilometer-alarm-evaluator:   active
== Support services ==
libvirtd:                               active
openvswitch:                            active
dbus:                                   active
tgtd:                                   active
rabbitmq-server:                        active
memcached:                              active
== Keystone users ==
+----------------------------------+------------+---------+----------------------+
|                id                |    name    | enabled |        email         |
+----------------------------------+------------+---------+----------------------+
| 6d93d08727ec470b9da2d280f5f93dce |   admin    |   True  |    test@test.com     |
| 666156e6053b479e8ee851abb4a2b8fd | ceilometer |   True  | ceilometer@localhost |
| cc47637169e7476fb9a39246e66b6f5d |   cinder   |   True  |   cinder@localhost   |
| 7e4f2dc81c5a467a851ea2b529ac4d3f |   glance   |   True  |   glance@localhost   |
| c2d6010d369b4373a2b17bc52dcc4063 |  neutron   |   True  |  neutron@localhost   |
| b7efe2f71f17460997c178a5ed7a562a |    nova    |   True  |    nova@localhost    |
| 81a7e484a6c742efb529e9221db539ef |   swift    |   True  |   swift@localhost    |
+----------------------------------+------------+---------+----------------------+
== Glance images ==
+--------------------------------------+---------------+-------------+------------------+----------+--------+
| ID                                   | Name          | Disk Format | Container Format | Size     | Status |
+--------------------------------------+---------------+-------------+------------------+----------+--------+
| de8d1e90-ec92-4113-b2c3-2206950b5481 | CirrOS31image | qcow2       | bare             | 13147648 | active |
+--------------------------------------+---------------+-------------+------------------+----------+--------+
== Nova managed services ==
+------------------+----------------------------------------+----------+---------+-------+----------------------------+-----------------+
| Binary           | Host                                   | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+------------------+----------------------------------------+----------+---------+-------+----------------------------+-----------------+
| nova-consoleauth | ip-192-169-142-127.ip.secureserver.net | internal | enabled | up    | 2014-05-22T14:06:06.000000 | -               |
| nova-scheduler   | ip-192-169-142-127.ip.secureserver.net | internal | enabled | up    | 2014-05-22T14:06:06.000000 | -               |
| nova-conductor   | ip-192-169-142-127.ip.secureserver.net | internal | enabled | up    | 2014-05-22T14:06:06.000000 | -               |
| nova-compute     | ip-192-169-142-127.ip.secureserver.net | nova     | enabled | up    | 2014-05-22T14:06:06.000000 | -               |
| nova-cert        | ip-192-169-142-127.ip.secureserver.net | internal | enabled | up    | 2014-05-22T14:06:06.000000 | -               |
| nova-compute     | ip-192-169-142-137.ip.secureserver.net | nova     | enabled | up    | 2014-05-22T14:06:07.000000 | -               |
+------------------+----------------------------------------+----------+---------+-------+----------------------------+-----------------+
== Nova networks ==
+--------------------------------------+---------+------+
| ID                                   | Label   | Cidr |
+--------------------------------------+---------+------+
| b1fa9d0d-346d-4ef9-8735-8ce00d82e036 | public  | -    |
| da3b969d-e11e-4d85-ad33-6eb64d5531fc | private | -    |
+--------------------------------------+---------+------+
== Nova instance flavors ==
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| ID | Name      | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| 1  | m1.tiny   | 512       | 1    | 0         |      | 1     | 1.0         | True      |
| 2  | m1.small  | 2048      | 20   | 0         |      | 1     | 1.0         | True      |
| 3  | m1.medium | 4096      | 40   | 0         |      | 2     | 1.0         | True      |
| 4  | m1.large  | 8192      | 80   | 0         |      | 4     | 1.0         | True      |
| 5  | m1.xlarge | 16384     | 160  | 0         |      | 8     | 1.0         | True      |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
== Nova instances ==
+--------------------------------------+----------+-----------+------------+-------------+------------------------------------+
| ID                                   | Name     | Status    | Task State | Power State | Networks                           |
+--------------------------------------+----------+-----------+------------+-------------+------------------------------------+
| 21541af7-f2b7-4e63-bc02-960dba062330 | CirrOS31 | SUSPENDED | -          | Shutdown    | private=10.0.0.10, 192.169.142.151 |
| d4dc868d-1316-4ac0-b189-f84a9445ef51 | CirrOS35 | ACTIVE    | -          | Running     | private=10.0.0.12, 192.169.142.152 |
+--------------------------------------+----------+-----------+------------+-------------+------------------------------------+

[root@ip-192-169-142-127 ~(keystone_admin)]# nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At
nova-consoleauth ip-192-169-142-127.ip.secureserver.net internal         enabled    :-)   2014-05-22 14:06:26
nova-scheduler   ip-192-169-142-127.ip.secureserver.net internal         enabled    :-)   2014-05-22 14:06:26
nova-conductor   ip-192-169-142-127.ip.secureserver.net internal         enabled    :-)   2014-05-22 14:06:26
nova-compute     ip-192-169-142-127.ip.secureserver.net nova             enabled    :-)   2014-05-22 14:06:26
nova-cert        ip-192-169-142-127.ip.secureserver.net internal         enabled    :-)   2014-05-22 14:06:26
nova-compute     ip-192-169-142-137.ip.secureserver.net nova             enabled    :-)   2014-05-22 14:06:27

[root@ip-192-169-142-127 ~(keystone_admin)]# neutron agent-list
+--------------------------------------+--------------------+----------------------------------------+-------+----------------+
| id                                   | agent_type         | host                                   | alive | admin_state_up |
+--------------------------------------+--------------------+----------------------------------------+-------+----------------+
| 4cebaab8-62a8-4513-8076-39ea55e57cdd | Metadata agent     | ip-192-169-142-127.ip.secureserver.net | :-)   | True           |
| 556b5afd-ac63-45d9-9001-7d2bee997f18 | Open vSwitch agent | ip-192-169-142-137.ip.secureserver.net | :-)   | True           |
| 79c9f9b0-91e3-4201-8b98-9eeab78f5bbc | DHCP agent         | ip-192-169-142-127.ip.secureserver.net | :-)   | True           |
| b0688bbe-9ab6-44d7-83f4-3fbfd6da1003 | Open vSwitch agent | ip-192-169-142-127.ip.secureserver.net | :-)   | True           |
| dd9c8a5e-2876-4f79-b06e-04a979a5fc87 | L3 agent           | ip-192-169-142-127.ip.secureserver.net | :-)   | True           |
+--------------------------------------+--------------------+----------------------------------------+-------+----------------+

*********************
On Controller :-
*********************

[root@ip-192-169-142-127 ~(keystone_admin)]# ovs-vsctl show
3f6c4c61-56d9-4436-84ca-99c1289d7644
    Bridge "br-eth1"
        Port "eth1"
            Interface "eth1"
        Port "phy-br-eth1"
            Interface "phy-br-eth1"
        Port "br-eth1"
            Interface "br-eth1"
                type: internal
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port "qg-8f154600-13"
            Interface "qg-8f154600-13"
                type: internal
        Port "eth0"
            Interface "eth0"
    Bridge br-int
        Port "qr-053f82ff-6d"
            tag: 1
            Interface "qr-053f82ff-6d"
                type: internal
        Port br-int
            Interface br-int
                type: internal
        Port "tapb7744a3b-05"
            tag: 1
            Interface "tapb7744a3b-05"
                type: internal
        Port "int-br-eth1"
            Interface "int-br-eth1"
        Port "qvo70f4f21a-34"
            tag: 1
            Interface "qvo70f4f21a-34"
    ovs_version: "2.0.1"

*******************
On Compute:-
*******************

[root@ip-192-169-142-137 ~]# ovs-vsctl show
1ddf47f0-88f4-4df1-ba6c-c73666e90e56
    Bridge br-int
        Port br-int
            Interface br-int
                type: internal
        Port "qvoc3fd92d1-1e"
            tag: 1
            Interface "qvoc3fd92d1-1e"
        Port "int-br-eth1"
            Interface "int-br-eth1"
    Bridge "br-eth1"
        Port "br-eth1"
            Interface "br-eth1"
                type: internal
        Port "eth1"
            Interface "eth1"
        Port "phy-br-eth1"
            Interface "phy-br-eth1"
    ovs_version: "2.0.1"

[root@ip-192-169-142-137 ~]# brctl show
bridge name    bridge id        STP enabled    interfaces
qbrc3fd92d1-1e        8000.aee2ccfcb2bd    no        qvbc3fd92d1-1e
                            tapc3fd92d1-1e












Login to Ubuntu VM running at Compute Node :-

[root@ip-192-169-142-127 ~]# ssh -l ubuntu -i oskey25.pem 192.169.142.153
Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0-24-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

 System information disabled due to load higher than 1.0

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

0 packages can be updated.
0 updates are security updates.


Last login: Fri May 23 03:01:11 2014 from ip-192-169-142-127.ip.secureserver.net


  

  

   Ubuntu VM deployed on Compute Node 192.169.1.137 :- 
  


Running links on VM