Xen Virtualization on Linux and Solaris: May 2014

Saturday, May 24, 2014

Two Real Node (Controller+Compute) RDO IceHouse Neutron OVS&VLAN Cluster on Fedora 20 Setup

Two boxes , each one having 2 NICs (p37p1,p4p1) for (Controller+NeutronServer) && Compute Nodes have been setup.

Setup configuration

- Controller node: Nova, Keystone, Cinder, Glance, Neutron (using Open vSwitch plugin && VLAN )
- Compute node: Nova (nova-compute), Neutron (openvswitch-agent)

icehouse1.localdomain   - Controller (192.168.1.127)
icehouse2.localdomain   - Compute   (192.168.1.137)

Before running `packstack --answer-file=TwoRealNode-answer.txt` SELINUX set to permissive on both nodes. Interfaces p4p1 on both nodes set to promiscuous mode (e.g. HWADDRESS was commented out).

Specific of answer-file on real F20 boxes :-

CONFIG_NOVA_COMPUTE_PRIVIF=p4p1
CONFIG_NOVA_NETWORK_PUBIF=p37p1
CONFIG_NOVA_NETWORK_PRIVIF=p4p1

CONFIG_NEUTRON_OVS_TENANT_NETWORK_TYPE=vlan
CONFIG_NEUTRON_OVS_VLAN_RANGES=physnet1:100:200
CONFIG_NEUTRON_OVS_BRIDGE_MAPPINGS=physnet1:br-p4p1
CONFIG_NEUTRON_OVS_BRIDGE_IFACES=br-p4p1:p4p1

Post installation steps :-

1. NetworkManager should be disabled on both nodes, service network enabled.
2. Syntax of ifcfg-* files of corresponding OVS ports should follow RHEL 6.5 notations rather then F20
3. Special care should be taken to bring up p4p1 (in my case)
4. Post install reconfiguration *.ini && *.conf http://textuploader.com/9oec
5. Configuration p4p1 interfaces

[root@icehouse1 network-scripts(keystone_admin)]# cat ifcfg-p4p1
TYPE=Ethernet
BOOTPROTO=none
DEVICE=p4p1
ONBOOT=yes
NM_CONTROLLED=no

[root@icehouse1 network-scripts(keystone_admin)]# cat ifcfg-br-ex
DEVICE="br-ex"
BOOTPROTO="static"
IPADDR="192.168.1.127"
NETMASK="255.255.255.0"
DNS1="83.221.202.254"
BROADCAST="192.168.1.255"
GATEWAY="192.168.1.1"
NM_CONTROLLED="no"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="yes"
IPV6INIT=no
ONBOOT="yes"
TYPE="OVSBridge"
DEVICETYPE="ovs"

[root@icehouse1 network-scripts(keystone_admin)]# cat ifcfg-p37p1
DEVICE="p37p1"
ONBOOT="yes"
TYPE="OVSPort"
DEVICETYPE="ovs"
OVS_BRIDGE=br-ex
NM_CONTROLLED=no
IPV6INIT=no

[root@icehouse1 ~(keystone_admin)]# ovs-vsctl add-port br-ex p37p1

Metadata access verification on Controller:-

[root@icehouse1 ~(keystone_admin)]# ip netns
qdhcp-a2bf6363-6447-47f5-a243-b998d206d593
qrouter-2462467b-ea0a-4a40-a093-493572010694

[root@icehouse1 ~(keystone_admin)]# ip netns exec qrouter-2462467b-ea0a-4a40-a093-493572010694 \
> iptables -S -t nat | grep 169

-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8775

[root@icehouse1 ~(keystone_admin)]# ip netns exec qrouter-2462467b-ea0a-4a40-a093-493572010694 \
> netstat -anpt
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:8775            0.0.0.0:*               LISTEN      6156/python

[root@icehouse1 ~(keystone_admin)]# ps -ef | grep 6156

root      5691 4082 0 07:58 pts/0    00:00:00 grep --color=auto 6156
root      6156     1 0 06:04 ?        00:00:00 /usr/bin/python /bin/neutron-ns-metadata-proxy --pid_file=/var/lib/neutron/external/pids/2462467b-ea0a-4a40-a093-493572010694.pid --metadata_proxy_socket=/var/lib/neutron/metadata_proxy --router_id=2462467b-ea0a-4a40-a093-493572010694 --state_path=/var/lib/neutron --metadata_port=8775 --verbose --log-file=neutron-ns-metadata-proxy-2462467b-ea0a-4a40-a093-493572010694.log --log-dir=/var/log/neutron

[root@icehouse1 ~(keystone_admin)]# netstat -anpt | grep 8775
tcp        0      0 0.0.0.0:8775            0.0.0.0:*               LISTEN      1224/python

[root@icehouse1 ~(keystone_admin)]# ps -aux | grep 1224
nova      1224 0.7 0.7 337092 65052 ?        Ss   05:59   0:46 /usr/bin/python /usr/bin/nova-api
boris     3789 0.0 0.1 504676 12248 ?        Sl   06:01   0:00 /usr/libexec/tracker-store

Verifying dhcp lease for private IPs for instances currently running :-

[root@icehouse1 ~(keystone_admin)]# ip netns exec qdhcp-a2bf6363-6447-47f5-a243-b998d206d593 ifconfig

lo: flags=73 mtu 65536
        inet 127.0.0.1 netmask 255.0.0.0
        inet6 ::1 prefixlen 128 scopeid 0x10
        loop txqueuelen 0 (Local Loopback)
        RX packets 3 bytes 1728 (1.6 KiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 3 bytes 1728 (1.6 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

tapa7e1ac48-7b: flags=67 mtu 1500
        inet 10.0.0.11 netmask 255.255.255.0 broadcast 10.0.0.255
        inet6 fe80::f816:3eff:fe9d:874d prefixlen 64 scopeid 0x20
        ether fa:16:3e:9d:87:4d txqueuelen 0 (Ethernet)
        RX packets 3364 bytes 626074 (611.4 KiB)
        RX errors 0 dropped 35 overruns 0 frame 0
        TX packets 2124 bytes 427060 (417.0 KiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

[root@icehouse1 ~(keystone_admin)]# ip netns exec qdhcp-a2bf6363-6447-47f5-a243-b998d206d593 tcpdump -ln -i tapa7e1ac48-7b

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tapa7e1ac48-7b, link-type EN10MB (Ethernet), capture size 65535 bytes
11:07:02.388376 ARP, Request who-has 10.0.0.11 tell 10.0.0.38, length 46
11:07:02.388399 ARP, Reply 10.0.0.11 is-at fa:16:3e:9d:87:4d, length 28
11:07:12.239833 IP 10.0.0.43.bootpc > 10.0.0.11.bootps: BOOTP/DHCP, Request from fa:16:3e:40:da:a1, length 300
11:07:12.240491 IP 10.0.0.11.bootps > 10.0.0.43.bootpc: BOOTP/DHCP, Reply, length 324
11:07:12.313087 ARP, Request who-has 10.0.0.43 (Broadcast) tell 0.0.0.0, length 46
11:07:13.313070 ARP, Request who-has 10.0.0.43 (Broadcast) tell 0.0.0.0, length 46
11:07:15.634980 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:6b:81:ff, length 280
11:07:15.635595 IP 10.0.0.11.bootps > 10.0.0.31.bootpc: BOOTP/DHCP, Reply, length 324
11:07:15.635954 IP 10.0.0.31 > 10.0.0.11: ICMP 10.0.0.31 udp port bootpc unreachable, length 360
11:07:17.254260 ARP, Request who-has 10.0.0.43 tell 10.0.0.11, length 28
11:07:17.254866 ARP, Reply 10.0.0.43 is-at fa:16:3e:40:da:a1, length 46
11:07:20.644135 ARP, Request who-has 10.0.0.11 tell 10.0.0.31, length 28
11:07:20.644157 ARP, Reply 10.0.0.11 is-at fa:16:3e:9d:87:4d, length 28
11:07:45.972179 IP 10.0.0.38.bootpc > 10.0.0.11.bootps: BOOTP/DHCP, Request from fa:16:3e:9d:67:df, length 300
11:07:45.973023 IP 10.0.0.11.bootps > 10.0.0.38.bootpc: BOOTP/DHCP, Reply, length 324
11:07:50.980701 ARP, Request who-has 10.0.0.11 tell 10.0.0.38, length 46
11:07:50.980725 ARP, Reply 10.0.0.11 is-at fa:16:3e:9d:87:4d, length 28
11:07:55.821920 IP 10.0.0.43.bootpc > 10.0.0.11.bootps: BOOTP/DHCP, Request from fa:16:3e:40:da:a1, length 300
11:07:55.822423 IP 10.0.0.11.bootps > 10.0.0.43.bootpc: BOOTP/DHCP, Reply, length 324
11:07:55.898024 ARP, Request who-has 10.0.0.43 (Broadcast) tell 0.0.0.0, length 46
11:07:56.897994 ARP, Request who-has 10.0.0.43 (Broadcast) tell 0.0.0.0, length 46
11:08:00.823637 ARP, Request who-has 10.0.0.11 tell 10.0.0.43, length 46

******************
On Controller
******************
[root@icehouse1 ~(keystone_admin)]# ovs-vsctl show
a675c73e-c707-4f29-af60-57fb7c3f81c4
    Bridge br-int
        Port "int-br-p4p1"
            Interface "int-br-p4p1"
        Port br-int
            Interface br-int
                type: internal
        Port "qr-bbba6fd3-a3"
            tag: 1
            Interface "qr-bbba6fd3-a3"
                type: internal
        Port "qvo61d82a0f-32"
            tag: 1
            Interface "qvo61d82a0f-32"
        Port "tapa7e1ac48-7b"
            tag: 1
            Interface "tapa7e1ac48-7b"
                type: internal
        Port "qvof8c8a1a2-51"
            tag: 1
            Interface "qvof8c8a1a2-51"
    Bridge br-ex
        Port "p37p1"
            Interface "p37p1"
        Port br-ex
            Interface br-ex
                type: internal
        Port "qg-3787602d-29"
            Interface "qg-3787602d-29"
                type: internal
    Bridge "br-p4p1"
        Port "p4p1"
            Interface "p4p1"
        Port "phy-br-p4p1"
            Interface "phy-br-p4p1"
        Port "br-p4p1"
            Interface "br-p4p1"
                type: internal
    ovs_version: "2.0.1"

****************
On Compute
****************

[root@icehouse2 ]# ovs-vsctl show
bf768fc8-d18b-4762-bdd2-a410fcf88a9b
    Bridge "br-p4p1"
        Port "br-p4p1"
            Interface "br-p4p1"
                type: internal
        Port "phy-br-p4p1"
            Interface "phy-br-p4p1"
        Port "p4p1"
            Interface "p4p1"
    Bridge br-int
        Port br-int
            Interface br-int
                type: internal
        Port "int-br-p4p1"
            Interface "int-br-p4p1"
        Port "qvoe5a82d77-d4"
            tag: 8
            Interface "qvoe5a82d77-d4"
    ovs_version: "2.0.1"

[root@icehouse1 ~(keystone_admin)]# openstack-status
== Nova services ==
openstack-nova-api:                     active
openstack-nova-cert:                    active
openstack-nova-compute:                 active
openstack-nova-network:                 inactive (disabled on boot)
openstack-nova-scheduler:               active
openstack-nova-volume:                  inactive (disabled on boot)
openstack-nova-conductor:               active
== Glance services ==
openstack-glance-api:                   active
openstack-glance-registry:              active
== Keystone service ==
openstack-keystone:                     active
== Horizon service ==
openstack-dashboard:                    active
== neutron services ==
neutron-server:                         active
neutron-dhcp-agent:                     active
neutron-l3-agent:                       active
neutron-metadata-agent:                 active
neutron-lbaas-agent:                    inactive (disabled on boot)
neutron-openvswitch-agent:              active
neutron-linuxbridge-agent:              inactive (disabled on boot)
neutron-ryu-agent:                      inactive (disabled on boot)
neutron-nec-agent:                      inactive (disabled on boot)
neutron-mlnx-agent:                     inactive (disabled on boot)
== Swift services ==
openstack-swift-proxy:                  active
openstack-swift-account:                active
openstack-swift-container:              active
openstack-swift-object:                 active
== Cinder services ==
openstack-cinder-api:                   active
openstack-cinder-scheduler:             active
openstack-cinder-volume:                active
openstack-cinder-backup:                inactive
== Ceilometer services ==
openstack-ceilometer-api:               active
openstack-ceilometer-central:           active
openstack-ceilometer-compute:           active
openstack-ceilometer-collector:         active
openstack-ceilometer-alarm-notifier:    active
openstack-ceilometer-alarm-evaluator:   active
== Support services ==
libvirtd:                               active
openvswitch:                            active
dbus:                                   active
tgtd:                                   active
rabbitmq-server:                        active
memcached:                              active
== Keystone users ==
+----------------------------------+------------+---------+----------------------+
|                id                |    name    | enabled |        email         |
+----------------------------------+------------+---------+----------------------+
| df9165cd160846b19f73491e0bc041c2 |   admin    |   True |    test@test.com     |
| bafe2fc4d51a400a99b1b41ef50d4afd | ceilometer |   True | ceilometer@localhost |
| df59d0782f174a34a3a73215300c64ca |   cinder   |   True |   cinder@localhost   |
| ca624394c9d941b6ad0a07363ab668b2 |   glance   |   True |   glance@localhost   |
| fb5125484a1f4b7aaf8503025eb018ba | neutron   |   True | neutron@localhost   |
| 64912bc3726c48db8f003ce79d8fe746 |    nova    |   True |    nova@localhost    |
| 6d8b48605d3b476097d89486813360c0 |   swift    |   True |   swift@localhost    |
+----------------------------------+------------+---------+----------------------+
== Glance images ==
+--------------------------------------+-----------------+-------------+------------------+-----------+--------+
| ID                                   | Name            | Disk Format | Container Format | Size      | Status |
+--------------------------------------+-----------------+-------------+------------------+-----------+--------+
| 8593a43a-2449-4b49-918f-9871011249a7 | CirrOS31        | qcow2       | bare             | 13147648 | active |
| 4be72a99-06e0-477d-b446-b597435455a9 | Fedora20image   | qcow2       | bare             | 210829312 | active |
| 28470072-f317-4a72-b3e8-3fffbe6a7661 | UubuntuServer14 | qcow2       | bare             | 253559296 | active |
+--------------------------------------+-----------------+-------------+------------------+-----------+--------+
== Nova managed services ==
+------------------+-----------------------+----------+---------+-------+----------------------------+-----------------+
| Binary           | Host                  | Zone     | Status | State | Updated_at                 | Disabled Reason |
+------------------+-----------------------+----------+---------+-------+----------------------------+-----------------+
| nova-consoleauth | icehouse1.localdomain | internal | enabled | up    | 2014-05-25T03:03:05.000000 | -               |
| nova-scheduler   | icehouse1.localdomain | internal | enabled | up    | 2014-05-25T03:03:05.000000 | -               |
| nova-conductor   | icehouse1.localdomain | internal | enabled | up    | 2014-05-25T03:03:13.000000 | -               |
| nova-compute     | icehouse1.localdomain | nova     | enabled | up    | 2014-05-25T03:03:10.000000 | -               |
| nova-cert        | icehouse1.localdomain | internal | enabled | up    | 2014-05-25T03:03:05.000000 | -               |
| nova-compute     | icehouse2.localdomain | nova     | enabled | up    | 2014-05-25T03:03:13.000000 | -               |
+------------------+-----------------------+----------+---------+-------+----------------------------+-----------------+
== Nova networks ==
+--------------------------------------+---------+------+
| ID                                   | Label   | Cidr |
+--------------------------------------+---------+------+
| 09e18ced-8c22-4166-a1a1-cbceece46884 | public | -    |
| a2bf6363-6447-47f5-a243-b998d206d593 | private | -    |
+--------------------------------------+---------+------+
== Nova instance flavors ==
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| ID | Name      | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| 1 | m1.tiny   | 512       | 1    | 0         |      | 1     | 1.0         | True      |
| 2 | m1.small | 2048      | 20   | 0         |      | 1     | 1.0         | True      |
| 3 | m1.medium | 4096      | 40   | 0         |      | 2     | 1.0         | True      |
| 4 | m1.large | 8192      | 80   | 0         |      | 4     | 1.0         | True      |
| 5 | m1.xlarge | 16384     | 160 | 0         |      | 8     | 1.0         | True      |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
== Nova instances ==
+--------------------------------------+--------------+-----------+------------+-------------+---------------------------------+
| ID                                   | Name         | Status    | Task State | Power State | Networks                        |
+--------------------------------------+--------------+-----------+------------+-------------+---------------------------------+
| b661a130-fdb7-41eb-aba5-588924634c9d | CirrOS302    | ACTIVE    | -          | Running     | private=10.0.0.31, 192.168.1.63 |
| 5d1dbb9d-7bef-4e51-be8d-4270ddd3d4cc | CirrOS351    | ACTIVE    | -          | Running     | private=10.0.0.39, 192.168.1.66 |
| ef73a897-8700-4999-ab25-49f25b896f34 | CirrOS370    | ACTIVE    | -          | Running     | private=10.0.0.40, 192.168.1.69 |
| 02718e21-edb9-4b59-8bb7-21e0290650fd | CirrOS390    | SUSPENDED | -          | Shutdown    | private=10.0.0.41, 192.168.1.67 |                           |
| 6992e37c-48c7-49b6-b6fc-8e35fe240704 | UbuntuSRV350 | SUSPENDED | -          | Shutdown    | private=10.0.0.38, 192.168.1.62 |
| 9953ed52-b666-4fe1-ac35-23621122af5a | VF20RS02     | ACTIVE    | -          | Running     | private=10.0.0.43, 192.168.1.71 |
+--------------------------------------+--------------+-----------+------------+-------------+---------------------------------+

[root@icehouse1 ~(keystone_admin)]# nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At
nova-consoleauth icehouse1.localdomain                internal         enabled    :-)   2014-05-25 03:05:25
nova-scheduler   icehouse1.localdomain                internal         enabled    :-)   2014-05-25 03:05:25
nova-conductor   icehouse1.localdomain                internal         enabled    :-)   2014-05-25 03:05:23
nova-compute     icehouse1.localdomain                nova             enabled    :-)   2014-05-25 03:05:20
nova-cert        icehouse1.localdomain                internal         enabled    :-)   2014-05-25 03:05:25
nova-compute     icehouse2.localdomain                nova             enabled    :-)   2014-05-25 03:05:23

[root@icehouse1 ~(keystone_admin)]# neutron agent-list
+--------------------------------------+--------------------+-----------------------+-------+----------------+
| id                                   | agent_type         | host                  | alive | admin_state_up |
+--------------------------------------+--------------------+-----------------------+-------+----------------+
| 6775fac7-d594-4272-8447-f136b54247e8 | L3 agent           | icehouse1.localdomain | :-)   | True           |
| 77fdc8a9-0d77-4f53-9cdd-1c732f0cfdb1 | Metadata agent     | icehouse1.localdomain | :-)   | True           |
| 8f70b2c4-c65b-4d0b-9808-ba494c764d99 | Open vSwitch agent | icehouse1.localdomain | :-)   | True           |
| a86f1272-2afb-43b5-a7e6-e5fc6df565b5 | Open vSwitch agent | icehouse2.localdomain | :-)   | True           |
| e72bdcd5-3dd1-4994-860f-e21d4a58dd4c | DHCP agent         | icehouse1.localdomain | :-)   | True           |
+--------------------------------------+--------------------+-----------------------+-------+----------------+

 Windows 2012 evaluation Server running on Compute Node :-

Thursday, May 22, 2014

Two Node (Controller+Compute) IceHouse Neutron OVS&VLAN Cluster on Fedora 20

Two KVMs have been created , each one having 2 virtual NICs (eth0,eth1) for
Controller && Compute Nodes setup. Before running `packstack --answer-file=twoNode-answer.txt` SELINUX set to permissive on both nodes. Interfaces eth1 on both nodes set to promiscuous mode (e.g. HWADDRESS was commented out). Testing VMs network performance on Compute Node , I had to disable checksum offloading on eth1 on Compute :-

# /sbin/ethtool --offload eth1 tx off

In case of further problems with network, switch to setup on non-default Libvirt's subnet ( for instance 192.169.142.0/24).

Creating non-default Libvirt subnet per
http://kashyapc.fedorapeople.org/virt/create-a-new-libvirt-bridge.txt

1. Create a new libvirt network (other than your default 198.162.x.x) file:

$ cat openstackvms.xml 
 <network>
   <name>openstackvms</name>
   <uuid>d0e9964a-f91a-40c0-b769-a609aee41bf2</uuid>
   <forward mode='nat'>
     <nat>
       <port start='1024' end='65535'/>
     </nat>
   </forward>
   <bridge name='virbr1' stp='on' delay='0' />
   <mac address='52:54:00:60:f8:6e'/>
   <ip address='192.169.142.1' netmask='255.255.255.0'>
     <dhcp>
       <range start='192.169.142.2' end='192.169.142.254' />
     </dhcp>
   </ip>
 </network>
      
2. Define the above network:

  $ virsh net-define openstackvms.xml


3. Start the network and enable it for "autostart"

  $ virsh net-start openstackvms
  $ virsh net-autostart openstackvms


4. List your libvirt networks to see if it reflects:

  $ virsh net-list
  Name                 State      Autostart     Persistent
  ----------------------------------------------------------
  default              active     yes           yes
  openstackvms         active     yes           yes


5. Optionally, list your bridge devices:

  $ brctl show
  bridge name     bridge id               STP enabled     interfaces
  virbr0          8000.5254003339b3       yes             virbr0-nic
  virbr1          8000.52540060f86e       yes             virbr1-nic

Link for download is here Answer-file

After `packstack --answer-file=twoNode-answer.txt` successful run.

OVS bridge br-ex && OVS port eth0 created manually

[root@ip-192-169-142-127 ~]# cat ifcfg-br-ex
DEVICE="br-ex"
BOOTPROTO="static"
IPADDR="192.169.142.127"
NETMASK="255.255.255.0"
DNS1="83.221.202.254"
BROADCAST="192.169.142.255"
GATEWAY="192.169.142.1"
NM_CONTROLLED="no"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="yes"
IPV6INIT=no
ONBOOT="yes"
TYPE="OVSBridge"
DEVICETYPE="ovs"

[root@ip-192-169-142-127 ~]# cat ifcfg-eth0
NAME="eth0"
ONBOOT="yes"
TYPE="OVSPort"
DEVICETYPE="ovs"
OVS_BRIDGE=br-ex
NM_CONTROLLED=no
IPV6INIT=no

[root@ip-192-169-142-127 ~]# ovs-vsctl add-port br-ex eth0

Service NetworkManager disabled, service network enabled.
System restarted with /etc/rc.d/rc.local :-

ifdown br-ex;
ifup br-ex ;

[root@ip-192-169-142-127 ~(keystone_admin)]# openstack-status
== Nova services ==
openstack-nova-api:                     active
openstack-nova-cert:                    active
openstack-nova-compute:                 active
openstack-nova-network:                 inactive (disabled on boot)
openstack-nova-scheduler:               active
openstack-nova-volume:                  inactive (disabled on boot)
openstack-nova-conductor:               active
== Glance services ==
openstack-glance-api:                   active
openstack-glance-registry:              active
== Keystone service ==
openstack-keystone:                     active
== Horizon service ==
openstack-dashboard:                    active
== neutron services ==
neutron-server:                         active
neutron-dhcp-agent:                     active
neutron-l3-agent:                       active
neutron-metadata-agent:                 active
neutron-lbaas-agent:                    inactive (disabled on boot)
neutron-openvswitch-agent:              active
neutron-linuxbridge-agent:              inactive (disabled on boot)
neutron-ryu-agent:                      inactive (disabled on boot)
neutron-nec-agent:                      inactive (disabled on boot)
neutron-mlnx-agent:                     inactive (disabled on boot)
== Swift services ==
openstack-swift-proxy:                  active
openstack-swift-account:                active
openstack-swift-container:              active
openstack-swift-object:                 active
== Cinder services ==
openstack-cinder-api:                   active
openstack-cinder-scheduler:             active
openstack-cinder-volume:                active
openstack-cinder-backup:                active
== Ceilometer services ==
openstack-ceilometer-api:               active
openstack-ceilometer-central:           active
openstack-ceilometer-compute:           active
openstack-ceilometer-collector:         active
openstack-ceilometer-alarm-notifier:    active
openstack-ceilometer-alarm-evaluator:   active
== Support services ==
libvirtd:                               active
openvswitch:                            active
dbus:                                   active
tgtd:                                   active
rabbitmq-server:                        active
memcached:                              active
== Keystone users ==
+----------------------------------+------------+---------+----------------------+
|                id                |    name    | enabled |        email         |
+----------------------------------+------------+---------+----------------------+
| 6d93d08727ec470b9da2d280f5f93dce |   admin    |   True |    test@test.com     |
| 666156e6053b479e8ee851abb4a2b8fd | ceilometer |   True | ceilometer@localhost |
| cc47637169e7476fb9a39246e66b6f5d |   cinder   |   True |   cinder@localhost   |
| 7e4f2dc81c5a467a851ea2b529ac4d3f |   glance   |   True |   glance@localhost   |
| c2d6010d369b4373a2b17bc52dcc4063 | neutron   |   True | neutron@localhost   |
| b7efe2f71f17460997c178a5ed7a562a |    nova    |   True |    nova@localhost    |
| 81a7e484a6c742efb529e9221db539ef |   swift    |   True |   swift@localhost    |
+----------------------------------+------------+---------+----------------------+
== Glance images ==
+--------------------------------------+---------------+-------------+------------------+----------+--------+
| ID                                   | Name          | Disk Format | Container Format | Size     | Status |
+--------------------------------------+---------------+-------------+------------------+----------+--------+
| de8d1e90-ec92-4113-b2c3-2206950b5481 | CirrOS31image | qcow2       | bare             | 13147648 | active |
+--------------------------------------+---------------+-------------+------------------+----------+--------+
== Nova managed services ==
+------------------+----------------------------------------+----------+---------+-------+----------------------------+-----------------+
| Binary           | Host                                   | Zone     | Status | State | Updated_at                 | Disabled Reason |
+------------------+----------------------------------------+----------+---------+-------+----------------------------+-----------------+
| nova-consoleauth | ip-192-169-142-127.ip.secureserver.net | internal | enabled | up    | 2014-05-22T14:06:06.000000 | -               |
| nova-scheduler   | ip-192-169-142-127.ip.secureserver.net | internal | enabled | up    | 2014-05-22T14:06:06.000000 | -               |
| nova-conductor   | ip-192-169-142-127.ip.secureserver.net | internal | enabled | up    | 2014-05-22T14:06:06.000000 | -               |
| nova-compute     | ip-192-169-142-127.ip.secureserver.net | nova     | enabled | up    | 2014-05-22T14:06:06.000000 | -               |
| nova-cert        | ip-192-169-142-127.ip.secureserver.net | internal | enabled | up    | 2014-05-22T14:06:06.000000 | -               |
| nova-compute     | ip-192-169-142-137.ip.secureserver.net | nova     | enabled | up    | 2014-05-22T14:06:07.000000 | -               |
+------------------+----------------------------------------+----------+---------+-------+----------------------------+-----------------+
== Nova networks ==
+--------------------------------------+---------+------+
| ID                                   | Label   | Cidr |
+--------------------------------------+---------+------+
| b1fa9d0d-346d-4ef9-8735-8ce00d82e036 | public | -    |
| da3b969d-e11e-4d85-ad33-6eb64d5531fc | private | -    |
+--------------------------------------+---------+------+
== Nova instance flavors ==
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| ID | Name      | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| 1 | m1.tiny   | 512       | 1    | 0         |      | 1     | 1.0         | True      |
| 2 | m1.small | 2048      | 20   | 0         |      | 1     | 1.0         | True      |
| 3 | m1.medium | 4096      | 40   | 0         |      | 2     | 1.0         | True      |
| 4 | m1.large | 8192      | 80   | 0         |      | 4     | 1.0         | True      |
| 5 | m1.xlarge | 16384     | 160 | 0         |      | 8     | 1.0         | True      |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
== Nova instances ==
+--------------------------------------+----------+-----------+------------+-------------+------------------------------------+
| ID                                   | Name     | Status    | Task State | Power State | Networks                           |
+--------------------------------------+----------+-----------+------------+-------------+------------------------------------+
| 21541af7-f2b7-4e63-bc02-960dba062330 | CirrOS31 | SUSPENDED | -          | Shutdown    | private=10.0.0.10, 192.169.142.151 |
| d4dc868d-1316-4ac0-b189-f84a9445ef51 | CirrOS35 | ACTIVE    | -          | Running     | private=10.0.0.12, 192.169.142.152 |
+--------------------------------------+----------+-----------+------------+-------------+------------------------------------+

[root@ip-192-169-142-127 ~(keystone_admin)]# nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At
nova-consoleauth ip-192-169-142-127.ip.secureserver.net internal         enabled    :-)   2014-05-22 14:06:26
nova-scheduler   ip-192-169-142-127.ip.secureserver.net internal         enabled    :-)   2014-05-22 14:06:26
nova-conductor   ip-192-169-142-127.ip.secureserver.net internal         enabled    :-)   2014-05-22 14:06:26
nova-compute     ip-192-169-142-127.ip.secureserver.net nova             enabled    :-)   2014-05-22 14:06:26
nova-cert        ip-192-169-142-127.ip.secureserver.net internal         enabled    :-)   2014-05-22 14:06:26
nova-compute     ip-192-169-142-137.ip.secureserver.net nova             enabled    :-)   2014-05-22 14:06:27

[root@ip-192-169-142-127 ~(keystone_admin)]# neutron agent-list
+--------------------------------------+--------------------+----------------------------------------+-------+----------------+
| id                                   | agent_type         | host                                   | alive | admin_state_up |
+--------------------------------------+--------------------+----------------------------------------+-------+----------------+
| 4cebaab8-62a8-4513-8076-39ea55e57cdd | Metadata agent     | ip-192-169-142-127.ip.secureserver.net | :-)   | True           |
| 556b5afd-ac63-45d9-9001-7d2bee997f18 | Open vSwitch agent | ip-192-169-142-137.ip.secureserver.net | :-)   | True           |
| 79c9f9b0-91e3-4201-8b98-9eeab78f5bbc | DHCP agent         | ip-192-169-142-127.ip.secureserver.net | :-)   | True           |
| b0688bbe-9ab6-44d7-83f4-3fbfd6da1003 | Open vSwitch agent | ip-192-169-142-127.ip.secureserver.net | :-)   | True           |
| dd9c8a5e-2876-4f79-b06e-04a979a5fc87 | L3 agent           | ip-192-169-142-127.ip.secureserver.net | :-)   | True           |
+--------------------------------------+--------------------+----------------------------------------+-------+----------------+

*********************
On Controller :-
*********************

[root@ip-192-169-142-127 ~(keystone_admin)]# ovs-vsctl show
3f6c4c61-56d9-4436-84ca-99c1289d7644
    Bridge "br-eth1"
        Port "eth1"
            Interface "eth1"
        Port "phy-br-eth1"
            Interface "phy-br-eth1"
        Port "br-eth1"
            Interface "br-eth1"
                type: internal
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port "qg-8f154600-13"
            Interface "qg-8f154600-13"
                type: internal
        Port "eth0"
            Interface "eth0"
    Bridge br-int
        Port "qr-053f82ff-6d"
            tag: 1
            Interface "qr-053f82ff-6d"
                type: internal
        Port br-int
            Interface br-int
                type: internal
        Port "tapb7744a3b-05"
            tag: 1
            Interface "tapb7744a3b-05"
                type: internal
        Port "int-br-eth1"
            Interface "int-br-eth1"
        Port "qvo70f4f21a-34"
            tag: 1
            Interface "qvo70f4f21a-34"
    ovs_version: "2.0.1"

*******************
On Compute:-
*******************

[root@ip-192-169-142-137 ~]# ovs-vsctl show
1ddf47f0-88f4-4df1-ba6c-c73666e90e56
    Bridge br-int
        Port br-int
            Interface br-int
                type: internal
        Port "qvoc3fd92d1-1e"
            tag: 1
            Interface "qvoc3fd92d1-1e"
        Port "int-br-eth1"
            Interface "int-br-eth1"
    Bridge "br-eth1"
        Port "br-eth1"
            Interface "br-eth1"
                type: internal
        Port "eth1"
            Interface "eth1"
        Port "phy-br-eth1"
            Interface "phy-br-eth1"
    ovs_version: "2.0.1"

[root@ip-192-169-142-137 ~]# brctl show
bridge name    bridge id        STP enabled    interfaces
qbrc3fd92d1-1e        8000.aee2ccfcb2bd    no        qvbc3fd92d1-1e
                            tapc3fd92d1-1e

Login to Ubuntu VM running at Compute Node :-

[root@ip-192-169-142-127 ~]# ssh -l ubuntu -i oskey25.pem 192.169.142.153
Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0-24-generic x86_64)

* Documentation: https://help.ubuntu.com/

System information disabled due to load higher than 1.0

Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud

0 packages can be updated.
0 updates are security updates.

Last login: Fri May 23 03:01:11 2014 from ip-192-169-142-127.ip.secureserver.net

Ubuntu VM deployed on Compute Node 192.169.1.137 :-

Running links on VM

Saturday, May 17, 2014

Two Node (Controller+Compute) IceHouse Neutron OVS&VLAN Cluster on CentOS 6.5

Two KVMs have been created , each one having 2 virtual NICs (eth0,eth1) for
Controller && Compute Nodes setup. Before running `packstack --answer-file=twoNode-answer.txt` SELINUX set to permissive on both nodes. Service NetworkManager disabled, service network enabled. Interfaces eth1 on both nodes set to promiscuous mode (e.g. HWADDRESS was commented out). Testing VMs network performance on Compute Node , I had to disable checksum offloading on eth1 on Compute :-

# /sbin/ethtool --offload eth1 tx off

In case of further problems with network, switch to setup on non-default Libvirt's subnet ( for instance 192.169.142.0/24).

View How to create non-default Libvirt subnet
http://kashyapc.fedorapeople.org/virt/create-a-new-libvirt-bridge.txt

Link for download is here Answer-file

After `packstack --answer-file=twoNode-answer.txt` successful run :-

[root@icehouse1 neutron(keystone_admin)]# openstack-status

== Nova services ==
openstack-nova-api:                     active
openstack-nova-cert:                    active
openstack-nova-compute:                 active
openstack-nova-network:                 dead      (disabled on boot)
openstack-nova-scheduler:               active
openstack-nova-conductor:               active
== Glance services ==
openstack-glance-api:                   active
openstack-glance-registry:              active
== Keystone service ==
openstack-keystone:                     active
== Horizon service ==
openstack-dashboard:                    active
== neutron services ==
neutron-server:                         active
neutron-dhcp-agent:                     active
neutron-l3-agent:                       active
neutron-metadata-agent:                 active
neutron-lbaas-agent:                    active
neutron-openvswitch-agent:              active
== Swift services ==
openstack-swift-proxy:                  active
openstack-swift-account:                active
openstack-swift-container:              active
openstack-swift-object:                 active
== Cinder services ==
openstack-cinder-api:                   active
openstack-cinder-scheduler:             active
openstack-cinder-volume:                active
openstack-cinder-backup:                active
== Ceilometer services ==
openstack-ceilometer-api:               active
openstack-ceilometer-central:           active
openstack-ceilometer-compute:           active
openstack-ceilometer-collector:         active
openstack-ceilometer-alarm-notifier:    active
openstack-ceilometer-alarm-evaluator:   active
== Support services ==
libvirtd:                               active
openvswitch:                            active
messagebus:                             active
tgtd:                                   active
rabbitmq-server:                        active
memcached:                              active
== Keystone users ==
+----------------------------------+------------+---------+----------------------+
|                id                |    name    | enabled |        email         |
+----------------------------------+------------+---------+----------------------+
| 4f12ef0e71c14001a083467641658abf |   admin    |   True |    test@test.com     |
| 162bc283029a4a3fbe48608c3eb0ed09 | ceilometer |   True | ceilometer@localhost |
| 16188cb1bff44e73841fe34960c996fb |   cinder   |   True |   cinder@localhost   |
| c04ccaf920b447b9913120de4621694d |   glance   |   True |   glance@localhost   |
| 5a56b1e0d4fc450ca2a167aa089e4f60 | neutron   |   True | neutron@localhost   |
| 404c881482c34577bdb07847c1514f35 |    nova    |   True |    nova@localhost    |
| 9145a9d4414a48cf9b2bd652d2e5415e |   swift    |   True |   swift@localhost    |
+----------------------------------+------------+---------+----------------------+
== Glance images ==
+--------------------------------------+---------------+-------------+------------------+----------+--------+
| ID                                   | Name          | Disk Format | Container Format | Size     | Status |
+--------------------------------------+---------------+-------------+------------------+----------+--------+
| 1cbfb4d8-ba0b-4553-b8f6-460cba1531c2 | Cirros31Image | qcow2       | bare             | 13147648 | active |
+--------------------------------------+---------------+-------------+------------------+----------+--------+
== Nova managed services ==
+------------------+-----------------------+----------+---------+-------+----------------------------+-----------------+
| Binary           | Host                  | Zone     | Status | State | Updated_at                 | Disabled Reason |
+------------------+-----------------------+----------+---------+-------+----------------------------+-----------------+
| nova-consoleauth | icehouse1.localdomain | internal | enabled | up    | 2014-05-17T23:14:48.000000 | -               |
| nova-scheduler   | icehouse1.localdomain | internal | enabled | up    | 2014-05-17T23:14:48.000000 | -               |
| nova-conductor   | icehouse1.localdomain | internal | enabled | up    | 2014-05-17T23:14:45.000000 | -               |
| nova-compute     | icehouse1.localdomain | nova     | enabled | up    | 2014-05-17T23:14:49.000000 | -               |
| nova-cert        | icehouse1.localdomain | internal | enabled | up    | 2014-05-17T23:14:51.000000 | -               |
| nova-compute     | icehouse2.localdomain | nova     | enabled | up    | 2014-05-17T23:14:46.000000 | -               |
+------------------+-----------------------+----------+---------+-------+----------------------------+-----------------+
== Nova networks ==
+--------------------------------------+--------+------+
| ID                                   | Label | Cidr |
+--------------------------------------+--------+------+
| 6af03a26-3577-4dbc-b1cf-2bd571de04d8 | public | -    |
| c358b6ac-ecd8-4ac7-8c54-4a97a1946969 | int    | -    |
| ec5c4d42-207a-4b38-b8b8-6426e4765756 | int1   | -    |
+--------------------------------------+--------+------+
== Nova instance flavors ==
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| ID | Name      | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| 1 | m1.tiny   | 512       | 1    | 0         |      | 1     | 1.0         | True      |
| 2 | m1.small | 2048      | 20   | 0         |      | 1     | 1.0         | True      |
| 3 | m1.medium | 4096      | 40   | 0         |      | 2     | 1.0         | True      |
| 4 | m1.large | 8192      | 80   | 0         |      | 4     | 1.0         | True      |
| 5 | m1.xlarge | 16384     | 160 | 0         |      | 8     | 1.0         | True      |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
== Nova instances ==
+--------------------------------------+----------+-----------+------------+-------------+--------------------------------+
| ID                                   | Name     | Status    | Task State | Power State | Networks                       |
+--------------------------------------+----------+-----------+------------+-------------+--------------------------------+
| c7546511-b4ae-45ef-b5c1-5b5beaee3ec9 | Cirros31 | SUSPENDED | -          | Shutdown    | int1=40.0.0.2, 192.168.122.153 |
+--------------------------------------+----------+-----------+------------+-------------+--------------------------------+

[root@icehouse1 neutron(keystone_admin)]# nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At

nova-consoleauth icehouse1.localdomain                internal         enabled    :-)   2014-05-17 23:14:58
nova-scheduler   icehouse1.localdomain           internal         enabled    :-)   2014-05-17 23:14:58
nova-conductor   icehouse1.localdomain              internal         enabled    :-)   2014-05-17 23:14:55
nova-compute     icehouse1.localdomain               nova             enabled    :-)   2014-05-17 23:14:59
nova-cert        icehouse1.localdomain                 internal         enabled    :-)   2014-05-17 23:15:01
nova-compute     icehouse2.localdomain               nova             enabled    :-)   2014-05-17 23:14:56

[root@icehouse1 neutron(keystone_admin)]# neutron agent-list
+--------------------------------------+--------------------+-----------------------+-------+----------------+
| id                                   | agent_type         | host                  | alive | admin_state_up |
+--------------------------------------+--------------------+-----------------------+-------+----------------+
| 23f809cf-839d-45ae-9050-48e73e4ba7cc | Open vSwitch agent | icehouse2.localdomain | :-)   | True           |
| 56eed6cc-3218-42e0-87dd-0daec7d41e2d | Metadata agent     | icehouse1.localdomain | :-)   | True           |
| 6787db4d-0124-428f-bfaf-91ab4c1c7a01 | Open vSwitch agent | icehouse1.localdomain | :-)   | True           |
| b3ac0969-023e-4d15-b40f-d1d056ff9132 | DHCP agent         | icehouse1.localdomain | :-)   | True           |
| da5b54e8-b8c5-4429-8938-907ce51dc0b7 | L3 agent           | icehouse1.localdomain | :-)   | True           |
+--------------------------------------+--------------------+-----------------------+-------+----------------+

SSH into Cirros instance && verification metadata

[root@icehouse1 ~]# ssh -l cirros -i oskey25.priv 192.168.122.155
The authenticity of host '192.168.122.155 (192.168.122.155)' can't be established.
RSA key fingerprint is f2:bd:fc:82:31:9b:6e:03:47:4c:0f:32:79:9d:10:06.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.122.155' (RSA) to the list of known hosts.

$ ifconfig
eth0      Link encap:Ethernet HWaddr FA:16:3E:26:93:2E
          inet addr:40.0.0.5 Bcast:40.0.0.255 Mask:255.255.255.0
          inet6 addr: fe80::f816:3eff:fe26:932e/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:297 errors:0 dropped:0 overruns:0 frame:0
          TX packets:330 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:33724 (32.9 KiB) TX bytes:33981 (33.1 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING MTU:16436 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

$ ping 83.221.202.254
PING 83.221.202.254 (83.221.202.254): 56 data bytes
64 bytes from 83.221.202.254: seq=0 ttl=58 time=2.501 ms
64 bytes from 83.221.202.254: seq=1 ttl=58 time=3.053 ms
64 bytes from 83.221.202.254: seq=2 ttl=58 time=2.626 ms
64 bytes from 83.221.202.254: seq=3 ttl=58 time=2.642 ms
64 bytes from 83.221.202.254: seq=4 ttl=58 time=4.335 ms
64 bytes from 83.221.202.254: seq=5 ttl=58 time=2.636 ms
64 bytes from 83.221.202.254: seq=6 ttl=58 time=2.963 ms
64 bytes from 83.221.202.254: seq=7 ttl=58 time=2.623 ms
64 bytes from 83.221.202.254: seq=8 ttl=58 time=2.712 ms
64 bytes from 83.221.202.254: seq=9 ttl=58 time=3.081 ms
64 bytes from 83.221.202.254: seq=10 ttl=58 time=2.889 ms
64 bytes from 83.221.202.254: seq=11 ttl=58 time=2.613 ms
64 bytes from 83.221.202.254: seq=12 ttl=58 time=2.512 ms
64 bytes from 83.221.202.254: seq=13 ttl=58 time=2.901 ms
64 bytes from 83.221.202.254: seq=14 ttl=58 time=2.491 ms
64 bytes from 83.221.202.254: seq=15 ttl=58 time=2.381 ms
64 bytes from 83.221.202.254: seq=16 ttl=58 time=2.876 ms
64 bytes from 83.221.202.254: seq=17 ttl=58 time=2.867 ms

$ uname -a
Linux cirros 3.2.0-37-virtual #58-Ubuntu SMP Thu Jan 24 15:48:03 UTC 2013 x86_64 GNU/Linux

$ curl http://169.254.169.254/openstack/latest/meta_data.json
{"random_seed": "A5Z/jEl2+Xc8UkIW1RjkHjMHqShSOeIGGfqtT9NGxpvEVONBai9T67yyn72S3n5sW75Drn8VxfJQpIZxYZ6Y0RQSzr/51UoREBN0t6GsL1IPqF3+/OM1YzYc7TXPx67KgEfb0FcVcXeS+0Cwb/RcH6nMRpafGO713aMWAxaiRvIxFoPJy2ZMb3bUZ1GSWW3hN7CKvECQPyYQmkTaIVzjjBIulvw1kYFVXeEQUs8Z0SZjGxWLwpSGp0HGvLdkl3ZJljMyT4lHaxJHVBRjEcnTv4krCdXRsjct2L8qzzoBukStRIW+RR8843r/AkdIfAv7LtRc4Tb99xjsKGCzrg1tll1fRdf1FxnGxti14V+P9EiMUekPBiEjYYGMf44edMVfX3cTPY9PMOMa4rw6jAJuEsmH43t8ggR+xUOEOQihCoA6Cv46lsM3I/oYyb21FLW84sWyiZtLgn6i2x1CJ2OlytBqha/k4MgtEEkDdXrcSx3JfIGLjpOi/9s39jBGK6pMTsx0cnTsyFe/hiWnAjflt11c6zWIMUDOk8QuMyWiBZOSpobohFsiW85SJ4xoP+RqmjFVRAs0ubns3Xd2pHodQR2H0Y4KqZIlH00Uli1vOOGiN0P2CtTguZOhZAru+UlF1AWTRAmCV3ZNwLj6xegeEh4nCy8H8hv7ibGfgMgc95c=", "uuid": "675f2943-ba79-47ae-b78b-cbfcd67d0a99", "availability_zone": "nova", "hostname": "cirros35.novalocal", "launch_index": 0, "public_keys": {"oskey25": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAszAmP1OUpiPBMzjIoo0mL2nx16QrBDcjEOnOPBdlHKTQi0wUXNt008WbF0VSVy2QKZVqgzWp+YqsWx2NV9i+PdajSlP0JRcJKGyK1KYnp9Dp8t3LLXBP/HunTK3hv5NbAE5v/CFZnU9Z7rn6H0fnce4yLztnh+2W2oDLXJ+9eI3fEm+Dm0xB7/R7Rrz6gX2jz55IF5ZQCtp/TIn+e2eSJijpaZQ4uROx8ir1E6pcfWLul131UF7FbV/Q6UXPTZa0zS92lo4OnTywdH61i5G8q/qQOPZ0w4lEBY5+qwnQSGdTg8alcEs4STlIFniU7Wu8OjtPFGWhFluyLaZhU4Zr1Q== Generated by Nova$ curl http://169.254.169.254/latest/meta-data/hostname
$ curl http://169.254.169.254/latest/meta-data/local-ipv4
40.0.0.5$
$ curl http://169.254.169.254/latest/meta-data/public-ipv4
192.168.122.155$
$