Xen Virtualization on Linux and Solaris: RDO Setup Two Real Node (Controller+Compute) IceHouse Neutron ML2&OVS&VLAN Cluster on CentOS 7

As of 07/14/2014 Bug https://ask.openstack.org/en/question/35705/attempt-of-rdo-aio-install-icehouse-on-centos-7/ is still pending and workaround suggested above should be applied during two node RDO packstack installation.
Successful implementation of Neutron ML2&&OVS&&VLAN multi node setup requires correct version of plugin.ini -> /etc/neutron/plugins/ml2/ml2_conf.ini which appears to be generated with errors by packstack.

Two boxes have been setup , each one having 2 NICs (enp2s0,enp5s1) for
Controller && Compute Nodes setup. Before running
`packstack --answer-file=TwoRealNode Neutron ML2&OVS&VLAN.txt` SELINUX set to permissive on both nodes.Both enp5s1's assigned IPs and set to promiscuous mode (192.168.0.127, 192.168.0.137 ). Services firewalld and NetworkManager disabled, IPv4 firewall with iptables and service network are enabled and running. Packstack is bind to public IP of interface enp2s0 192.169.1.127, Compute Node is 192.169.1.137 ( view answer-file ).

Setup configuration

- Controller node: Nova, Keystone, Cinder, Glance, Neutron (using Open vSwitch plugin && VLAN )
- Compute node: Nova (nova-compute), Neutron (openvswitch-agent)

icehouse1.localdomain - Controller (192.168.1.127)
icehouse2.localdomain - Compute (192.168.1.137)

Status after packstack install and updating /etc/neutron/plugin.ini as shown bellow

[root@icehouse1 neutron]# cat plugin.ini
[ml2]
type_drivers = vlan
tenant_network_types = vlan
mechanism_drivers = openvswitch
[ml2_type_vlan]
network_vlan_ranges = physnet1:100:200
[ovs]
network_vlan_ranges = physnet1:100:200
tenant_network_type = vlan
enable_tunneling = False
integration_bridge = br-int
bridge_mappings = physnet1:br-enp5s1
local_ip = 192.168.1.127
[AGENT]
polling_interval = 2
[SECURITYGROUP]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver

[root@icehouse1 ~(keystone_admin)]# ls -l /etc/neutron
total 60
-rw-r--r--. 1 root root 193 Jul 13 09:06 api-paste.ini
-rw-r-----. 1 root neutron 3853 Jul 13 09:05 dhcp_agent.ini
-rw-r-----. 1 root neutron 208 Jul 13 09:06 fwaas_driver.ini
-rw-r-----. 1 root neutron 3431 Jul 13 09:06 l3_agent.ini
-rw-r-----. 1 root neutron 1400 Jun 8 01:38 lbaas_agent.ini
-rw-r-----. 1 root neutron 1481 Jul 13 09:06 metadata_agent.ini
-rw-r-----. 1 root neutron 19148 Jul 13 09:06 neutron.conf
lrwxrwxrwx. 1 root root 37 Jul 13 09:06 plugin.ini -> /etc/neutron/plugins/ml2/ml2_conf.ini
drwxr-xr-x. 4 root root 34 Jul 13 09:06 plugins
-rw-r-----. 1 root neutron 6148 Jun 8 01:38 policy.json
-rw-r--r--. 1 root root 78 Jun 24 15:14 release
-rw-r--r--. 1 root root 1216 Jun 8 01:38 rootwrap.conf

# systemctl disable NetworkManager
# systemctl stop NetworkManager
# chkconfig network on

Assigning IP to OVS bridge br-ex and making enp2s0 OVS port of br-ex :-

[root@icehouse1 network-scripts(keystone_admin)]# cat ifcfg-br-ex
DEVICE="br-ex"
BOOTPROTO="static"
IPADDR="192.168.1.127"
NETMASK="255.255.255.0"
DNS1="83.221.202.254"
BROADCAST="192.168.1.255"
GATEWAY="192.168.1.1"
NM_CONTROLLED="no"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="yes"
IPV6INIT=no
ONBOOT="yes"
TYPE="OVSBridge"
DEVICETYPE="ovs"

[root@icehouse1 network-scripts(keystone_admin)]# cat ifcfg-enp2s0
DEVICE=enp2s0
ONBOOT="yes"
TYPE="OVSPort"
DEVICETYPE="ovs"
OVS_BRIDGE=br-ex
NM_CONTROLLED=no
IPV6INIT=no

# service network restart

************************

Metadata Trobleshooting
************************

[root@icehouse1 ~(keystone_admin)]# netstat -antp| grep 8775
tcp 0 0 0.0.0.0:8775 0.0.0.0:* LISTEN 1580/python
[root@icehouse1 ~(keystone_admin)]# ps -ef | grep 1580
nova 1580 1 0 07:50 ? 00:01:28 /usr/bin/python /usr/bin/nova-api
nova 5192 1580 0 07:51 ? 00:00:00 /usr/bin/python /usr/bin/nova-api
nova 5193 1580 0 07:51 ? 00:00:00 /usr/bin/python /usr/bin/nova-api
nova 5218 1580 0 07:51 ? 00:00:04 /usr/bin/python /usr/bin/nova-api
nova 5219 1580 0 07:51 ? 00:00:05 /usr/bin/python /usr/bin/nova-api
nova 5226 1580 0 07:51 ? 00:00:00 /usr/bin/python /usr/bin/nova-api
nova 5228 1580 0 07:51 ? 00:00:00 /usr/bin/python /usr/bin/nova-api
root 19054 15837 0 12:16 pts/0 00:00:00 grep --color=auto 1580

[root@icehouse1 ~(keystone_admin)]# ip netns
qrouter-9bfc940b-ca64-4e36-9597-188f67ba44a4
qrouter-a3fad1f2-7c02-4952-ba35-fc1e836a1805
qrouter-9d4283e9-8d53-415e-aa5f-427e3ca69da1
qdhcp-12e1088e-1edf-488a-b8c9-0a4085ea2803
qdhcp-1722093c-1852-4eab-a76f-1f9e135880a8
qdhcp-430a0ef1-7fc8-40d4-9d25-d69e068b1c98

[root@icehouse1 ~(keystone_admin)]# ip netns exec qrouter-9bfc940b-ca64-4e36-9597-188f67ba44a4 iptables -S -t nat | grep 169.254
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697

[root@icehouse1 ~(keystone_admin)]# ip netns exec qrouter-9bfc940b-ca64-4e36-9597-188f67ba44a4 netstat -antp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:9697 0.0.0.0:* LISTEN 6954/python

[root@icehouse1 ~(keystone_admin)]# ps -ef | grep 6954
root 6954 1 0 07:52 ? 00:00:00 /usr/bin/python /bin/neutron-ns-metadata-proxy --pid_file=/var/lib/neutron/external/pids/9bfc940b-ca64-4e36-9597-188f67ba44a4.pid --metadata_proxy_socket=/var/lib/neutron/metadata_proxy --router_id=9bfc940b-ca64-4e36-9597-188f67ba44a4 --state_path=/var/lib/neutron --metadata_port=9697 --verbose --log-file=neutron-ns-metadata-proxy-9bfc940b-ca64-4e36-9597-188f67ba44a4.log --log-dir=/var/log/neutron
root 19599 15837 0 12:20 pts/0 00:00:00 grep --color=auto 6954