Tuesday, January 19, 2016

How to create a tenant's network and the packet will go out with the same fixed ip of the vm on RDO Liberty ?


This article is  simply a joke in response to "very special" question.
It isn't necessary to perceive me seriously as one serious edition.
Actually , the fault was mine I forgot to click on "Humor" section
instead of "tutorial". Thanks God Linux.com understood me correctly
See https://www.linux.com/community/blogs/133-general-linux/880030-how-to-create-a-tenants-network-and-the-packet-will-go-out-with-the-same-fixed-ip-of-the-vm-/


The source is obviously ask.openstack.org. Question as it sounds originally :-

Can i create a tenant network and the packet will go out with the same fixed ip of the vm?  Assuming the tenant network created is routable or identifiable in the physical network.

The answer is "YES". However , it is going to switch off Neutron L3 routing
and bring you back to simple flat networking with no DNAT/SNAT routing
involved.

Here is important to understand, that bridged neutron external networking
with SNAT disabled , won't provide you  outbound internet connectivity ,
unless your VMs would have floating IPs assigned, however in case of using external network provider non bridged external networking will come into play. Via tap-interface (tap-e3b5d93-e6) attached to br-int neutron  flow will be forwarded to  Port (int-br-ex) , which makes a veth-pair with Port (phy-br-ex) . The last one belongs to bridge br-ex, the bridge which due to OVS configuration (presumably been tuned properly)  will provide vice/versa Internet connectivity.

Neutron work flow on RDO Liberty AIO Node will look like this

[root@ip-192-169-142-57 ~]# ovs-vsctl show
853a65bf-1584-4ef7-b94e-867171f20fd6
    Bridge br-tun
        fail_mode: secure
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
    Bridge br-int
        fail_mode: secure
        Port "tape3b5bd93-e6"
            tag: 1
            Interface "tape3b5bd93-e6"    =======>
                type: internal
        Port "qvo5377db58-81"
            tag: 1
            Interface "qvo5377db58-81"
        Port "qvoa4f0faaa-52"
            tag: 1
            Interface "qvoa4f0faaa-52"
        Port br-int
            Interface br-int
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port int-br-ex                    <========| 
            Interface int-br-ex                    |
                type: patch                        |
                options: {peer=phy-br-ex}          | 
    Bridge br-ex                               VETH PAIR
        Port br-ex                                 |
            Interface br-ex                        |
                type: internal                     |
        Port phy-br-ex                             | 
            Interface phy-br-ex   <================|
                type: patch
                options: {peer=int-br-ex}
        Port "eth0"
            Interface "eth0"
    ovs_version: "2.4.0"

[root@ip-192-169-142-57 ~]# ip netns
qdhcp-92c6b85e-4798-4553-b5e8-795f592e170e

[root@ip-192-169-142-57 ~]# ip netns exec qdhcp-92c6b85e-4798-4553-b5e8-795f592e170e ip route
default via 192.169.142.1 dev tape3b5bd93-e6 
169.254.0.0/16 dev tape3b5bd93-e6  proto kernel  scope link  src 169.254.169.254 
192.169.142.0/24 dev tape3b5bd93-e6  proto kernel  scope link  src 192.169.142.150 
The next question is how to achieve this really nice picture,
which brings you back in times of flat legacy (nova) networking.

**********************************************************************************
After AIO RDO setup make sure you configuration files look as follows :-
**********************************************************************************
[root@ip-192-169-142-57 neutron(keystone_admin)]# cat ml2_conf.conf
[ml2]
type_drivers = flat,vxlan
tenant_network_types = vxlan
mechanism_drivers =openvswitch
path_mtu = 0
[ml2_type_flat]
flat_networks =*
[ml2_type_vlan]
[ml2_type_gre]
[ml2_type_vxlan]
vni_ranges =10:100
vxlan_group =224.0.0.1
[ml2_type_geneve]
[securitygroup]
enable_security_group = True 
 
[root@ip-192-169-142-57 neutron(keystone_admin)]# cat dhcp_agent.conf
[DEFAULT]
debug = False
resync_interval = 30
interface_driver =neutron.agent.linux.interface.OVSInterfaceDriver
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
enable_isolated_metadata = True    <========= Directive 1
enable_metadata_network = False
dhcp_domain = openstacklocal
dnsmasq_config_file =/etc/neutron/dnsmasq-neutron.conf
dhcp_broadcast_reply = False
dhcp_delete_namespaces = False
root_helper=sudo neutron-rootwrap /etc/neutron/rootwrap.conf
state_path=/var/lib/neutron
[AGENT] 

 
[root@ip-192-169-142-57 neutron(keystone_admin)]# cat l3_agent.conf


[DEFAULT]
debug = False
interface_driver =neutron.agent.linux.interface.OVSInterfaceDriver
handle_internal_only_routers = True
external_network_bridge =
metadata_port = 9697
send_arp_for_ha = 3
periodic_interval = 40
periodic_fuzzy_delay = 5
enable_metadata_proxy = True
router_delete_namespaces = False
agent_mode = legacy
[AGENT]
 
[root@ip-192-169-142-57 ~(keystone_admin)]# cat openvswitch_agent.conf
[ovs]
integration_bridge = br-int
tunnel_bridge = br-tun
local_ip =192.169.142.57
bridge_mappings = physnet1:br-ex <====== Directive 2
enable_tunneling=True
[agent]
polling_interval = 2
tunnel_types =vxlan
vxlan_udp_port =4789
l2_population = False
arp_responder = False
prevent_arp_spoofing = True
enable_distributed_routing = False
drop_flows_on_start=False
[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
********************
When done :-
********************
# openstack-service restart neutron  
********************************
Update your ifcfg-* files
********************************
network-scripts(keystone_admin)]# cat ifcfg-br-ex
DEVICE="br-ex"
BOOTPROTO="static"
IPADDR="192.169.142.57"
NETMASK="255.255.255.0"
DNS1="83.221.202.254"
BROADCAST="192.169.142.255"
GATEWAY="192.169.142.1"
NM_CONTROLLED="no"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="yes"
IPV6INIT=no
ONBOOT="yes"
TYPE="OVSIntPort"
OVS_BRIDGE=br-ex
DEVICETYPE="ovs"


[root@ip-192-169-142-57 network-scripts(keystone_admin)]# cat ifcfg-eth0
DEVICE="eth0"
ONBOOT="yes"
TYPE="OVSPort"
DEVICETYPE="ovs"
OVS_BRIDGE=br-ex
NM_CONTROLLED=no
IPV6INIT=no
***************************
Then run script
***************************
#!/bin/bash -x
chkconfig network on
systemctl stop NetworkManager
systemctl disable NetworkManager
service network restart


Reboot

******************************************************************************
Presuming that your host has static ip on 192.169.142.0/24
create external network as follows :-
******************************************************************************

[root@ip-192-169-142-57 (keyystone_admin)]#  neutron net-create flatnet    --shared --provider:network_type flat --provider:physical_network physnet1 --router:external

[root@ip-192-169-142-57 (keyystone_admin)]# neutron subnet-create --name sub-flatnet --gateway 192.169.142.1  --allocation-pool start=192.169.142.150,end=192.168.142.254  --enable-dhcp --dns-nameserver 83.221.202.254  flatnet 192.169.142.0/24
                                         
[root@ip-192-169-142-57 (keyystone_admin)]# neutron subnet-update --host-route  destination=169.254.169.254/32,nexthop=192.169.142.151 flatnet    <== Directive 3                                 

************************************
Due to directives 1,2,3  we'll get
*************************************
[root@ip-192-169-142-57 ~(keystone_admin)]# ip netns exec qdhcp-92c6b85e-4798-4553-b5e8-795f592e170e netstat -antp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name   
tcp        0      0 0.0.0.0:80                      0.0.0.0:*    LISTEN      6055/python2       
tcp        0      0 192.169.142.150:53      0.0.0.0:*    LISTEN      6337/dnsmasq       
tcp        0      0 169.254.169.254:53      0.0.0.0:*    LISTEN      6337/dnsmasq       
tcp6       0      0 fe80::f816:3eff:fe10:53 :::*             LISTEN      6337/dnsmasq       

[root@ip-192-169-142-57 ~(keystone_admin)]# ps -f --pid 6055 | fold -s -w 82
UID        PID  PPID  C STIME TTY          TIME CMD
neutron   6055     1  0 16:17 ?        00:00:00 /usr/bin/python2
/bin/neutron-ns-metadata-proxy
--pid_file=/var/lib/neutron/external/pids/92c6b85e-4798-4553-b5e8-795f592e170e.pid
 --metadata_proxy_socket=/var/lib/neutron/metadata_proxy
--network_id=92c6b85e-4798-4553-b5e8-795f592e170e --state_path=/var/lib/neutron
--metadata_port=80 --metadata_proxy_user=986 --metadata_proxy_group=981 --verbose
--log-file=neutron-ns-metadata-proxy-92c6b85e-4798-4553-b5e8-795f592e170e.log
--log-dir=/var/log/neutron

[root@ip-192-169-142-57 ~(keystone_admin)]# netstat -lxp | grep metadata
unix  2      [ ACC ]     STREAM     LISTENING     32909    1271/python2         /var/lib/neutron/metadata_proxy

[root@ip-192-169-142-57 ~(keystone_admin)]#  ps -f --pid 1271 | fold -s -w 82
UID        PID  PPID  C STIME TTY          TIME CMD
neutron   1271     1  0 16:16 ?        00:00:08 /usr/bin/python2
/usr/bin/neutron-metadata-agent --config-file
/usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf
--config-file /etc/neutron/metadata_agent.ini --config-dir
/etc/neutron/conf.d/common --config-dir
/etc/neutron/conf.d/neutron-metadata-agent --log-file
/var/log/neutron/metadata-agent.log

********************************************************************************
Hence metadata access will be handled as in good old times via qdhcp
namespace generated  by external flat network 192.169.142.0/24
********************************************************************************

[root@ip-192-169-142-57 ~(keystone_admin)]# ip netns exec qdhcp-92c6b85e-4798-4553-b5e8-795f592e170e ip route
default via 192.169.142.1 dev tape3b5bd93-e6
169.254.0.0/16 dev tape3b5bd93-e6  proto kernel  scope link  src 169.254.169.254
192.169.142.0/24 dev tape3b5bd93-e6  proto kernel  scope link  src 192.169.142.150

[root@ip-192-169-142-57 ~(keystone_admin)]# ip netns exec qdhcp-92c6b85e-4798-4553-b5e8-795f592e170e ifconfig
lo: flags=73  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10
        loop  txqueuelen 0  (Local Loopback)
        RX packets 1  bytes 576 (576.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1  bytes 576 (576.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tape3b5bd93-e6: flags=4163  mtu 1500
        inet 192.169.142.150  netmask 255.255.255.0  broadcast 192.169.142.255

        inet6 fe80::f816:3eff:fe10:91c7  prefixlen 64  scopeid 0x20
        ether fa:16:3e:10:91:c7  txqueuelen 0  (Ethernet)
        RX packets 406  bytes 42677 (41.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 229  bytes 29830 (29.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0



So VM been started by admin will get access to Nova Metadata Sever and will be routed to Internet and vice/versa via OVS bridge br-ex avoiding any neutron router.