*******************************************************************************
UPDATE : As of 03/11/2015 Patch bellow merged upstream
In meantime instructions in UPDATE : As of 03/09/2015 are already a history
View : https://review.openstack.org/#/c/163022/
View : https://git.openstack.org/cgit/stackforge/nova-docker/
*******************************************************************************
UPDATE : As of 03/09/2015
View What is missing commit 9d06520645f28d96ef905a709f8ff0c27842b58b in nova-docker master branch ?
for details and explanation what is wrong with commit mentioned above.
To succeed with Nova Docker driver build on Ubuntu 14.04.2 proceed as
follows, otherwise you will be able load driver via stack.sh run, but network
- floating and private IPs wouldn't work . Nova will just boot container and nothing else. Patch bellow is easy to apply manually . It will result bringing container's interface up, and network alive and ready to work for you.
$ git clone http://github.com/stackforge/nova-docker.git
$ cd nova-docker
Apply patch
$ sudo pip install .
********************************************************************************
Recently new patch https://review.openstack.org/#/c/154750/ merged
https://github.com/stackforge/nova-docker.git what made possible
to test Nova-docker driver built via current git tree status with the most
recent openstack code obtained by devstack by cloning https://git.openstack.org/openstack-dev/devstack However,nova-docker containers have been lost after every reboot due to bridge br-ex came up with no IP and running ./rejoin-stack.sh didn't help much. This post describes workaround for this issue.
First part of article actually follows http://blog.oddbit.com/2015/02/11/installing-novadocker-with-devstack/
written by Lars Kellogg-Stedman with non-critical changes in local.conf file.
Second part of article provides workaround making created nova-docker
instances and all devstack environment recoverable between reboots.
Reproducing the first part I also installed horizon launching nova-docker containers and assigning floating IPs clicking by mouse ( via admin login working with preinstalled Demo project )
Run as root ( post install ) to open way out for VMs
*************************************************************************
# iptables -t nat -A POSTROUTING -o eth0 -j
*************************************************************************
$ sudo apt-get update
$ sudo apt-get -y install git git-review python-pip python-dev
$ sudo apt-get -y upgrade
$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
$ sudo sh -c "echo deb https://get.docker.com/ubuntu docker main \
> /etc/apt/sources.list.d/docker.list"
$ sudo apt-get update
$ sudo apt-get install lxc-docker
*********************************************
Update /etc/default/docker and setting:
*********************************************
DOCKER_OPTS='-G ubuntu'
#service docker restart
*******************************
Installing nova-docker
*******************************
This block is a subject to change as far as commits done
after e9dcf7e790e4df2f9025b19896173995a32692fc
in particular 85071220cbc3c1edb4a4c67db3e7060284f35c6b
will be tested as not disabling floating IPs.
$ git clone http://github.com/stackforge/nova-docker.git
$ cd nova-docker
$ git checkout e9dcf7e790e4df2f9025b19896173995a32692fc
$ sudo pip install .
***************************************************************************
UPDATE 03/12/2015 To get floating IPs working in meantime I have
***************************************************************************
$ git clone http://github.com/stackforge/nova-docker.git
$ cd nova-docker
$ git revert -m 1 661998214962d3e86063196bda0b3a619b7f4e26
$ sudo pip install .
************************************
UPDATE 03/13/2015
************************************
$ git clone http://github.com/stackforge/nova-docker.git
$ cd nova-docker
$ sudo pip install .
Seems to be working , however I've noticed strange issue with
# iptables -t nat -A POSTROUTING -o eth0 -j
This directive has a potential danger to lock your floating IPs, if you MASQUERADE a concrete sub-net which is providing floating IPs.
Configuring devstack
*****************************
Now we're ready to get devstack up and running. Start by cloning the repository:
$ git clone https://git.openstack.org/openstack-dev/devstack
$ cd devstack
1. Create local.conf under devstack ( original version )
***************
local.conf
***************
[[local|localrc]]
ADMIN_PASSWORD=secret
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
SERVICE_TOKEN=super-secret-admin-token
VIRT_DRIVER=novadocker.virt.docker.DockerDriver
DEST=$HOME/stack
SERVICE_DIR=$DEST/status
DATA_DIR=$DEST/data
LOGFILE=$DEST/logs/stack.sh.log
LOGDIR=$DEST/logs
# The default fixed range (10.0.0.0/24) conflicted with an address
# range I was using locally.
FIXED_RANGE=10.254.1.0/24
NETWORK_GATEWAY=10.254.1.1
# Services
disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service horizon
disable_service tempest
# Introduce glance to docker images
[[post-config|$GLANCE_API_CONF]]
[DEFAULT]
container_formats=ami,ari,aki,bare,ovf,ova,docker
# Configure nova to use the nova-docker driver
[[post-config|$NOVA_CONF]]
[DEFAULT]
compute_driver=novadocker.virt.docker.DockerDriver
*****************************************************************************
My version of local.conf which allows define floating pool as you need,
a bit more flexible then original
*****************************************************************************
[[local|localrc]]
HOST_IP=192.168.1.57
ADMIN_PASSWORD=secret
MYSQL_PASSWORD=secret
RABBIT_PASSWORD=secret
SERVICE_PASSWORD=secret
FLOATING_RANGE=192.168.10.0/24
FLAT_INTERFACE=eth0
Q_FLOATING_ALLOCATION_POOL=start=192.168.10.150,end=192.168.10.254
PUBLIC_NETWORK_GATEWAY=192.168.10.15
SERVICE_TOKEN=super-secret-admin-token
VIRT_DRIVER=novadocker.virt.docker.DockerDriver
DEST=$HOME/stack
SERVICE_DIR=$DEST/status
DATA_DIR=$DEST/data
LOGFILE=$DEST/logs/stack.sh.log
LOGDIR=$DEST/logs
# The default fixed range (10.0.0.0/24) conflicted with an address
# range I was using locally.
FIXED_RANGE=10.254.1.0/24
NETWORK_GATEWAY=10.254.1.1
# Services
disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service horizon
disable_service tempest
# Introduce glance to docker images
[[post-config|$GLANCE_API_CONF]]
[DEFAULT]
container_formats=ami,ari,aki,bare,ovf,ova,docker
# Configure nova to use the nova-docker driver
[[post-config|$NOVA_CONF]]
[DEFAULT]
compute_driver=novadocker.virt.docker.DockerDriver
**************************************
Corresponding iptables entry
**************************************
# iptables -t nat -A POSTROUTING -o eth0 -j
At this point you are ready to run :-
$ ./stack.sh
*****************************************************************************
Attention skipping this step causes message "No hosts available"
when launching, either causes failure to launch nova-docker instances
in case of stack.sh rerun after ./unstack.sh
******************************************************************************
$ sudo cp nova-docker/etc/nova/rootwrap.d/docker.filters \
/etc/nova/rootwrap.d/
$ . openrc admin
For docker pull && docker save
$ . openrc demo
To launch instances
*********************************************************************************
Next issue , you have run `sudo ./unstack.sh` , rebooted box hosting devstack instance and OVS bridge "br-ex" came up with no IP no matter which one of local.conf has been used for ./stack.sh deployment.
Before running ./rejoin-stack.sh following actions have to be undertaken
*********************************************************************************
This version is supposed to work with second version of local.conf
PUBLIC_NETWORK_GATEWAY=192.168.10.15
sudo ip addr flush dev br-ex
sudo ip addr add 192.168.10.15/24 dev br-ex
sudo ip link set br-ex up
sudo route add -net 10.254.1.0/24 gw 192.168.10.15
******************************************************
Verify correct environment installed:-
******************************************************
ubuntu@ubuntu-System-Product-Name:~$ ifconfig
br-ex Link encap:Ethernet HWaddr de:64:4b:ba:a7:48
inet addr:192.168.10.15 Bcast:0.0.0.0 Mask:255.255.255.0
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:2186 errors:0 dropped:0 overruns:0 frame:0
TX packets:2649 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1801780 (1.8 MB) TX bytes:2194422 (2.1 MB)
br-int Link encap:Ethernet HWaddr b2:cf:54:c5:a0:49
inet6 addr: fe80::b007:79ff:fe87:4260/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:648 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:120474 (120.4 KB) TX bytes:648 (648.0 B)
br-tun Link encap:Ethernet HWaddr 3a:fb:71:08:1a:45
inet6 addr: fe80::899:bcff:fed6:8d8d/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:648 (648.0 B)
docker0 Link encap:Ethernet HWaddr 56:84:7a:fe:97:99
inet addr:172.17.42.1 Bcast:0.0.0.0 Mask:255.255.0.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth0 Link encap:Ethernet HWaddr 90:e6:ba:2d:11:eb
inet addr:192.168.1.37 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::92e6:baff:fe2d:11eb/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:64604 errors:0 dropped:0 overruns:0 frame:0
TX packets:37999 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:88470764 (88.4 MB) TX bytes:3455868 (3.4 MB)
eth1 Link encap:Ethernet HWaddr 00:0c:76:e0:1e:c5
inet6 addr: fe80::20c:76ff:fee0:1ec5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:239 errors:0 dropped:0 overruns:0 frame:0
TX packets:389 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:58024 (58.0 KB) TX bytes:75526 (75.5 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:30804 errors:0 dropped:0 overruns:0 frame:0
TX packets:30804 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:10921200 (10.9 MB) TX bytes:10921200 (10.9 MB)
ns44923080-eb Link encap:Ethernet HWaddr 9a:db:d0:5a:ad:02
inet6 addr: fe80::98db:d0ff:fe5a:ad02/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:621 errors:0 dropped:0 overruns:0 frame:0
TX packets:289 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:119156 (119.1 KB) TX bytes:55649 (55.6 KB)
ns9cb8e46e-35 Link encap:Ethernet HWaddr 6e:f3:23:93:b4:11
inet6 addr: fe80::6cf3:23ff:fe93:b411/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:637 errors:0 dropped:0 overruns:0 frame:0
TX packets:271 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:121878 (121.8 KB) TX bytes:52144 (52.1 KB)
tap44923080-eb Link encap:Ethernet HWaddr ee:b3:16:a3:f9:ed
inet6 addr: fe80::ecb3:16ff:fea3:f9ed/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:289 errors:0 dropped:0 overruns:0 frame:0
TX packets:621 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:55649 (55.6 KB) TX bytes:119156 (119.1 KB)
tap8897281a-3f Link encap:Ethernet HWaddr 9a:2a:eb:a5:3d:60
inet6 addr: fe80::982a:ebff:fea5:3d60/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2236 errors:0 dropped:0 overruns:0 frame:0
TX packets:3452 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1812589 (1.8 MB) TX bytes:2351741 (2.3 MB)
tap9cb8e46e-35 Link encap:Ethernet HWaddr 06:3c:cc:e5:30:4a
inet6 addr: fe80::43c:ccff:fee5:304a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:271 errors:0 dropped:0 overruns:0 frame:0
TX packets:637 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:52144 (52.1 KB) TX bytes:121878 (121.8 KB)
virbr0 Link encap:Ethernet HWaddr e2:93:d0:a0:2c:f6
inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
ubuntu@ubuntu-System-Product-Name:~$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 eth0
10.254.1.0 192.168.10.15 255.255.255.0 UG 0 0 0 br-ex
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
192.168.1.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0
192.168.10.0 0.0.0.0 255.255.255.0 U 0 0 0 br-ex
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
****************************************
At this point run you may run
****************************************
cd devstack ; ./rejoin-stack.sh
and it will bring your devstack environment back
********************************************************************
Actually, on Ubuntu 14.04 box doing this kind of testing
********************************************************************
root@ubuntu-P5Q3 :~# cat /etc/rc.local
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
ip addr flush dev br-ex ;
ip addr add 192.168.10.15/24 dev br-ex ;
ip link set br-ex up ;
route add -net 10.254.1.0/24 gw 192.168.10.15 ;
exit 0
*****************************************************************
Establishing access to public devstack net from LAN
UPDATE : As of 03/11/2015 Patch bellow merged upstream
In meantime instructions in UPDATE : As of 03/09/2015 are already a history
View : https://review.openstack.org/#/c/163022/
View : https://git.openstack.org/cgit/stackforge/nova-docker/
*******************************************************************************
UPDATE : As of 03/09/2015
View What is missing commit 9d06520645f28d96ef905a709f8ff0c27842b58b in nova-docker master branch ?
for details and explanation what is wrong with commit mentioned above.
To succeed with Nova Docker driver build on Ubuntu 14.04.2 proceed as
follows, otherwise you will be able load driver via stack.sh run, but network
- floating and private IPs wouldn't work . Nova will just boot container and nothing else. Patch bellow is easy to apply manually . It will result bringing container's interface up, and network alive and ready to work for you.
$ git clone http://github.com/stackforge/nova-docker.git
$ cd nova-docker
Apply patch
diff --git a/novadocker/virt/docker/vifs.py b/novadocker/virt/docker/vifs.py
index a2e7b23..1d159f7 100644
--- a/novadocker/virt/docker/vifs.py
+++ b/novadocker/virt/docker/vifs.py
@@ -248,6 +248,8 @@ class DockerGenericVIFDriver(object):
run_as_root=True)
utils.execute('ip', 'netns', 'exec', container_id, 'ip', 'addr',
'add', ip, 'dev', if_remote_name, run_as_root=True)
+ utils.execute('ip', 'netns', 'exec', container_id, 'ip', 'link',
+ 'set', if_remote_name,'up',run_as_root=True)
if gateway is not None:
utils.execute('ip', 'netns', 'exec', container_id,
'ip', 'route', 'replace', 'default', 'via',
Then build driver$ sudo pip install .
********************************************************************************
Recently new patch https://review.openstack.org/#/c/154750/ merged
https://github.com/stackforge/nova-docker.git what made possible
to test Nova-docker driver built via current git tree status with the most
recent openstack code obtained by devstack by cloning https://git.openstack.org/openstack-dev/devstack However,nova-docker containers have been lost after every reboot due to bridge br-ex came up with no IP and running ./rejoin-stack.sh didn't help much. This post describes workaround for this issue.
First part of article actually follows http://blog.oddbit.com/2015/02/11/installing-novadocker-with-devstack/
written by Lars Kellogg-Stedman with non-critical changes in local.conf file.
Second part of article provides workaround making created nova-docker
instances and all devstack environment recoverable between reboots.
Reproducing the first part I also installed horizon launching nova-docker containers and assigning floating IPs clicking by mouse ( via admin login working with preinstalled Demo project )
Run as root ( post install ) to open way out for VMs
*************************************************************************
# iptables -t nat -A POSTROUTING -o eth0 -j
MASQUERADE*************************************************************************
$ sudo apt-get update
$ sudo apt-get -y install git git-review python-pip python-dev
$ sudo apt-get -y upgrade
$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
$ sudo sh -c "echo deb https://get.docker.com/ubuntu docker main \
> /etc/apt/sources.list.d/docker.list"
$ sudo apt-get update
$ sudo apt-get install lxc-docker
*********************************************
Update /etc/default/docker and setting:
*********************************************
DOCKER_OPTS='-G ubuntu'
#service docker restart
*******************************
Installing nova-docker
*******************************
This block is a subject to change as far as commits done
after e9dcf7e790e4df2f9025b19896173995a32692fc
in particular 85071220cbc3c1edb4a4c67db3e7060284f35c6b
will be tested as not disabling floating IPs.
$ git clone http://github.com/stackforge/nova-docker.git
$ cd nova-docker
$ git checkout e9dcf7e790e4df2f9025b19896173995a32692fc
$ sudo pip install .
***************************************************************************
UPDATE 03/12/2015 To get floating IPs working in meantime I have
***************************************************************************
$ git clone http://github.com/stackforge/nova-docker.git
$ cd nova-docker
$ git revert -m 1 661998214962d3e86063196bda0b3a619b7f4e26
$ sudo pip install .
************************************
UPDATE 03/13/2015
************************************
$ git clone http://github.com/stackforge/nova-docker.git
$ cd nova-docker
$ sudo pip install .
Seems to be working , however I've noticed strange issue with
# iptables -t nat -A POSTROUTING -o eth0 -j
MASQUERADEThis directive has a potential danger to lock your floating IPs, if you MASQUERADE a concrete sub-net which is providing floating IPs.
iptables -t nat -A POSTROUTING -o eth0 -j
*****************************Configuring devstack
*****************************
Now we're ready to get devstack up and running. Start by cloning the repository:
$ git clone https://git.openstack.org/openstack-dev/devstack
$ cd devstack
1. Create local.conf under devstack ( original version )
***************
local.conf
***************
[[local|localrc]]
ADMIN_PASSWORD=secret
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
SERVICE_TOKEN=super-secret-admin-token
VIRT_DRIVER=novadocker.virt.docker.DockerDriver
DEST=$HOME/stack
SERVICE_DIR=$DEST/status
DATA_DIR=$DEST/data
LOGFILE=$DEST/logs/stack.sh.log
LOGDIR=$DEST/logs
# The default fixed range (10.0.0.0/24) conflicted with an address
# range I was using locally.
FIXED_RANGE=10.254.1.0/24
NETWORK_GATEWAY=10.254.1.1
# Services
disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service horizon
disable_service tempest
# Introduce glance to docker images
[[post-config|$GLANCE_API_CONF]]
[DEFAULT]
container_formats=ami,ari,aki,bare,ovf,ova,docker
# Configure nova to use the nova-docker driver
[[post-config|$NOVA_CONF]]
[DEFAULT]
compute_driver=novadocker.virt.docker.DockerDriver
*****************************************************************************
My version of local.conf which allows define floating pool as you need,
a bit more flexible then original
*****************************************************************************
[[local|localrc]]
HOST_IP=192.168.1.57
ADMIN_PASSWORD=secret
MYSQL_PASSWORD=secret
RABBIT_PASSWORD=secret
SERVICE_PASSWORD=secret
FLOATING_RANGE=192.168.10.0/24
FLAT_INTERFACE=eth0
Q_FLOATING_ALLOCATION_POOL=start=192.168.10.150,end=192.168.10.254
PUBLIC_NETWORK_GATEWAY=192.168.10.15
SERVICE_TOKEN=super-secret-admin-token
VIRT_DRIVER=novadocker.virt.docker.DockerDriver
DEST=$HOME/stack
SERVICE_DIR=$DEST/status
DATA_DIR=$DEST/data
LOGFILE=$DEST/logs/stack.sh.log
LOGDIR=$DEST/logs
# The default fixed range (10.0.0.0/24) conflicted with an address
# range I was using locally.
FIXED_RANGE=10.254.1.0/24
NETWORK_GATEWAY=10.254.1.1
# Services
disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service horizon
disable_service tempest
# Introduce glance to docker images
[[post-config|$GLANCE_API_CONF]]
[DEFAULT]
container_formats=ami,ari,aki,bare,ovf,ova,docker
# Configure nova to use the nova-docker driver
[[post-config|$NOVA_CONF]]
[DEFAULT]
compute_driver=novadocker.virt.docker.DockerDriver
**************************************
Corresponding iptables entry
**************************************
# iptables -t nat -A POSTROUTING -o eth0 -j
MASQUERADEAt this point you are ready to run :-
$ ./stack.sh
*****************************************************************************
Attention skipping this step causes message "No hosts available"
when launching, either causes failure to launch nova-docker instances
in case of stack.sh rerun after ./unstack.sh
******************************************************************************
$ sudo cp nova-docker/etc/nova/rootwrap.d/docker.filters \
/etc/nova/rootwrap.d/
$ . openrc admin
For docker pull && docker save
$ . openrc demo
To launch instances
*********************************************************************************
Next issue , you have run `sudo ./unstack.sh` , rebooted box hosting devstack instance and OVS bridge "br-ex" came up with no IP no matter which one of local.conf has been used for ./stack.sh deployment.
Before running ./rejoin-stack.sh following actions have to be undertaken
*********************************************************************************
This version is supposed to work with second version of local.conf
PUBLIC_NETWORK_GATEWAY=192.168.10.15
sudo ip addr flush dev br-ex
sudo ip addr add 192.168.10.15/24 dev br-ex
sudo ip link set br-ex up
sudo route add -net 10.254.1.0/24 gw 192.168.10.15
******************************************************
Verify correct environment installed:-
******************************************************
ubuntu@ubuntu-System-Product-Name:~$ ifconfig
br-ex Link encap:Ethernet HWaddr de:64:4b:ba:a7:48
inet addr:192.168.10.15 Bcast:0.0.0.0 Mask:255.255.255.0
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:2186 errors:0 dropped:0 overruns:0 frame:0
TX packets:2649 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1801780 (1.8 MB) TX bytes:2194422 (2.1 MB)
br-int Link encap:Ethernet HWaddr b2:cf:54:c5:a0:49
inet6 addr: fe80::b007:79ff:fe87:4260/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:648 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:120474 (120.4 KB) TX bytes:648 (648.0 B)
br-tun Link encap:Ethernet HWaddr 3a:fb:71:08:1a:45
inet6 addr: fe80::899:bcff:fed6:8d8d/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:648 (648.0 B)
docker0 Link encap:Ethernet HWaddr 56:84:7a:fe:97:99
inet addr:172.17.42.1 Bcast:0.0.0.0 Mask:255.255.0.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth0 Link encap:Ethernet HWaddr 90:e6:ba:2d:11:eb
inet addr:192.168.1.37 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::92e6:baff:fe2d:11eb/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:64604 errors:0 dropped:0 overruns:0 frame:0
TX packets:37999 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:88470764 (88.4 MB) TX bytes:3455868 (3.4 MB)
eth1 Link encap:Ethernet HWaddr 00:0c:76:e0:1e:c5
inet6 addr: fe80::20c:76ff:fee0:1ec5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:239 errors:0 dropped:0 overruns:0 frame:0
TX packets:389 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:58024 (58.0 KB) TX bytes:75526 (75.5 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:30804 errors:0 dropped:0 overruns:0 frame:0
TX packets:30804 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:10921200 (10.9 MB) TX bytes:10921200 (10.9 MB)
ns44923080-eb Link encap:Ethernet HWaddr 9a:db:d0:5a:ad:02
inet6 addr: fe80::98db:d0ff:fe5a:ad02/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:621 errors:0 dropped:0 overruns:0 frame:0
TX packets:289 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:119156 (119.1 KB) TX bytes:55649 (55.6 KB)
ns9cb8e46e-35 Link encap:Ethernet HWaddr 6e:f3:23:93:b4:11
inet6 addr: fe80::6cf3:23ff:fe93:b411/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:637 errors:0 dropped:0 overruns:0 frame:0
TX packets:271 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:121878 (121.8 KB) TX bytes:52144 (52.1 KB)
tap44923080-eb Link encap:Ethernet HWaddr ee:b3:16:a3:f9:ed
inet6 addr: fe80::ecb3:16ff:fea3:f9ed/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:289 errors:0 dropped:0 overruns:0 frame:0
TX packets:621 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:55649 (55.6 KB) TX bytes:119156 (119.1 KB)
tap8897281a-3f Link encap:Ethernet HWaddr 9a:2a:eb:a5:3d:60
inet6 addr: fe80::982a:ebff:fea5:3d60/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2236 errors:0 dropped:0 overruns:0 frame:0
TX packets:3452 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1812589 (1.8 MB) TX bytes:2351741 (2.3 MB)
tap9cb8e46e-35 Link encap:Ethernet HWaddr 06:3c:cc:e5:30:4a
inet6 addr: fe80::43c:ccff:fee5:304a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:271 errors:0 dropped:0 overruns:0 frame:0
TX packets:637 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:52144 (52.1 KB) TX bytes:121878 (121.8 KB)
virbr0 Link encap:Ethernet HWaddr e2:93:d0:a0:2c:f6
inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
ubuntu@ubuntu-System-Product-Name:~$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 eth0
10.254.1.0 192.168.10.15 255.255.255.0 UG 0 0 0 br-ex
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
192.168.1.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0
192.168.10.0 0.0.0.0 255.255.255.0 U 0 0 0 br-ex
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
****************************************
At this point run you may run
****************************************
cd devstack ; ./rejoin-stack.sh
and it will bring your devstack environment back
********************************************************************
Actually, on Ubuntu 14.04 box doing this kind of testing
********************************************************************
root@ubuntu-P5Q3 :~# cat /etc/rc.local
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
ip addr flush dev br-ex ;
ip addr add 192.168.10.15/24 dev br-ex ;
ip link set br-ex up ;
route add -net 10.254.1.0/24 gw 192.168.10.15 ;
exit 0
*****************************************************************
Establishing access to public devstack net from LAN
*****************************************************************
Vncviewer started from Ubuntu VM with devstack environment installed
connecting to vncserver screen running on Ubuntu Rastasheep nova-docker instance
Run on Devstack Node
# Add route to LAN
$ sudo route add -net 192.168.1.0/24 gw 192.168.1.57
Run on LAN box
# Add route to devstack public network via HOST_IP
$ sudo route add -net 192.168.10.0/24 gw 192.168.1.57
where 192.168.1.57 HOST_IP on Devstack Node
192.168.10.0/24 devstack's public network
192.168.1.0/24 LAN address Vncviewer started from Ubuntu VM with devstack environment installed
connecting to vncserver screen running on Ubuntu Rastasheep nova-docker instance
Running Glassfish 4.1 nova-docker container on real Ubuntu 14.04 box
SQLDeveloper connection to Oracle XE database running inside nova-docker
container
Launching nova-docker container via CLI on real Ubuntu 14.04 box
ubuntu@ubuntu-P5Q3 :~/devstack$ nova boot --image rastasheep/ubuntu-sshd:latest --flavor m1.small UbuntuRST
+--------------------------------------+----------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| adminPass | n56arrfUdTLY |
| config_drive | |
| created | 2015-02-16T20:18:38Z |
| flavor | m1.small (2) |
| hostId | |
| id | 85acb8d4-2387-4a21-9b77-321480f03163 |
| image | rastasheep/ubuntu-sshd:latest (87956634-9708-4d63-8daf-cdd15d288d86) |
| key_name | - |
| metadata | {} |
| name | UbuntuRST |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | BUILD |
| tenant_id | 2f34beaaa0684e899f28c1b6fef521ac |
| updated | 2015-02-16T20:18:38Z |
| user_id | a78cae8feb1f40b081db787629a407af |
+--------------------------------------+----------------------------------------------------------------------+
ubuntu@ubuntu-P5Q3 :~/devstack$ nova list
+--------------------------------------+------------------+--------+------------+-------------+------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------------------+--------+------------+-------------+------------------------------------+
| 85acb8d4-2387-4a21-9b77-321480f03163 | UbuntuRST | ACTIVE | - | Running | private=10.254.1.6 |
| fc0a6180-d177-4f04-bdf6-382820c5f8da | derbyGlassfish41 | ACTIVE | - | Running |
| private=10.254.1.5, 192.168.10.152 |
+--------------------------------------+------------------+--------+------------+--------------
ubuntu@ubuntu-P5Q3 :~/devstack$ nova floating-ip-create
+----------------+-----------+----------+--------+
| Ip | Server Id | Fixed Ip | Pool |
+----------------+-----------+----------+--------+
| 192.168.10.153 | - | - | public |
+----------------+-----------+----------+--------+
ubuntu@ubuntu-P5Q3 :~/devstack$ nova floating-ip-associate UbuntuRST 192.168.10.153
ubuntu@ubuntu-P5Q3 :~/devstack$ ping -c 3 192.168.10.153
PING 192.168.10.153 (192.168.10.153) 56(84) bytes of data.
64 bytes from 192.168.10.153: icmp_seq=1 ttl=63 time=0.667 ms
64 bytes from 192.168.10.153: icmp_seq=2 ttl=63 time=0.274 ms
64 bytes from 192.168.10.153: icmp_seq=3 ttl=63 time=0.084 ms
--- 192.168.10.153 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.084/0.341/0.667/0.243 ms
ubuntu@ubuntu-P5Q3 :~/devstack$ ssh root@192.168.10.153
The authenticity of host '192.168.10.153 (192.168.10.153)' can't be established.
ECDSA key fingerprint is cf:f3:e5:fd:ce:d9:99:b6:79:2d:34:73:e8:a3:2e:21.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.10.153' (ECDSA) to the list of known hosts.
root@192.168.10.153's password:
root@instance-00000004:~# ps -ef
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 20:18 ? 00:00:00 /usr/sbin/sshd -D
root 5 1 0 20:22 ? 00:00:00 sshd: root@pts/0
root 7 5 0 20:22 pts/0 00:00:00 -bash
root 18 7 0 20:22 pts/0 00:00:00 ps -ef
root@instance-00000004:~# ifconfig
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
nsa7183e2e-09 Link encap:Ethernet HWaddr fa:16:3e:3d:0f:68
inet addr:10.254.1.6 Bcast:10.254.1.255 Mask:255.255.255.0
inet6 addr: fe80::f816:3eff:fe3d:f68/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2378 errors:0 dropped:12 overruns:0 frame:0
TX packets:1425 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2586320 (2.5 MB) TX bytes:132646 (132.6 KB)
*************************************************************
Login via qdhcp-namespace into UbuntuRST
*************************************************************
ubuntu@ubuntu-P5Q3 :~/devstack$ sudo ip netns exec qdhcp-c9e35028-bb1b-4141-b02b-9f35c7524dd2 ssh root@10.254.1.6
The authenticity of host '10.254.1.6 (10.254.1.6)' can't be established.
ECDSA key fingerprint is cf:f3:e5:fd:ce:d9:99:b6:79:2d:34:73:e8:a3:2e:21.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.254.1.6' (ECDSA) to the list of known hosts.
root@10.254.1.6's password:
Last login: Mon Feb 16 20:22:28 2015 from 192.168.10.15
root@instance-00000004:~# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=55 time=19.3 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=55 time=18.3 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=55 time=19.2 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=55 time=18.4 ms
References
1. https://gist.github.com/charlesflynn/5576114
