Tuesday, April 28, 2015

Setup Nova-Docker driver && Openstack Kilo on Ubuntu 14.04 in devstack environment recoverable between reboots

  Step by step instruction for  setting up  Nova-Docker driver && Openstack Kilo on Ubuntu 14.04 in devstack environment recoverable between reboots. Also routing across LAN is described for remote access to Devstack (stack.sh) public network. I've tried to cover all known as of time of writing problems preventing ./rejoin-stack.sh from successful run. However , I cannot issue any warranty that tomorrow another daemon wouldn't rejoin stack instance after reboot. The only way for safe solution would be RDO Kilo Release expected in may of 2015 (for myself of course). This post is written with major concern of successful loading by Kilo Nova-Docker Driver in development environment.

Proceed as follows :-

$ sudo apt-get update
$ sudo apt-get -y install git git-review python-pip python-dev
$ sudo apt-get -y upgrade

$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
$ sudo sh -c "echo deb https://get.docker.com/ubuntu docker main  \
   > /etc/apt/sources.list.d/docker.list"
$ sudo apt-get update
$ sudo apt-get install lxc-docker

*********************************************
Update  /etc/default/docker and setting:
*********************************************
DOCKER_OPTS='-G ubuntu'

#service docker restart

Logout from root and verify

ubuntu@ubuntu-WKS:~$ docker -v
Docker version 1.6.0, build 4749651

*******************************
Installing nova-docker
*******************************
$ git clone http://github.com/stackforge/nova-docker.git
$ cd nova-docker
$ sudo pip install .


iptables -t nat -A POSTROUTING -o eth0 -j
*****************************
Configuring devstack
*****************************

Now we're ready to get devstack up and running. Start by cloning the repository:

$ git clone https://git.openstack.org/openstack-dev/devstack
$ cd devstack
$ git checkout -b kilo origin/stable/kilo

******************************************
Create local.conf under devstack
******************************************
[[local|localrc]]
HOST_IP=192.168.1.142
ADMIN_PASSWORD=secret
MYSQL_PASSWORD=secret
RABBIT_PASSWORD=secret
SERVICE_PASSWORD=secret
FLOATING_RANGE=192.168.12.0/24
FLAT_INTERFACE=eth0
Q_FLOATING_ALLOCATION_POOL=start=192.168.12.150,end=192.168.12.254
PUBLIC_NETWORK_GATEWAY=192.168.12.15

SERVICE_TOKEN=super-secret-admin-token
VIRT_DRIVER=novadocker.virt.docker.DockerDriver

DEST=/opt/stack
SERVICE_DIR=$DEST/status
DATA_DIR=$DEST/data
LOGFILE=$DEST/logs/stack.sh.log
LOGDIR=$DEST/logs

# The default fixed range (10.0.0.0/24) conflicted with an address
# range I was using locally.
FIXED_RANGE=10.254.1.0/24
NETWORK_GATEWAY=10.254.1.1

# Services
disable_service n-net
enable_service n-cauth
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service horizon
disable_service tempest

# Introduce glance to docker images
[[post-config|$GLANCE_API_CONF]]
[DEFAULT]
container_formats=ami,ari,aki,bare,ovf,ova,docker

# Configure nova to use the nova-docker driver
[[post-config|$NOVA_CONF]]
[DEFAULT]
compute_driver=novadocker.virt.docker.DockerDriver

**************************************
Corresponding iptables entry
**************************************
# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

At this point you are ready to run :-

$ ./stack.sh

*****************************************************************************
Attention skipping this step causes message "No hosts available"
when launching, either causes failure to launch nova-docker instances
in case of stack.sh rerun after ./unstack.sh
******************************************************************************

$ sudo cp nova-docker/etc/nova/rootwrap.d/docker.filters \
  /etc/nova/rootwrap.d/

$ .   openrc admin

For docker pull && docker save
************************************************************************
To enable security rules and launch NovaDocker Container :-
************************************************************************

$ cd dev*

$ . openrc demo 

$ neutron security-group-rule-create --protocol icmp \
  --direction ingress --remote-ip-prefix 0.0.0.0/0 default

$ neutron security-group-rule-create --protocol tcp \
  --port-range-min 22 --port-range-max 22 \
  --direction ingress --remote-ip-prefix 0.0.0.0/0 default

$ neutron security-group-rule-create --protocol tcp \
  --port-range-min 80 --port-range-max 80 \
  --direction ingress --remote-ip-prefix 0.0.0.0/0 default

Uploading docker image to glance

$ . openrc admin
$  docker pull rastasheep/ubuntu-sshd:14.04
$  docker save rastasheep/ubuntu-sshd:14.04 | glance image-create --is-public=True   --container-format=docker --disk-format=raw --name rastasheep/ubuntu-sshd:14.04

Launch new instance via uploaded image :-
$ . openrc demo
$  nova boot --image "rastasheep/ubuntu-sshd:14.04" --flavor m1.tiny
    --nic net-id=private-net-id UbuntuDocker
 

Enable security rules via dashboard  :-



*******************************************************************************
 You have run `sudo ./unstack.sh` , rebooted box hosting devstack  instance and OVS bridge "br-ex" came up with no IP no matter which one of local.conf has been used for ./stack.sh deployment.
Before running ./rejoin-stack.sh following actions have to be undertaken
(just add to /etc/rc.local)
*******************************************************************************

    sudo ip addr flush dev br-ex
    sudo ip addr add 192.168.12.15/24 dev br-ex

    sudo ip link set br-ex up
    sudo route add -net 10.254.1.0/24 gw 192.168.12.15



You also might experience problems with rejoining cinder-volume daemon, in this case copy to &HOME folder corresponding files  and run before ./rejoin-stack.sh from your $HOME ./cinder.sh


ubuntu@ubuntu-WKS:~$ cat cinder.sh
cp stack-volumes-default-backing-file /opt/stack/data/stack-volumes-default-backing-file
cp stack-volumes-lvmdriver-1-backing-file /opt/stack/data/stack-volumes-lvmdriver-1-backing-file
sudo losetup /dev/loop0 /opt/stack/data/stack-volumes-default-backing-file
sudo losetup /dev/loop1 /opt/stack/data/stack-volumes-lvmdriver-1-backing-file


*******************************************************************************
To access NovaDocker instances  running within  stack (devstack) AIO instance on Ubuntu 14.04 host 192.168.1.142  from another boxes located on the same
office LAN having address 192.168.1.0/24 manage as follows :-
*******************************************************************************

***************************
Run on Devstack Node
***************************
# Add route to LAN
$ sudo route add -net  192.168.1.0/24 gw 192.168.1.142

**************************
Run on LAN's box
**************************
# Add route to devstack public network  via HOST_IP
$ sudo route add -net 192.168.12.0/24 gw 192.168.1.142

where 192.168.1.142  HOST_IP of  Devstack Node running stack instance
192.168.12.0/24  is  Devstack's public  network. 192.168.1.0/24 is  LAN address

Routing table on Devstack node should look like :-

ubuntu@ubuntu-WKS:~$ route -n

Kernel IP routing table

Destination      Gateway              Genmask         Flags  Metric  Ref    Use Iface
0.0.0.0              192.168.1.1         0.0.0.0               UG     0          0        0  eth0
10.254.1.0        192.168.12.150   255.255.255.0    UG    0          0        0  br-ex
172.17.0.0        0.0.0.0                 255.255.0.0        U       0          0        0  docker0
192.168.1.0      192.168.1.142     255.255.255.0    UG     0         0        0  eth0
192.168.1.0      0.0.0.0                 255.255.255.0     U      1          0        0  eth0
192.168.12.0    0.0.0.0                 255.255.255.0     U      0          0        0  br-ex

   


References

http://blog.oddbit.com/2015/02/11/installing-novadocker-with-devstack/