Virtual Firewall with Qemu ( ipcop / smoothwall )

Index

  1. Introduction
  2. Requirements
    1. My notes:
      1. Check your kernel
      2. Install Lunux Headers
      3. Build qemu
      4. Build kqemu
      5. make kqemu start at boot
  3. Network Changes
    1. Bring down Host network interface
    2. Edit interfaces
    3. Bring up Host Network interfaces
  4. Naming
  5. Edit interfaces for tap
  6. VDE is the Virtual Distributed Ethernet
  7. Working Notes
    1. Vservers
    2. Start uml images automatically
    3. Start qemu images automatically
    4. qemu firewall command line
  8. SMP System Notes
  9. Thank you

Introduction

Qemu, User-Mode-Linux and Xen have been used for years to consolidate equipment. This howto will explain what is needed to setup qemu as a firewall for other virtual machines. Two firewalls will be setup on the same machine. They will share a single network card to the internet “Red Zone” and they will provide a separate “Green Zones” for other virtual machines to use.

Note: I have tested this on Ubuntu, and Voyage Linux. Visit custom research papers for samples.

Requirements

  1. Debian based system

  2. Qemu .0.9.0 installed and working.

  3. Kqemu 1.3.0.pre11 installed and working

My notes:

This section will be expanded to include complete build information.

Check your kernel

cat /proc/version

Install Lunux Headers

apt-cache search linux-header
{{{

Install the headers that match your kernel.
{{{
apt-get install linux-headers-server

Build qemu

Edit the configure line to match your linux-headers. 

apt-get install build-essential linux-headers-386 libsdl1.2-dev zlib1g-dev checkinstall fakeroot bcc gcc-3.4 g++-3.4 nasm socat
wget http://bellard.org/qemu/qemu-0.9.0.tar.gz
wget http://bellard.org/qemu/kqemu-1.3.0pre11.tar.gz
tar -xzf qemu-0.9.0.tar.gz
cd qemu-0.9.0
./configure --prefix=/usr/local --cc=gcc-3.4 --host-cc=gcc-3.4 --kernel-path=/usr/src/linux-headers-2.6.15-28-686/
# as root
make install

Build kqemu

Edit the configure line to match your linux-headers. 

cd ..
tar -xzf ../kqemu-1.3.0pre11.tar.gz
cd kqemu-1.3.0pre11
./configure --prefix=/usr/local --cc=gcc-3.4 --host-cc=gcc-3.4 --kernel-path=/usr/src/linux-headers-2.6.15-28-686/
make 
#as root
make install

make kqemu start at boot

echo kqemu >>/etc/modules

Network Changes

Note: My isp provides me with a router / firewall with Nat. The gateway address is this box.

Bring down Host network interface

#as root
ifdown eth0

Edit interfaces

Edit /etc/network/interfaces 

#auto eth0
#iface eth0 inet static
#address 10.50.1.20
#netmask 255.255.255.0
#gateway 10.50.1.1

# DHCP only works with network cards no wifi
auto brRed
iface brRed inet static
address 10.100.1.20
network 10.50.1.0
netmask 255.255.255.0
broadcast 10.50.1.255
gateway 10.50.1.1
bridge_maxwait 5
bridge_ports eth0
bridge_fd 9
bridge_hello 2
bridge_maxage 12
bridge_stp off
post-up brctl addbr 87g-br
post-down brctl delbr 87g-br
post-up brctl addbr 86g-br
post-down brctl delbr 86g-br

Bring up Host Network interfaces

# as root
ifup brRed

At this point, we have replaced eth0 with brRed. The host system can access the internet from behind the ISP's router/ firewall.

ifconfig will show that brRed has replaced eth0 as the host interface . To display the other bridges use : 

brctl show

Naming

I try to keep my names short. I start with the public IP address, add a color code, and then the host. 

86g-br      # Green bridge interfaces
86g-www     # Green tap interface  for a web host
86g-mysql   # Green tap interface for a database
...etc

Edit interfaces for tap

Add one tap device for each vm you will be using. Each firewall has two tap interfaces.

edit /etc/network/interfaces. 

#auto 87r-fw
iface 87r-fw inet static
address xxx.xxx.xxx.87
netmask 255.255.255.248
gateway xxx.xxx.xxx.88
post-up brctl addif brRed 87r-fw
post-down brctl delif brRed 87r-fw

#auto 87g-fw
iface 87g-fw inet static
address 192.168.0.20
netmask 255.255.255.0
post-up brctl addif 87g-br 87g-fw
post-down brctl delif 87g-br 87g-fw

#auto 87g-ubun
iface 87g-ubun inet static
address 192.168.1.21
netmask 255.255.255.0
post-up brctl addif 87g-br 87g-ubun
post-down brctl delif 87g-br 87g-ubun


#auto 86r-fw
iface 86r-fw inet static
address xxx.xxx.xxx.86
netmask 255.255.255.248
gateway xxx.xxx.xxx.88
post-up brctl addif brRed 86r-fw
post-down brctl delif brRed 86r-fw

#auto 86g-fw
iface 86g-fw inet static
address 192.168.1.20
netmask 255.255.255.0
post-up brctl addif 86g-br 86g-fw
post-down brctl delif 86g-br 86g-fw

#auto 86g-ubun
iface 86g-ubun inet static
address 192.168.1.21
netmask 255.255.255.0
post-up brctl addif 86g-br 86g-ubun
post-down brctl delif 86g-br 86g-ubun

VDE is the Virtual Distributed Ethernet

http://wiki.virtualsquare.org/index.php/VDE_Basic_Networking

Working Notes

This need a lot of clean up from this point down. I plan to rebuild on of my production boxes and clean up this how to. At his point I have picked qemu as the base for most of my vservers. I like uml better but the image build process is to hard to reverse. I want to build my server on real hardware and move them to a guest for production. If the Hosts system hard drive fails, the next drive in the system will boot and become the hosts system.

qemu will do this.

Vservers

create a path to store uml, qemu, and xen images. 

mkdir -p /vservers/qemu
mkdir    /vservers/uml
mkdir  /vservers/xen.

Note: I'm not currently running xen on my hosts. I do not plan to go into any detail on xen.

Start uml images automatically

Edit /etc/init.d/uml 

# http://deb.riseup.net/miscellaneous/uml/#_etc_initd_uml
# I'm not sure what I started with.  If this is your code please add your name and a link.
 if [  -e /etc/uml-guests ]; then
        . /etc/uml-guests
      else
         #GUESTS='host1 host2'
        echo "no /etc/uml-guests"
        exit 0
      fi

UML_USER=''
BasePath='vservers/uml'
if [[ ! -z $2 ]]; then
        GUESTS=$2
fi
 
case "$1" in
  start)
    for host in $GUESTS; do
      if [[  -e /$BasePath/$host ]]; then
                echo -n "Starting $host..."
        cd /$BasePath/$host/
        su $UML_USER -c "screen -dmS $host ./$host > /dev/null" >/dev/null
        echo "  done"
      else
        echo "$host failed to start"
      fi
    done
  ;;   
  stop)
   for host in $GUESTS; do
      echo -n "Asking  $host to stop..."
      if [[ -e /$BasePath/$host/ ]]; then
        su $UML_USER -c " uml_mconsole $host exec /sbin/init 0"  >/dev/null

        echo "  done.";
      else
        echo "  error: uml_mconsole failed to init 0"
      fi
    done
    sleep 30  # make sure your guest have time to come do clean.
  ;;
  restart)
     echo "not supported";
  ;;
  *)
    echo "Usage: $0 {start|stop} {machine}" >&2
        exit 1
  ;;
esac
exit 0

edit /etc/uml-guests 

# a list of guests to start
#GUESTS='host1 host2'

Start qemu images automatically

# http://deb.riseup.net/miscellaneous/uml/#_etc_initd_uml # I'm not sure what I started with. If this is your code please add your name and a link. # aliguori - qemu irc thank you for the socat information. # dalias - screen irc thank you for working a dead end with me.

UML_USER= BasePath='vservers/qemu' if [[ ! -z $2 ]]; then

fi

case "$1" in

esac exit 0

edit /etc/qemu-guests 

# a list of guests to start
#GUESTS='host1 host2'

I found this script at some point. If this is your, please add your name.

qemu firewall command line

qemu -kernel-kqemu   -boot c  -hda hda1.img -m 128 -net nic,vlan=0,macaddr=00:16:3e:00:00:00 -net tap,vlan=0,ifname=86r-fw,script=/etc/qemu-ifup-brRed  
-net nic,vlan=1,macaddr=00:16:3e:00:00:01 -net tap,vlan=1,ifname=86g-fw,script=/etc/qemu-ifup-86g-br -hdb hd256M.img

qemu-ifup-Red 

#!/bin/sh
#sudo -p "Password for $0:" /sbin/ifconfig $1 172.20.0.1
sudo /sbin/ifconfig $1 0.0.0.0 promisc up
sudo /usr/sbin/brctl addif brRed $1

/etc/qemu-ifup-86g-br 

#!/bin/sh
#sudo -p "Password for $0:" /sbin/ifconfig $1 172.20.0.1
#!/bin/sh
sudo /sbin/ifconfig $1 0.0.0.0 promisc up
sudo /usr/sbin/brctl addif 86g-br $1

SMP System Notes

When Running qemu on a SMP based host, use one of the following options:

Option 1 On the host run "taskset -c X qemu ...etc" X is equal to the processor you will run qemu on. qemu will not be move from one cpu to the other cpu. qemu will live on processor X for all time. If you have more then one qemu guest, make sure start them on different processors.

Option 2 On the guest use "notsc" as a kernel option. ( I did not test this. ) 

#grub menu.lst
title           Ubuntu, kernel 2.6.20-15-generic
root            (hd0,0)
kernel          /boot/vmlinuz-2.6.20-15-generic root=UUID=642877d7-5754-4640-8c33-0713a7fe18fe ro quiet notsc
initrd          /boot/initrd.img-2.6.20-15-generic
quiet
savedefault

Thank you andrzej zaborowski for your help on this issue.

why http://lists.gnu.org/archive/html/qemu-devel/2007-03/msg00652.html qemu forum http://qemu-forum.ipi.fi/viewtopic.php?t=3714&highlight=taskset

For google qemu race condition kqemu ubuntu 7.4 feisty lockup hang Linux

Thank you

I have read many howto on qemu and UML( user mode linux).

Thank you for all of your help,

Nicholas A. Schembri State College PA USA

Note: This is my rant. I'm not starting a war. The developers are much better at this stuff. They make the best products in the world. I would be lost without them. Thank you qemu,User mode Linux, and xen.

I like UMl's monitor. UMl has better memory control. You can boot a guest and then reduce the memory allocated. But UMl need to be able to install straight from a distro CD ISO or boot from a qemu/vmware image file.

KQemu needs to add SMP support so that qemu -smp X balances the process on the host system better. It might be nice if Kqemu had the memory allocation option that UMl has. Qemu needs to allow a multiple monitor / vm control program to access a running guest. This will allow better control for remote admins. I'm not sure about checkpointing. I see a lot of good things you could do but the image files are an issue. If I'm going to checkpoint a system and move it over the net. I'm not going to push a 10G file to do it. High Availability makes more since.

As for Xen.... I'm not sure. I have not worked with it enough. I like some the util's.

Virtual Firewall with qemu (last edited 2009-10-07 17:45:52 by louise)