Gerf.Org just switched to Xen. It’s
running in a
domU on hardware that is massively more powerful than it used
to be. The original hardware (until about 3 years ago) was a 333Mhz box with
20gigs of disk-space. Since then it has been running on newer hardware.
However, the hardware was flakey and getting flakier.
So it became time to upgrade to some new hardware. Xen was a natural choice
because I can remotely power off a
domU and it allows me to move my print
server, firewall, etc. all into one box. I only have to manage one
array and it uses less power. Plus it’s cool.
Of course, it’s not really his fault. Xen 3.0 hit beta right before he published it; The new networking stuff wasn’t really in place yet and certainly wasn’t ready for release yet.
Anyway, the Xen Wiki doesn’t help much either; it doesn’t give any examples and makes things that should be simple, complicated.
As you read this, remember that I’m using Xen 3.2.0 and Xen Tools 3.8.4 as ships with Ubuntu Hardy Heron 8.04. Xen is a rapidly changing project and this doesn’t apply to older versions and I doubt it applies 100% to newer versions.
The goal here is to create three bridges: LAN, WAN and DMZ. The hardware in
question has two NICs: one facing internal and one facing external. The DMZ
will be on a dummy NIC that is entirely virtual. This was I can host
external, internal and quarantined systems all as
domUs in the Xen
The network-bridge script is used to create bridges. You can think of a
bridge as a virtual network hub. In Xen, the bridge appears as a device in
lo does. This bridge can have multiple
hooked up to it, in addition to a real physical NIC.
This is implemented via the
network-bridge script. You can pass in various
arguments to control how it creates the bridge. You can also just do it all
yourself, but I’m not interested in that. Too much work.
The basic usage is something like this (from a
(network-script "network-bridge netdev=eth1 bridge=foo")
peth1(it prepends a ‘p’ to the name)
- Creates a device in
You would then connect to it in your
domU’s cfg file by adding
vif = [ 'ip=192.168.0.10,mac=00:16:3E:aa:bb:cc,bridge=foo' ]
So this new zen would be connected to this bridge. Packets would be routed
out foo and through
peth1 (formally known as
What you can do now, is replace
network-bridge (which is in
/etc/xen/scripts, btw) with a script of our own. I’m calling it
#!/bin/bash dir=$(dirname "$0") "$dir/network-bridge" "$@" vifnum=0 netdev=eth0 bridge=lan "$dir/network-bridge" "$@" vifnum=1 netdev=eth1 bridge=wan "$dir/network-bridge" "$@" vifnum=2 netdev=dummy0 bridge=dmz # EOF
I’m not sure the
vifnum is needed, but I don’t think it can hurt. I’m using
dummy0 for the dmz because the dmz is entirely located inside the xen box.
None of that traffic should escape my xen box.
xend-config.sxp needs to be modified as well:
Not very tough.
To use this in a
.cfg file is just like above.
A simple externally facing domU:
vif = [ 'ip=192.168.0.10,mac=00:16:3E:aa:bb:cc,bridge=wan' ]
An internally facing
vif = [ 'ip=192.168.0.10,mac=00:16:3E:aa:bb:dd,bridge=lan' ]
vif = [ 'ip=192.168.0.10,mac=00:16:3E:aa:bb:11,bridge=wan', 'ip=192.168.0.10,mac=00:16:3E:aa:bb:22,bridge=lan', 'ip=192.168.0.10,mac=00:16:3E:aa:bb:33,bridge=dmz', ]
I’ve left the ip addresses all the same because I’m too lazy to come up with
example ones. You can also specify
dhcp as well.
I hope that helps. At the very least, I’ve documented what I did so that when I forget all this in 6 months I can just look here. ;-)
Let me just say “thank you” for this. I’ve been so hopelessly confused by various recipes I’ve read to do this, but I could follow yours and now have my ideal scenario up and running which has elluded me for three weeks now!
If it could be improved at all, I would perhaps spend those extra minutes and make up some more appropriate IP addresses as I found thinking of it that way made it easier to envisage which machines were on one network and which on another.
I like your article on Xen Networking, thanks!
Assuming i duplicate and rename the original file “network-bridge”, to “my-network-bridge” is the following code all thats required for that file, or is it inserted into the code somewhere?
dir=$(dirname “$0”) “$dir/network-bridge” ”$@” vifnum=0 netdev=eth0 bridge=lan “$dir/network-bridge” ”$@” vifnum=1 netdev=eth1 bridge=wan “$dir/network-bridge” ”$@” vifnum=2 netdev=dummy0 bridge=dmz
Many thanks in advance
I’m not sure what you’re asking. You don’t need to duplicate the network-bridge file, you should just make a new one. In my case, I have the three networking interfaces that I’m using. If you aren’t using a DMZ then you can delete that.
it was a good idea to post your howto here !
I have followed this setup but I cant seem to implement it on hvm domu. I have 3 nics but when i run the domu it recognises the nics but cannot ping out.
Hmm… I dunno much about HVM, so I don’t know how that may impact anything. Maybe you have a different (newer or older) versions of Xen?
I am using xen 3.3.1 Debian Lenny AMD64.
For Debian Lenny I needed to just add the lines in network-bridge-custom like this exactly :) :-
dir=$(dirname “$0”) “$dir/network-bridge” ”$@” netdev=eth0 “$dir/network-bridge” ”$@” netdev=eth1 “$dir/network-bridge” ”$@” netdev=eth2
Now all my routes is good. In the vm.cfg just use ethX and not xenbro :) and the dom0 is accessible from all interfaces so your ifup script is helpful to isolate only one interface.
hehe sorry i just copied and pasted this I made in another site lol