
Vnets: Virtual Networks for Virtual Machines

Mike Wray <mike.wray@hp.com>

2005/12/13

0) Introduction
---------------

Vnets provide virtual private LANs for virtual machines.
This is done using bridging and multipoint tunneling. A virtual interface
on a vnet can only see other interfaces on the same vnet - it cannot
see the real network, and the real network cannot see it either.

Virtual interfaces on the same vnet can be on the same machine
or on different machines, they can still talk. The hosting machines
can even be on different subnets if you configure vnet forwarding,
or have multicast routing enabled.


1) Installing vnet support
--------------------------

Assuming the code has been installed (make install in the parent directory),
configure xend to use 'network-vnet' instead of the default 'network' to
start up networking. This just loads the vnet module when networking starts.

In /etc/xend/xend-config.sxp:

Configure the network script:

(network-script        network-vnet)

Restart xend.

Alternatively insert the vnet module using 'vn insmod',
preferably before xend starts.

2) Creating vnets
-----------------

Xend already implements commands to add/remove vnets and
bridge to them. To add a vnet use

xm vnet-create <vnet config file>

For example, if vnet97.sxp contains:

(vnet (id 97) (bridge vnet97) (vnetif vnif97) (security none))

do

xm vnet-create vnet97.sxp

This will define a vnet with id 97 and no security. The bridge for the
vnet is called vnet97 and the virtual interface for it is vnif97.
To add an interface on a vm to this vnet simply set its bridge to vnet97
in its configuration.

In Python:

vif="bridge=vnet97"

In sxp:

(dev (vif (mac aa:00:00:01:02:03) (bridge vnet97)))

By default vnets use udp encapsulation, but if you use etherip encapsulation
you will also have to reduce the MTU of the corresponding
device in the domain (because of the tunneling). Reducing the MTU may improve
performance for udp encapsulation, but is not necessary.

For example, for eth0 (in the domain, not dom0) use

ifconfig eth0 mtu 1400

or, better, put

MTU=1400

in /etc/sysconfig/network-scripts/ifcfg-eth0. You may also have to change or remove
cached config files for eth0 under /etc/sysconfig/networking.

Once configured, vnets are persistent in the xend database.
To remove a vnet use

xm vnet-delete <vnet id>

To list vnets use

xm vnet-list

To get information on one or more vnet ids use

xm vnet-list <vnet id>...

You can also manage vnets using the vn utility which talks
directly to the vnet implementation. The source is in ../scripts/vn
and is installed in /usr/sbin/vn.

3) Troubleshooting
------------------

The vnet module should appear in 'lsmod'.
If a vnet has been configured it should appear in the output of 'xm vnet-list'.
Its bridge and interface should appear in 'ifconfig'.
It should also show in 'brctl show', with its attached interfaces.

You can 'see into' a vnet from dom0 if you put an IP address on the bridge.
For example, if you have vnet97 and a vm with ip addr 10.0.0.12 connected to it,
then

ifconfig vnet97 10.0.0.20 up

should let you ping 10.0.0.12 via the vnet97 bridge.

4) Examples
-----------

These assume a vnet with a bridge 'vnet97' has been created.

Here's the full config for a vm on vnet 97, using ip addr 10.0.0.12:

(vm
 (name dom12)
 (memory '64')
 (cpu '1')
 (console '8502')
 (image
  (linux
   (kernel /boot/vmlinuz-2.6-xenU)
   (ip 10.0.0.12:1.2.3.4::::eth0:off)
   (root /dev/sda1)
   (args 'rw fastboot 4')
  )
 )
 (device (vbd (uname phy:hda2) (dev sda1) (mode w)))
 (device (vif (mac aa:00:00:11:00:12) (bridge vnet97)))
)

If you run another vm on the same vnet:

(vm
 (name dom11)
 (memory '64')
 (cpu '1')
 (console '8501')
 (image
  (linux
   (kernel /boot/vmlinuz-2.6-xenU)
   (ip 10.0.0.11:1.2.3.4::::eth0:off)
   (root /dev/sda1)
   (args 'rw fastboot 4')
  )
 )
 (device (vbd (uname phy:hda3) (dev sda1) (mode w)))
 (device (vif (mac aa:00:00:11:00:11) (bridge vnet97)))
)

the vms should be able to talk over the vnet. Check with ping.
If they are both on the same machine the connection will simply
be the vnet97 bridge, if they are on separate machines their
packets will be tunneled in udp (or etherip). They should be able to
see each other, but not the real network.


