HowTO - FreeBSD 9.x Jails – vnet

Important: This article is not a “how-to” for jail and vnet; it is written to give guidance on how you can use /etc/jail.conf and VNET.
I do not claim it is 100% exhaustive, but I hope it will be useful to people who are already familiar with FreeBSD jails and want to use them with VIMAGE.

A Bit of Theory

What is a FreeBSD Jail

Jails enhance the traditional chroot concept in several ways.
With a standard chroot, processes are confined only within the filesystem they can see. All other system resources (users, running processes, the network subsystem) are shared between the chroot and the host OS.
Unlike chroot, a Jail isolates not only the filesystem but also running processes, network resources (VIMAGE/VNET or FIB), and system users.

What is VIMAGE (vnet)

VIMAGE is a kernel module for network virtualization. VIMAGE (or vnet) creates a “private,” more accurately an isolated, network stack.
It requires its own network interface, which can be moved into the Jail—making it visible only inside the jail and not to the host OS.

What is an EPAIR Interface

An epair is one half of a pair of virtual network interfaces connected by a virtual crossover “cable” (for example, epair1a ↔ epair1b).

Sample Topology


Internet
↓
FreeBSD host
↓ natd via rl0
bridge0 ── epair1a ↔ epair1b ── testA.jail.local (lo0, epair1b)
└─ epair2a ↔ epair2b ── testB.jail.local (lo0, epair2b)
└─ epair3a ↔ epair3b ── testC.jail.local (lo0, epair3b)
  • The “a” side (e.g. epair1a) is attached to bridge0.
  • The “b” side (e.g. epair1b) is placed inside the Jail.
  • Traffic flows: Jail ↔ epair1b ↔ epair1a → bridge0 → Internet (via NAT).

Note: With Jail+vnet the network stack is isolated. You can create other useful interfaces (loopback, VLANs, etc.), but the kernel still sees all interfaces, which imposes some limitations.

Configuration

1. FreeBSD Kernel

When recompiling the kernel, enable:

options         VIMAGE

2. Jail Environment

I prefer a separate ZFS dataset for each jail, for example:

/jails/testjail

You can also use a skeleton template or ezjail.

3. Jail World

After buildworld, install into your jail directory:

cd /usr/src
make installworld   DESTDIR=/jails/testjail
make distribution   DESTDIR=/jails/testjail
  • installworld copies the binaries.
  • distribution copies the base configuration files.

Main Configuration Files

/etc/rc.conf

cloned_interfaces="bridge0"
firewall_enable="YES"
firewall_script="/etc/ipfw.conf"
natd_enable="YES"
natd_flags="-f /etc/natd.conf"
jail2_enable="YES"
jail2_list="testjail"

/etc/ipfw.conf

#!/bin/sh
fwcmd="/sbin/ipfw -q"
eif="rl0"

${fwcmd} -f flush
# NAT rule
${fwcmd} add 65532 divert natd ip from any to any via ${eif}

/etc/natd.conf

interface     rl0
dynamic       yes
same_ports    yes

# Redirect port 80 to 10.10.15.100:80
redirect_port tcp 10.10.15.100:80 192.168.0.10:80

/etc/jail.conf

# --- Default Settings ---
vnet;                             # Enable VIMAGE/VNET
mount.devfs;                      # Mount /dev in each jail
exec.start = "ifconfig lo0 127.0.0.1/8";
exec.stop  = "/bin/sh /etc/rc.shutdown";

# --- Jail Settings ---
testjail {
    path            = /jails/testjail;
    host.hostname   = testjail.jail.local;
    jid             = 1;
    mount.fstab     = /etc/fstab.testjail.local;
    devfs_ruleset   = 5;

    exec.prestart  = "ifconfig epair1 create";
    exec.prestart += "ifconfig bridge0 addm epair1a up";
    exec.prestart += "ifconfig bridge0 alias 10.10.15.1/24";
    exec.prestart += "ifconfig epair1a up";

    vnet.interface = "epair1b";

    exec.start     = "ifconfig epair1b 10.10.15.100/24";
    exec.start    += "route add default 10.10.15.1";
    exec.start    += "/bin/sh /etc/rc";

    exec.poststop  = "ifconfig bridge0 deletem epair1a";
    exec.poststop += "ifconfig bridge0 -alias 10.10.15.1";
    exec.poststop += "ifconfig epair1a destroy";
}

Tip: You can use almost any variable from sysctl -a | grep jail inside /etc/jail.conf.

Understanding /etc/jail.conf

  • Anything outside of { } applies as default to all jails.
  • Anything inside { } is specific to that named jail.
  • You can have unlimited named blocks, one per jail.

Example Structure

# Default settings
default_setting_1;
default_setting_2;

jail1 {
    # local settings...
}

jail2 {
    # local settings...
}

Key Variables

  • exec.prestart — runs before the jail starts
  • exec.start — runs when the jail starts
  • exec.poststop — runs after the jail stops

The first assignment uses =, and subsequent additions use +=, whether in the default section or inside a jail block.

Starting and Stopping

  • Start:

    jail -c testjail

  • Stop:

    jail -r testjail
Tags: jails freebsd