Subscribe to the RSS Feed

 

FreeBSD 8 VIMAGE + epair howto

The following text is about to show you how to use the new feature of FreeBSD 8: VIMAGE in a multi-jail environment.

If you don’t know how to build your own custom kernel image, follow the detailed instructions of the corresponding FreeBSD Handbook chapter .

test# ifconfig epair create
epair0a
test# jail -c vnet name=tibi1 host.hostname=tibi1 path=/ persist
test# jls
   JID  IP Address      Hostname                      Path
     1  -               tibi1                         /
test# jail -c vnet name=tibi2 host.hostname=tibi2 path=/ persist
test# jls
   JID  IP Address      Hostname                      Path
     1  -               tibi1                         /
     2  -               tibi2                         /

So we have two instances and an epair device. Let’s see the interface list on the host.

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=3<RXCSUM,TXCSUM>
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet6 ::1 prefixlen 128
        inet 127.0.0.1 netmask 0xff000000
epair0a: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:c0:64:00:04:0a
epair0b: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:c0:64:00:05:0b

Both sides of the pair is in the host system. Put one end into one of your jails with the ifconfig vnet command and verify the results by running ifconfig inside your jail.

test# ifconfig epair0b vnet 1
test# jexec 1 ifconfig
lo0: flags=8008<LOOPBACK,MULTICAST> metric 0 mtu 16384
        options=3<RXCSUM,TXCSUM>
epair0b: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:c0:64:00:05:0b

OK, we have a layer 2 connection. Let’s add some IPs and run a ping test
test# jexec 1 ifconfig epair0b 192.168.11.2
test# ifconfig epair0a 192.168.11.1
test# ping 192.168.11.2
PING 192.168.11.2 (192.168.11.2): 56 data bytes
64 bytes from 192.168.11.2: icmp_seq=0 ttl=64 time=0.576 ms
64 bytes from 192.168.11.2: icmp_seq=1 ttl=64 time=0.081 ms
^C
--- 192.168.11.2 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.081/0.328/0.576/0.247 ms

It works!

Let’s do the same with your other jail


test# ifconfig epair1b vnet 2
test# jexec 2 ifconfig epair1b 192.168.11.3

Oh wait, these are completely different set of epair interfaces, you can’t use the same IP subnet on them. In order to mash them together on the host side, you have to make a bridge.
test# ifconfig bridge create
bridge0
test# ifconfig bridge0 addm epair0a addm epair1a up
test#

The commands above will create a new bridge interface, and add the host side of both epair interfaces to the bridge.
You can see it with ifconfig as well:
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=3<RXCSUM,TXCSUM>
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet6 ::1 prefixlen 128
        inet 127.0.0.1 netmask 0xff000000
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:c0:64:00:04:0a
        inet 192.168.11.1 netmask 0xffffff00 broadcast 192.168.11.255
epair1a: flags=8942<BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:c0:64:00:05:0a
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether a6:4b:75:2d:2b:9b
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 5 priority 128 path cost 14183
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 14183

Let’s put the host IP we set for epair0a earlier on the bridge interface instead and bring UP the host side of epair1. (Note: If you assign an IP to an interface, its state should automatically change to UP)

test# ifconfig epair0a -alias
test# ifconfig bridge0 192.168.11.1
test# ifconfig epair1a up
test# ifconfig bridge0
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether a6:4b:75:2d:2b:9b
        inet 192.168.11.1 netmask 0xffffff00 broadcast 192.168.11.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 5 priority 128 path cost 14183
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 14183

Running ping tests from the second jail, you can now ping your host and your other jail(s) too.

test# jexec 2 ping 192.168.11.1
PING 192.168.11.1 (192.168.11.1): 56 data bytes
64 bytes from 192.168.11.1: icmp_seq=0 ttl=64 time=0.193 ms
^C
--- 192.168.11.1 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.193/0.193/0.193/0.000 ms
test# jexec 2 ping 192.168.11.2
PING 192.168.11.2 (192.168.11.2): 56 data bytes
64 bytes from 192.168.11.2: icmp_seq=0 ttl=64 time=0.410 ms
64 bytes from 192.168.11.2: icmp_seq=1 ttl=64 time=0.089 ms
^C
--- 192.168.11.2 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.089/0.249/0.410/0.160 ms

Remember, now that you have separate networking stacks for each of your jails, the choice of topology is yours.

Posted 2009/12/06 01:56 by jos


Comments

  1. Dec 31, 10:59 AM

    xdsgrrr Says:

    How to do it this without burden of installing new jail for every routing context
    i see vimage become very hard comparing to first version i don’t want to install on my routers a bunch of x number of freebsd binaries a want only virtual routing context provided by vimage is this posible ?

  2. Feb 23, 05:57 AM

    tibor Says:

    Use the same path= for them as it is shown above

  3. Mar 12, 04:39 PM

    Vittorio Says:

    hi,
    how do I set my host’s rc.conf in order to have the jails configured automatically at boot?
    As far I know the /etc/rc.d/jail cannot handle the “-c vnet” flags…
    Am I wrong?
    in any case useful HOWTO
    Vittorio

  4. Mar 24, 09:08 PM

    jos Says:

    Quote from the article:
    “If you use rc.conf you could try adding the “vnet” parameter to your jail_x_flags variable for automatic start up.”

Leave a Comment

Add your comment. Preview then Submit.


Hidden


Textile Help