VIMAGE: vnet, epair and lots of jails on bridgeX - routing

[O. Hartmann]

Hello,

I fight with the following problem without any kind of success and I need some
help and/or advice.

We are running several CURRENT and 11.1-RELENG-p6 boxes. CURRENT is at the most
recent version as of today.

VIMAGE is compiled in into all kernels.
IPFW is compiled into all kernels and is the one and only firewall used.
On CURRENT, the host's ipfw is set to "OPEN" (using the rc.-scripts so far). By
convention, I address the host running the kernel by "host".

Every jail is created/configured with its own "vnet" cloned network stack
(vnet=new).

All hosts do have at least three physical NICs. The host itself is supposed to
be member of the "friendly" network via a dedicated NIC. The two remaining NICs
are split into fractions belonging to an "hostile" network on which I'd like to
place exposed jails (for now), and to the "friendly" network, on which also
jails will be hosted, but via a dedicated NIC.

Inbetween those two networks, the host will have a third, intermediate,
network, call it the "service" network.

The following will be true for ALL jails created, including the host itself:

net.link.bridge.pfil_member=0
net.link.bridge.pfil_bridge=0
net.link.bridge.pfil_onlyip=0

First, I clone/create three bridge(4) devices, bridge0 (considered to be the
"glue" between the "service" jails), bridge1 (considered to be the glue between
the jails on the friednly network side) and bridge2, which is the glue between
the jails on the hostile side. bridge1 has eth1 as a member, which provides the
physical access to the friendly network, eth2 is member of bridge2, which
provides access to the hostile network.

By convention, when creating epair(4), the a-portion belongs to the jail itself
and is assigned with an IPv6 address. The b-portion of the epair(4) is member
of its bridge according to its realm (friendly, service or hostile network).

Additionally, there is a special jail, the router, which has three epair(4)
devices, the b-portion of the epair is member of the appropriate bridge(4) and
this router jail has static routes assigned, pointing to the appropriate
epairXXXa that is suppoesd to be the link into the correct bridge/network. IPFW
is set to open on this jail (for now). On this special
jail it is set: net.inet.ip.forwarding=1.

I hope, the topology is clear so far. All epairs or epair endpoints as well as
the bridges are UP! Double checked this.

Jails on bridge0 (service net) have IPs in the range 10.10.0.0/24, the
b-portion of the routing jail's epair is member of bridge0, as described above,
and the a-portion of the epair has IP 10.10.0.1. Default route on each jeail
on bridge0 is set to 10.10.0.1 accordingly.

Consider a similar setup on the other jails on the friendly and hostile
network, except the fact that their bridges do have a physical NIC to which
they may have access to a real network.

The setup might not be ideal and/or applicable for the purpose of separartion
of networks virtually, but that shouldn't be the subject here. More important
is that I assume that I haven't understood some essentials, because the setup
doens't work as expected. Furthermore, it behaves on FreeBSD 11.1-RELENG-p6
sometimes completely unpredictable - but in that special case, I think I ran
IPFW on the host as "WORKSTATION" and dynamic rules may play an important role
here. But focussing on the CURRENT box, the host's IPFW is set to OPEN.

With jexec -l hostA I gain access to host A on the "service" bridge0 and I
want to ping its neighbour, hostB, on the same bridge and in the same net. It
doesn't work! From the routing jail, I CAN NOT ping any host on bridge0. The
routing jail has these network settings:

[... routing jail ...]
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet 127.0.0.1 netmask 0xff000000
groups: lo
[epair to bridge0 - service net]
epair4000a: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:57:d0:00:07:0a
inet 10.10.0.1 netmask 0xffffff00 broadcast 10.10.0.255
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair
[epair to bridge1, friendly net]
epair4001a: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:57:d0:00:09:0a
inet 192.168.11.1 netmask 0xffffff00 broadcast 192.168.11.255
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair
[epair to bridge2, hostile net]
epair4002a: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:57:d0:00:0b:0a
inet 10.10.10.1 netmask 0xfffffc00 broadcast 10.10.10.255
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair

routing:
netstat -Warn
Routing tables

Internet:
Destination Gateway Flags Use Mtu Netif Expire
10.10.0.0/24 link#2 U 11 1500 epair4000a
10.10.0.1 link#2 UHS 4 16384 lo0
10.10.10.0/24 link#4 U 210 1500 epair4002a
10.10.10.1 link#4 UHS 44 16384 lo0
127.0.0.1 link#1 UH 0 16384 lo0
192.168.11.0/24 link#3 U 9 1500 epair4001a
192.168.11.1 link#3 UHS 0 16384 lo0

Consider a jail hostCC on bridge2 in the hostile network, IP 10.10.10.128.
I can ping that jail, although it has conceptionally the very same setup as the
unreachable jails on bridge0!

It is weird. On bridge0, no jail can be pinged, it looks like the ethernet is
somehwo down on that bridge. I would expect to ping each host member of the
very same bridge! On 11.1-RELENG-p6, there are other weird issues, I was able
to ping those jails, even ssh to them, but that vanished after several
restarts of the jails system (each bridge, epair is created by jail.conf and
destroyed after the jails has been deactivated and doing so a considerable
amount brings down the FreeBSD 11.1-RELENG-p6 host verys successfully - it
crashes!).

So, since VIMAGE is now default in CURRENT's GENERIC, I consider its
functionality at least "predictable", but I fail somehow here.

Does someone have a deeper insight or realise the mistake I'm celebrating here?

Thanks in adavnce,

Oliver

[O. Hartmann]

Am Fri, 09 Feb 2018 16:43:17 +0000

Am Thu, 8 Feb 2018 09:31:15 +0100
Is this problem to trivial?

I read through it yesterday and found myself in the position that I need
a whiteboard or paper and pencil or an ASCII art of your situation. But
by the time I made it to the question I was basically lost. Could you
massively simplify this and maybe produce the ASCII art?
/bz
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-current

All right.

I'm not much of an artist and at this very moment, I haven't much experience with neat
ASCII art tools. But I'll provide a sketch later, but I also will simplify the situation.

Consider three "vswitches", basically based on the creation of bridges, bridge0, bridge1,
bridge2. Create at least three individual vnet-jails attached to each vbridge. Those
jails have epair pseudo devices. The jail itself owns the "a-part" of the epair and the
b-part is "member of the bridge". Each jail's epairXXXa has an IP assigned of the network
the vswitch is part of. I mention a- and b-part of the epair here, because I thought it
could matter, but I think for symmetry reasons it doesn't.

Now consider a further, special jail. This jail is supposed to have three epair devices,
each one is reaching into one of the vbridges. This jail is the router/routing jail.
Later, this jail should filter via IPFW the traffic between the three vbridges according
to rules, but this doesn't matter here, beacuase the basics are not working as expected.

Now the problems. It doesn't matter on which jail of the three vswitches I login, the
moment a vbridge has more than two member epairs (one is alway member of the routing
jail, now consider a database jail and a webserver jail), pinging each jail or the
routing jail fails. It works sometimes for a couple of ICMP packets and then stops.

If each vbridge has only one member jail, I have NO PROBLEMS traversing accordingly to
the static routing rules from one vbridge to any other, say from vbridge1 to vbridge0 or
vbridge2 and any permutation of that.

The moment any of the bridges gets an additional member epair interface (so the bridge
has at least three members including the on reaching into the virtual router jail) the
vbridge seems to operate unpredictable (to me). Pinging jails memeber of that vbridge
are unreachable.

Technical information:

The kernel has options IPFIREWALL, VIMAGE. The host's ipfw (kernel) declines packets by
default. Each jail is configured to have ipfw "open".

Thanks for the patience.

Kind regards,

O. Hartmann

[Marko Zec]

On Sat, 10 Feb 2018 08:52:21 +0100

Post by O. Hartmann
Am Fri, 09 Feb 2018 16:43:17 +0000

Am Thu, 8 Feb 2018 09:31:15 +0100
Is this problem to trivial?

I read through it yesterday and found myself in the position that I
need a whiteboard or paper and pencil or an ASCII art of your
situation. But by the time I made it to the question I was
basically lost. Could you massively simplify this and maybe
produce the ASCII art?
/bz
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to

All right.
I'm not much of an artist and at this very moment, I haven't much
experience with neat ASCII art tools. But I'll provide a sketch
later, but I also will simplify the situation.
Consider three "vswitches", basically based on the creation of
bridges, bridge0, bridge1, bridge2. Create at least three individual
vnet-jails attached to each vbridge. Those jails have epair pseudo
devices. The jail itself owns the "a-part" of the epair and the
b-part is "member of the bridge". Each jail's epairXXXa has an IP
assigned of the network the vswitch is part of. I mention a- and
b-part of the epair here, because I thought it could matter, but I
think for symmetry reasons it doesn't.
Now consider a further, special jail. This jail is supposed to have
three epair devices, each one is reaching into one of the vbridges.
This jail is the router/routing jail. Later, this jail should filter
via IPFW the traffic between the three vbridges according to rules,
but this doesn't matter here, beacuase the basics are not working as
expected.
Now the problems. It doesn't matter on which jail of the three
vswitches I login, the moment a vbridge has more than two member
epairs (one is alway member of the routing jail, now consider a
database jail and a webserver jail), pinging each jail or the routing
jail fails. It works sometimes for a couple of ICMP packets and then
stops.
If each vbridge has only one member jail, I have NO PROBLEMS
traversing accordingly to the static routing rules from one vbridge
to any other, say from vbridge1 to vbridge0 or vbridge2 and any
permutation of that.
The moment any of the bridges gets an additional member epair
interface (so the bridge has at least three members including the on
reaching into the virtual router jail) the vbridge seems to operate
unpredictable (to me). Pinging jails memeber of that vbridge are
unreachable.
The kernel has options IPFIREWALL, VIMAGE. The host's ipfw (kernel)
declines packets by default. Each jail is configured to have ipfw
"open".
Thanks for the patience.

If you could provide a script which sets up the topology you described
in two lengthy posts then others could reproduce this, and your chances
of getting useful feedback would certainly increase.

We also have a graphical tool (https://github.com/imunes/imunes) which
can set up a topology like you described in a few clicks of a mouse,
albeit using netgraph and ng_eiface instead of epairs, but I assume this
is irellevant as long as you are not aiming for maximum packet
throughputs. If you attempt to use this tool, note that selecting
"stpswitch" will create if_bridge instances, whereas "lanswitch"
creates ng_bridge instances.

Good luck,

Marko

Post by O. Hartmann
Kind regards,
O. Hartmann

[O. Hartmann]

On Sat, 10 Feb 2018 09:54:49 +0100

Post by Marko Zec
On Sat, 10 Feb 2018 08:52:21 +0100

Post by O. Hartmann
Am Fri, 09 Feb 2018 16:43:17 +0000

Am Thu, 8 Feb 2018 09:31:15 +0100
Is this problem to trivial?

I read through it yesterday and found myself in the position that I
need a whiteboard or paper and pencil or an ASCII art of your
situation. But by the time I made it to the question I was
basically lost. Could you massively simplify this and maybe
produce the ASCII art?
/bz
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to

All right.
I'm not much of an artist and at this very moment, I haven't much
experience with neat ASCII art tools. But I'll provide a sketch
later, but I also will simplify the situation.
Consider three "vswitches", basically based on the creation of
bridges, bridge0, bridge1, bridge2. Create at least three individual
vnet-jails attached to each vbridge. Those jails have epair pseudo
devices. The jail itself owns the "a-part" of the epair and the
b-part is "member of the bridge". Each jail's epairXXXa has an IP
assigned of the network the vswitch is part of. I mention a- and
b-part of the epair here, because I thought it could matter, but I
think for symmetry reasons it doesn't.
Now consider a further, special jail. This jail is supposed to have
three epair devices, each one is reaching into one of the vbridges.
This jail is the router/routing jail. Later, this jail should filter
via IPFW the traffic between the three vbridges according to rules,
but this doesn't matter here, beacuase the basics are not working as
expected.
Now the problems. It doesn't matter on which jail of the three
vswitches I login, the moment a vbridge has more than two member
epairs (one is alway member of the routing jail, now consider a
database jail and a webserver jail), pinging each jail or the routing
jail fails. It works sometimes for a couple of ICMP packets and then
stops.
If each vbridge has only one member jail, I have NO PROBLEMS
traversing accordingly to the static routing rules from one vbridge
to any other, say from vbridge1 to vbridge0 or vbridge2 and any
permutation of that.
The moment any of the bridges gets an additional member epair
interface (so the bridge has at least three members including the on
reaching into the virtual router jail) the vbridge seems to operate
unpredictable (to me). Pinging jails memeber of that vbridge are
unreachable.
The kernel has options IPFIREWALL, VIMAGE. The host's ipfw (kernel)
declines packets by default. Each jail is configured to have ipfw
"open".
Thanks for the patience.

If you could provide a script which sets up the topology you described
in two lengthy posts then others could reproduce this, and your chances
of getting useful feedback would certainly increase.
We also have a graphical tool (https://github.com/imunes/imunes) which
can set up a topology like you described in a few clicks of a mouse,
albeit using netgraph and ng_eiface instead of epairs, but I assume this
is irellevant as long as you are not aiming for maximum packet
throughputs. If you attempt to use this tool, note that selecting
"stpswitch" will create if_bridge instances, whereas "lanswitch"
creates ng_bridge instances.
Good luck,
Marko

Post by O. Hartmann
Kind regards,
O. Hartmann

_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-current

Hello Marko,

thanks for your response. First of all: I looked at "imunes". From the first
glimpse it looks great! Something really usefull; I regret not having a port
for this tool or the chance to package it via poudriere.

The problems I faced seem to be related to a bug Olivier Cochard-Labbe pointed
me at:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=176671

Checking the MAC of the epairs created revealed, that either doubles or even
more occur on the host side of the epair (in my case, all the b-parts of an
epair), or, if there is nothing irregular, then the a-parts (owned by the
VIMAGE jail) have same MAC. The more jails I create, the more ambiguous MACs
are present.

It is the first time I ran into a more complex network topology and thanks for
the hint using netgraph.

Kind regards,

Oliver

[Olivier Cochard-Labbé]

Post by O. Hartmann
The moment any of the bridges gets an additional member epair interface (so the bridge
has at least three members including the on reaching into the virtual router jail) the
vbridge seems to operate unpredictable (to me). Pinging jails memeber of that vbridge
are unreachable.

​First idea:
Did you try with a more simple setup, like with just 3 jails members of one
bridge ?
=> I've tried it on a -head, and all 4 members (3 jails and the host)
reach to communicate.

Second idea:
Can you check that all epairs have different MAC address each ?​
I hit this bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=176671

Regards,

Olivier

[O. Hartmann]

On Sat, 10 Feb 2018 11:52:18 +0100

Post by Olivier Cochard-Labbé

Post by O. Hartmann
The moment any of the bridges gets an additional member epair interface (so the bridge
has at least three members including the on reaching into the virtual router jail) the
vbridge seems to operate unpredictable (to me). Pinging jails memeber of that vbridge
are unreachable.

Did you try with a more simple setup, like with just 3 jails members of one
bridge ?
=> I've tried it on a -head, and all 4 members (3 jails and the host)
reach to communicate.

Yes, I did.

I used to setup one bridge (bridge0) and three jails. Each jail owns its
epairXXa device with IP assigned. epairXXb of each jail is member of the
bridge. Bridge is up, epairXXb is up.

Post by Olivier Cochard-Labbé
Can you check that all epairs have different MAC address each ?​
I hit this bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=176671

Very good shot! Thanks. No, they do not have all of them unique MAC adrresses
and in some occassions members of the very same bridge have the very same MAC
addresses, mostly the a-part of the epair:

From console:
[...]
ng_ether_ifnet_arrival_event: can't re-name node epair10250a
==>> epair20128a: Ethernet address: 02:ce:d0:00:07:0a
epair20128b: Ethernet address: 02:ce:d0:00:13:0b
epair20128a: link state changed to UP
epair20128b: link state changed to UP
epair20128b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair20128a
==>> epair20129a: Ethernet address: 02:ce:d0:00:07:0a
epair20129b: Ethernet address: 02:ce:d0:00:14:0b
epair20129a: link state changed to UP
epair20129b: link state changed to UP
epair20129b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair20129a

epair20XXX are member of bridge2 and dedicated epairs of jails.

The following is the desastrous picture of bridge1:

[...]
==>> epair235a: Ethernet address: 02:ce:d0:00:07:0a
epair235b: Ethernet address: 02:ce:d0:00:0d:0b
epair235a: link state changed to UP
epair235b: link state changed to UP
epair235b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair235a
==>> epair237a: Ethernet address: 02:ce:d0:00:07:0a
epair237b: Ethernet address: 02:ce:d0:00:0e:0b
epair237a: link state changed to UP
epair237b: link state changed to UP
epair237b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair237a
==>> epair251a: Ethernet address: 02:ce:d0:00:07:0a
epair251b: Ethernet address: 02:ce:d0:00:0f:0b
epair251a: link state changed to UP
epair251b: link state changed to UP
epair251b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair251a
==>> epair238a: Ethernet address: 02:ce:d0:00:07:0a
epair238b: Ethernet address: 02:ce:d0:00:10:0b
epair238a: link state changed to UP
epair238b: link state changed to UP
epair238b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair238a
[...]


This is on CURRENT (FreeBSD 12.0-CURRENT #36 r329150: Mon Feb 12 06:30:47 CET
2018 amd64).

I did check on Friday in the bureau and didn't catch it since I was checking on
each jail, but the console log accessed via dmesg revealed the problem very
easily.

I did a check on the 11.1-RELENG-p6 box and I got the same picture there,
different, but very similar setup.

So I didn't see it in the masses of epairs our setup requires :-(

I'll go now for setting MAC addresses manually and check functionality again.

Post by Olivier Cochard-Labbé
Regards,
Olivier

Kind regards,

Oliver

[O. Hartmann]

On Sat, 10 Feb 2018 11:52:18 +0100

Post by Olivier Cochard-Labbé

Post by O. Hartmann
The moment any of the bridges gets an additional member epair interface (so the bridge
has at least three members including the on reaching into the virtual router jail) the
vbridge seems to operate unpredictable (to me). Pinging jails memeber of that vbridge
are unreachable.

Did you try with a more simple setup, like with just 3 jails members of one
bridge ?
=> I've tried it on a -head, and all 4 members (3 jails and the host)
reach to communicate.
Can you check that all epairs have different MAC address each ?​
I hit this bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=176671

Wow, that PR is from 2013(!) and it is still not solved?

Post by Olivier Cochard-Labbé
Regards,
Olivier
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-current

[O. Hartmann]

On Mon, 12 Feb 2018 09:37:47 +0100

Post by O. Hartmann
On Sat, 10 Feb 2018 11:52:18 +0100

Post by Olivier Cochard-Labbé

Post by O. Hartmann
The moment any of the bridges gets an additional member epair interface
(so the bridge
has at least three members including the on reaching into the virtual
router jail) the
vbridge seems to operate unpredictable (to me). Pinging jails memeber of
that vbridge
are unreachable.

Did you try with a more simple setup, like with just 3 jails members of one
bridge ?
=> I've tried it on a -head, and all 4 members (3 jails and the host)
reach to communicate.
Can you check that all epairs have different MAC address each ?​
I hit this bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=176671

Wow, that PR is from 2013(!) and it is still not solved?

Post by Olivier Cochard-Labbé
Regards,
Olivier
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-current

_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-current

After rebooting recent CURRENT, the view with "ifconfig -a ether" looked good
so far, each epair/bridge has its unique MAC.

But then, login on the jails and checking the epair's counterpart owned by the
VIMAGE jail, I found almost EVERY jail has the same MAC, even those jails
members of the same bridge:

[...]
jail 11 on bridge2
epair20129a: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
1500 options=8<VLAN_MTU>
ether 02:68:d0:00:07:0a

jail 10 on bridge2
epair20128a: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
1500 options=8<VLAN_MTU>
ether 02:68:d0:00:07:0a

jail 9 on bridge 1
epair10250a: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
1500 options=8<VLAN_MTU>
ether 02:68:d0:00:07:0a

jail 8 on bridge1
epair10238a: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
1500 options=8<VLAN_MTU>
ether 02:68:d0:00:07:0a

jail 7 on bridge0
epair238a: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:68:d0:00:07:0a

epair251a: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:68:d0:00:07:0a

The way I create epairs and put them into a jail's context/domain is straight
forward. In jail.conf, I have more generic setup with variables like:

# DNS Master
ns1 {
$if = "2";
$ip4_addr = "10.10.0.${if}";
$ip4_cidr = "24";
$ip4_my_default_route = "10.10.0.1";
$vnet_if = "epair${if}";
$home_bridge = "${if_bridge_core}";

depend= "vrouter";

allow.raw_sockets= "1";
}


and in the common portion of jail.conf definitions, I use this:

[...]

vnet = "new";
vnet.interface = "${vnet_if}a";
persist;

exec.clean;

exec.start= "";
exec.start+= "/sbin/ifconfig ${vnet_if}a inet
${ip4_addr}/${ip4_cidr} up"; exec.start+= "/bin/sh /etc/rc";
exec.start+= "/sbin/route add default ${ip4_my_default_route}";
exec.start+= "/sbin/sysctl net.link.bridge.pfil_member=0";
exec.start+= "/sbin/sysctl net.link.bridge.pfil_bridge=0";
exec.start+= "/sbin/sysctl net.link.bridge.pfil_onlyip=0";

exec.stop= "/bin/sh /etc/rc.shutdown";

exec.prestart= "";
exec.prestart+= "ifconfig ${vnet_if} create";
exec.prestart+= "ifconfig ${vnet_if}b up";
exec.prestart+= "ifconfig ${home_bridge} addm ${vnet_if}b up";

exec.poststop= "ifconfig ${home_bridge} deletem ${vnet_if}b";
exec.poststop+= "ifconfig ${vnet_if}b destroy";

exec.consolelog= "/var/log/jail_${name}_console.log";


The big question here is: when a jail with VIMAGE kernel "swallows" a
epair-pseudo device, it leaves the ciontext or visibility of the host. How can
the FreeBSD VIMAGE kernel then know about what former epair's MAC was? Is this
mechanism maybe the culprit? It is just a thought, so I do not want to be
beheaded - I'm not much into system development.


Kind regards,

O. Hartmann