What's Going On With Jail Networking?

[Grand_Moff_Twerkin]

I set my jails up using NAT which is fine and good for the most part but if I ever add or remove jails and reboot the host then the jail IP's always change. This is very bad because then I have to go through all my jails and update the new IP's and is a major PITA.

1.) How can I either lock the IP's IOcage is using for it's NAT jails, so they don't change when I add or delete jails and reboot the host?
2.) Is there any good scripts that will keep track of running jails and update the host files in them all?
3.) Does IOcage run some sort or internal DNS so that I can just point it toward host names?
4.) Can I use port forwarding from my router? If so how?
5.) Why will pointing the jails toward the host on their running port not work? I have done this with Docker containers on Linux.

Which brings me to DHCP, I have never been able to get a jail running through DHCP. I'm sure that it isn't my pfSense (v2.60) box because I have been testing with Virtualbox using a bridged adapter it has zero problem assigning a dynamic IP in the same range as the host or what I would call my LAN.

As for static IP's those work somewhat but are not reliable at all, I can start the jail and try to ping Google and it will tell me `the host is down` or `no route to host` so I guess it can't find the default gateway or resolve DNS. As soon as I restart DHCP on the router though the jail will work but if I have to restart the jail I have to restart DHCP again. I have tried to figure out why this is but I've got to the point where I've burned through the hours and have had enough.

I just want something that works so I've come to ask "What is going on with jail networking?". Is this FreeBSD? Is it TrueNAS's middleware? Why are there threads going all the way back to FreeNAS 9 about this problem? Why hasn't it been solved?

 

[artlessknave]

Virtualbox using a bridged adapter

this tells me you have TrueNAS virtualized. if you are virtualizing TrueNAS, the jails networking becomes more complex.
please read the forum rules, and if I am not incorrect about virtualizing, read the parts about why virtualized TrueNAS is highly not recomended.

most of the people who can answer questions like these will not do so when the forum rules haven't at least been followed.

 

[Grand_Moff_Twerkin]

No, I am not virtualizing TrueNAS it's running on metal. I was running Virtualbox on a separate machine in order to test my DHCP server was functioning properly.

 

[artlessknave]

it's running on metal

ah, ok, that's simpler then. you do need to check out the forum rules though.

 

[Grand_Moff_Twerkin]

I guess my main problem is that my bridging seems to be incorrectly configured been back and forth so far in this thread. I am supposed to turn off DHCP on my LAGG and assign the IP to the bridge0 (which doesn't appear under network interfaces), still unsure on how to acomplish this. Do I delete the bridge0 interface through the CLI and then recreate it in the GUI? Then what do I do about the statically assigned LAGG IP in my pfSense router? Does the newly created bridge get the same MAC as the LAGG? Had a hard enough time creating the LAGG in the first place, don't really want to make the box unreachable. Network dump debug attached.

 

[artlessknave]

when had jails (I got mad and nuked them all F*^&%$ers) they were all DHCP. on a LAGG.
that *should* work fine. im even usually basically the same motherboard (x11ssm)
I see you do have your system; if I didnt notice it before, I apologize; if you just added it, thanks! (trying to be more positive!)

 

[artlessknave]

I looked at that thread, and I have nothing to add there. I remember something being a little off when I first setup the jails, but I got it working in a way that seems like what you are having trouble with, but I dont remember enough details about it.
I slapped together a new jail, simply hit DHCP, VNET, and auto interface, and the jail can ping google.
Im using pfsense as well.
the trueNAS/freeNAS/SCALE version should be in your system config. are you using an old version or anything? as I was creating this test jail, I definitely remember having issues, but that was a long time ago.

 

[Grand_Moff_Twerkin]

I'm using TrueNAS Core 12.0-U8, I just put latest because it will change and I will forget to update it.

In the GUI if you choose DHCP it auto selects VNET and BPF as well and won't let me continue without all three.

I thought I might sidestep this rigamarole my spinning up a VM and using Docker and NFS but the interface it spins up also does not get an IP so no internet.

 

[artlessknave]

probably wont help but, eh, here are some screenshots. the vm is linux mint

 

[Grand_Moff_Twerkin]

Do you have any bridges set up under network interfaces? What does ifconfig look like for bridge0? I still don't get how I'm going to do this without wrecking the connections.

 

[Grand_Moff_Twerkin]

Problem solved turns out it was setting the LAGG to load balance, once I switched it to fail over everything is working fine and as it should. SOLVED!

 

[Patrick M. Hausen]

Are you using lagg without an LACP capable switch? Don't

 

[Grand_Moff_Twerkin]

I thought when I bought the switch it was LACP capable, I'll have to look into it further. But now that I know that it can cause your jails and VM's not to work it's really not worth the trouble enabling lagg. I was under the impression that I could be copying lots of files over scp or video to the server while the other port could handle serving everyone else in the house. If I looked into it would probably find that I wasn't even saturating one of the ports half the time.

Hours lost lesson learned won't be doing that again any time soon.

 

[Patrick M. Hausen]

The main point even with failover or loadbalancing instead of LACP is that ethernet channel bundling is something that in most cases must be actively configured and maintained on both ends of the connection. You cannot enable a bond/lagg/port-channel only on one side and expect that to work.

Even with ESXi that does not support LACP without the distributed vSwitch feature you must configure the port-channel on the other end, e.g. Cisco (which is what I mostly use).

HTH,
Patrick

 

[jgreco]

If I looked into it would probably find that I wasn't even saturating one of the ports half the time.

Hours lost lesson learned won't be doing that again any time soon

Have a gander over at

Resource - LACP ... friend or foe?

Too many LACP discussions lately. Here's what you need to know. 1) Link aggregation groups are an interesting way to boost network throughput. They're typically managed by a protocol called Link Aggregation Control Protocol (LACP). Link...

www.truenas.com

 

[artlessknave]

The main point even with failover or loadbalancing instead of LACP is that ethernet channel bundling is something that in most cases must be actively configured and maintained on both ends of the connection. You cannot enable a bond/lagg/port-channel only on one side and expect that to work.

my failover lagg0 works fine. doesnt even connect to the same switch...there are no switch side settings (quantas, the settings make no sense, line switching and nothing else)

 

[Patrick M. Hausen]

With failover only one port is active at a time, so yes, expected to work. Sorry for over-generalising. But LACP to two switches if not supported - no way. Multi-chassis-LACP is yet another special feature most of the time implemented via "stacking" where the two switches in question share one single control plane.