Connecting to TrueNAS ... After creating or starting jail

[yusisushi]

Every time I either start or create a jail this happens, about 30seconds later the webUI stops responding and it will stay like this forever until I reboot:

I read multiple posts about this but nothing resolved it for me.

What I've tried

1. I read online that restarting middleware should help but it does not resolve anything for me.
   
   Code:

   user@truenas:~ $ sudo service middlewared statusmiddlewared is running as pid 1926
   
   user@truenas:~ $ sudo service middlewared restart
   Stopping middlewared.
   Waiting for PIDS: 1701.
   
   user@truenas:~ $ sudo service middlewared status
   middlewared is running as pid 1816

   
   
   
2. I already updated to the latest version
   
   Code:

   cat /etc/os-release...
   VERSION="13.1-RELEASE-p7
   ...

   
   
   
   
   
   
3. I already Checked The gateway and DNS settings

Everything seems to work except from the webinterface. I tried different browsers and devices to connect.A reboot from cli reverts the webinterface back to working for a bit, after 15minutes or so I get the same error while connecting.

 

[sretalla]

What jails are you running and does one (or more) of them use port 80 or 443 and NAT?

 

[yusisushi]

What jails are you running and does one (or more) of them use port 80 or 443 and NAT?

Currently only one active running jail that is a unifi controller and it doesn't use these standard ports. I'm not 100% sure if the jails cause the UI to dissapear because sometimes it also seems to happen at random times, but a jail is guaranteed to cause the issue.

 

[sretalla]

only one active running jail that is a unifi controller and it doesn't use these standard ports.

Is it NAT or VNET?

 

[yusisushi]

Is it NAT or VNET?

Hi, sorry for not clarifying earlier. It appears both;

Now I've done some additional testing and the issue does not seem to happen with any jail except the template Jail I created.
I tried creating this jail both as a clone or basejail, both times with NAT and VNET option.

Notable: when the UI of TrueNAS becomes unavailable, the webinterface of my jails stay reachable.

 

[sretalla]

What if you only take the VNET option? does the jail still block the UI?

 

[yusisushi]

What if you only take the VNET option? does the jail still block the UI?

Hi, I can confirm disabling NAT and only using VNET the same still happens

 

[sretalla]

Can you share the output from ifconfig when the problem is happening?

 

[yusisushi]

Can you share the output from ifconfig when the problem is happening?

I sure can. Below is after reboot. no issues

Code:

user@truenas:~ $ ifconfig
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=82098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether 00:e0:4f:15:4e:5b
        inet6 fe80::2e0:4fff:fe15:4e5b%re0 prefixlen 64 scopeid 0x1
        inet6 2a02:578:857f:1700:2e0:4fff:fe15:4e5b prefixlen 64 autoconf
        inet 10.85.0.252 netmask 0xffffff00 broadcast 10.85.0.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
        groups: pflog
vnet0.1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: UniFi-controller as nic: epair0b
        options=8<VLAN_MTU>
        ether 02:e0:4f:ef:1c:f8
        hwaddr 02:40:70:85:25:0a
        inet 172.16.0.1 netmask 0xfffffffc broadcast 172.16.0.3
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=2b<PERFORMNUD,ACCEPT_RTADV,IFDISABLED,AUTO_LINKLOCAL>

And this is after starting the template jail which replicates the issue;

Code:

user@truenas:~ $ ifconfig
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=82098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether 00:e0:4f:15:4e:5b
        inet6 fe80::2e0:4fff:fe15:4e5b%re0 prefixlen 64 scopeid 0x1
        inet6 2a02:578:857f:1700:2e0:4fff:fe15:4e5b prefixlen 64 autoconf
        inet 10.85.0.252 netmask 0xffffff00 broadcast 10.85.0.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
        groups: pflog
vnet0.1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: UniFi-controller as nic: epair0b
        options=8<VLAN_MTU>
        ether 02:e0:4f:ef:1c:f8
        hwaddr 02:40:70:85:25:0a
        inet 172.16.0.1 netmask 0xfffffffc broadcast 172.16.0.3
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=2b<PERFORMNUD,ACCEPT_RTADV,IFDISABLED,AUTO_LINKLOCAL>

It doesn't appear the template jail even started yet, I'm not sure if it does eventually. I'll check again in a few minutes
-> edit: can confirm 30minutes later still no extra jail running, still no webUI, still no(?) changes I can see in ifconfig output

Code:

@truenas:~ $ jls
   JID  IP Address      Hostname                      Path
     1                  unifi-c                       /mnt/tank1/iocage/jails/UniFi-controller/root

 

[sretalla]

If that jail never starts, I would suggest deleting it so you can't cause the problem anymore.

I'm a little bothered to not see a bridge interface auto-created when the jail starts if VNET is properly set up.

I also note you're using a Realtek NIC... https://www.truenas.com/community/resources/is-my-realtek-ethernet-really-that-bad.196/

Are your other jails using VNET?

 

[yusisushi]

If that jail never starts, I would suggest deleting it so you can't cause the problem anymore.

I'm a little bothered to not see a bridge interface auto-created when the jail starts if VNET is properly set up.

I also not you're using a Realtek NIC... https://www.truenas.com/community/resources/is-my-realtek-ethernet-really-that-bad.196/

Are your other jails using VNET?

Hi

Sorry for the late reply, all of my jails use VNET and NAT.

For my networkcard, it is the integrated NIC of the motherboard, and due to space constraints I have few other options.

 

[nikinp]

Dear @sretalla
I have the same issue and have seen this before on a previous build (pretty much same spec as my system 1). I am attempting to use Jail which require port 80 / 443 to be opened. The creation/starting of the jails seem to log me out of the UI and this is happening also during running some scripted jail creations. The instability is extremely frustrating and worrying.
Appreciate if you are able to help me please!

 

[sretalla]

use Jail which require port 80 / 443 to be opened. The creation/starting of the jails seem to log me out of the UI

You're going to need to be much more specific about what you're doing to create the jail and how you intend it to work before I can provide any help.