Change TrueNAS IP, battling with jail network

[georgelza]

Hi guys

I changed my TrueNAS IP from 192.168.0.14 -> 172.16.10.24, I can access my TrueNAS.

I had 3 jails : with ip's 192.168.0.9, 192.168.0.5 and 192.168.0.6
it had vnet's assigned as interface. but can't select pic a vnet now.

please advise.
G

 

[georgelza]

let me add, I create a temporary jail, and defined that as a vnet also, it / TrueNAS created the jail and started it. with a vnet defined.
some how I'm not even capable of editing the current jail, it's grey'd out.
G

 

[sretalla]

some how I'm not even capable of editing the current jail, it's grey'd out.

You can't edit a running jail.

 

[Patrick M. Hausen]

You can only edit jails in the shutdown state. Not while they are running.
What you can do is use iocage set/get on the command line to view and change jail parameters. Even when running.

 

[georgelza]

You can't edit a running jail.

it's not running.
if it wa running i would not have this problem.
G

 

[sretalla]

TrueNAS created the jail and started it.

it's not running.

Which is it?

In either case:

use iocage set/get on the command line to view and change jail parameters. Even when running.

 

[georgelza]

ok... see problems...
I created a temporary jail, to test if that created a vnet, yes it did, and it assigned the required IP.
the comment about can not edit settings, was mis leading, i was referring to the my 3 "problem" jails, the 4th one, the test one I know I can't edit while running.
G

 

[georgelza]

I've been able to tick untick tick boxes, in the end I got a vnet0 selected... (looks like it was created after I created the test jail) and then assign the ip, subnet and gw... but when i click start it fails, see below.

G

 

[georgelza]

any chance either of you gentleman might be willing to jump onto a zoom or MS Teals call and help me fix this, my problem... I'm fixing/changing my network |(which is Unifi based) and my controller is one of the jails, so to fix the network I need to now get this jail working.
G

 

[sretalla]

I can't see which jail the first screenshot is from, but it seems BPF and VNET are set there.

if you can post the output from iocage get all Unifi-Controller-6 we may be able to spot the problem.

 

[georgelza]

Those images was of the Unify-Controller-6 Jail,

and here is the output of the command.

CONFIG_VERSION:28
allow_chflags:0
allow_mlock:0
allow_mount:0
allow_mount_devfs:0
allow_mount_fusefs:0
allow_mount_nullfs:0
allow_mount_procfs:0
allow_mount_tmpfs:0
allow_mount_zfs:0
allow_quotas:0
allow_raw_sockets:0
allow_set_hostname:0
allow_socket_af:0
allow_sysvipc:0
allow_tun:0
allow_vmm:0
assign_localhost:0
available:readonly
basejail:0
boot:0
bpf:1
children_max:0
cloned_release:12.2-RELEASE
comment:none
compression:lz4
compressratio:readonly
coredumpsize:off
count:1
cpuset:off
cputime:off
datasize:off
dedup:off
defaultrouter:172.16.10.1
defaultrouter6:auto
depends:none
devfs_ruleset:4
dhcp:0
enforce_statfs:2
exec_clean:1
exec_created:/usr/bin/true
exec_fib:0
exec_jail_user:root
exec_poststart:/usr/bin/true
exec_poststop:/usr/bin/true
exec_prestart:/usr/bin/true
exec_prestop:/usr/bin/true
exec_start:/bin/sh /etc/rc
exec_stop:/bin/sh /etc/rc.shutdown
exec_system_jail_user:0
exec_system_user:root
exec_timeout:60
host_domainname:none
host_hostname:Unify-Controller-6
host_hostuuid:Unify-Controller-6
host_time:0
hostid:031b021c-040d-05ce-0706-1c0700080009
hostid_strict_check:0
interfaces:vnet0:bridge0
ip4:new
ip4_addr:vnet0|172.16.10.10/24
ip4_saddrsel:0
ip6:new
ip6_addr:none
ip6_saddrsel:0
ip_hostname:0
jail_zfs:0
jail_zfs_dataset:iocage/jails/Unify-Controller-6/data
jail_zfs_mountpoint:none
last_started:2021-10-05 06:08:59
localhost_ip:none
login_flags:-f root
mac_prefix:06aedb
maxproc:off
memorylocked:off
memoryuse:off
min_dyn_devfs_ruleset:1000
mount_devfs:1
mount_fdescfs:1
mount_linprocfs:0
mount_procfs:0
mountpoint:readonly
msgqqueued:off
msgqsize:off
nat:0
nat_backend:ipfw
nat_forwards:none
nat_interface:none
nat_prefix:172.16
nmsgq:off
notes:none
nsem:off
nsemop:off
nshm:off
nthr:off
openfiles:off
origin:readonly
owner:root
pcpu:off
plugin_name:none
plugin_repository:none
priority:99
pseudoterminals:off
quota:none
readbps:off
readiops:off
release:12.2-RELEASE-p4
reservation:none
resolver:/etc/resolv.conf
rlimits:off
rtsold:0
securelevel:2
shmsize:off
stacksize:off
state:down
stop_timeout:30
swapuse:off
sync_state:none
sync_target:none
sync_tgt_zpool:none
sysvmsg:new
sysvsem:new
sysvshm:new
template:0
type:jail
used:readonly
vmemoryuse:off
vnet:0
vnet0_mac:06aedba63966 06aedba63967
vnet0_mtu:auto
vnet1_mac:none
vnet1_mtu:auto
vnet2_mac:none
vnet2_mtu:auto
vnet3_mac:none
vnet3_mtu:auto
vnet_default_interface:auto
vnet_default_mtu:1500
vnet_interfaces:none
wallclock:off
writebps:off
writeiops:off

 

[sretalla]

OK, so with no DHCP, you shouldn't need bpf, which should eliminate that error.

 

[georgelza]

removed bpf
tried to start, error received back.
G

 

[sretalla]

OK, then let's have a look at that...

ifconfig from the host

 

[georgelza]

oot@vaultx[~]# ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=81249b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER>
ether e0:d5:5e:57:5a:95
inet 172.16.10.24 netmask 0xffffff00 broadcast 172.16.10.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
groups: pflog
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:59:52:f2:b0:00
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: vnet0.4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 5 priority 128 path cost 2000
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.4: flags=8942<BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: associated with jail: test as nic: epair0b
options=8<VLAN_MTU>
ether 06:ae:db:ae:1b:75
hwaddr 02:69:c0:25:be:0a
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=1<PERFORMNUD>
root@vaultx[~]#

 

[georgelza]

so agree see not vnet01, 2, or 3...
the vnet0.4 is the test jail i created.
G

 

[sretalla]

Also I see you have vnet set to 0 on the jail (and it should probably be 1 instead)

 

[sretalla]

I don't see where your em0 is a member of the bridge together with the VNET. Is that intentional? (are you trying to keep your jails from connecting out of the host?)

For example, my bridge0 (which contains members of all my jails and the igb interface which is my primary NIC)

Code:

bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:e6:ca:b2:00:00
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vnet0.294 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 11 priority 128 path cost 2000
    member: vnet0.293 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 13 priority 128 path cost 2000
    member: vnet0.252 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 9 priority 128 path cost 2000
    member: vnet0.149 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 7 priority 128 path cost 2000
    member: vnet0.7 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 12 priority 128 path cost 2000
    member: vnet0.5 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 10 priority 128 path cost 2000
    member: vnet0.3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 8 priority 128 path cost 2000
    member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 6 priority 128 path cost 2000
    member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 1 priority 128 path cost 20000
    groups: bridge

 

[sretalla]

You can fix that (temporarily) by using ifconfig bridge0 addm em0 up

 

[georgelza]

I redid the bridge quickly
promised I did select em0 as a member to the bridge, when I created originally.

see below

Code:

root@vaultx[~]# ifconfig
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=812099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER>
    ether e0:d5:5e:57:5a:95
    inet 172.16.10.24 netmask 0xffffff00 broadcast 172.16.10.255
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
    inet 127.0.0.1 netmask 0xff000000
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
    groups: pflog
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:59:52:f2:b0:00
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 1 priority 128 path cost 20000
    member: vnet0.4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 5 priority 128 path cost 2000
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.4: flags=8942<BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: test as nic: epair0b
    options=8<VLAN_MTU>
    ether 06:ae:db:ae:1b:75
    hwaddr 02:69:c0:25:be:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=1<PERFORMNUD>
root@vaultx[~]#