ipfw and truenas -- UPDATE PEBKAC

[sef]

I've searched (and I admit I'm not good at that), but couldn't find anything.

I finally ugpraded my NAS from FreeNAS 11 to TrueNAS Core 12. And... now there's a firewall (ipfw) loaded and running? I can't find any way to configure it or disable it?

Blocking all UDP meant that, as an example, ntp wouldn't work. So... did I miss anything?

UPDATE
It turns out that, a couple of years ago, I added a post-init script. I am pretty sure I did this as part of an experiment related to a bug, but who knows ;). Regardless, I found it, deleted it, and all is good again. yay me!

 

[Kris Moore]

Do you happen to have any plugins / jails running? Iocage does use ipfw for its NAT abilities.

 

[jgreco]

ipfw has been loaded and running in FreeNAS for years, but it is typically just the default allow rule listed in it.

ipfw add 65500 pass all from any to any

would be the classic way to allow all traffic.

What does "ipfw -l" report for you?

 

[winnielinnie]

What does "ipfw -l" report for you?

That command didn't work when I tried it. What worked for me was:
ipfw list

Which resulted in this output:
65535 allow ip from any to any

 

[sef]

ipfw has been loaded and running in FreeNAS for years, but it is typically just the default allow rule listed in it.

ipfw add 65500 pass all from any to any

would be the classic way to allow all traffic.

What does "ipfw -l" report for you?

"ipfw -l" fails, because it's not a valid option. "ipfw list" shows:

00100 allow ip from any to any via lo0
00500 allow ip from table(internal) to any in
00500 allow ip from any to table(internal) out
01000 reject udp from any to me 111
01500 deny udp from me to any
65535 allow ip from any to any

Rule 1500 is the bad one: that means, among other things, that ntp can't work outside of the local network, which resulted in a cascading set of failures eventually. (Was it supposed to be limited to port 111?)

 

[sef]

Do you happen to have any plugins / jails running? Iocage does use ipfw for its NAT abilities.

I do on one system, don't on another, and they both have the same ipfw rules. The ipfw rules given (see below) are not for NAT.

 

[jgreco]

"ipfw -l" fails, because it's not a valid option. "ipfw list" shows:

Meant "ipfw l", sorry.

Rule 1500 is the bad one: that means, among other things, that ntp can't work outside of the local network, which resulted in a cascading set of failures eventually. (Was it supposed to be limited to port 111?)

Yeah, looks like a naive ruleset written by someone unfamiliar with the operational aspects of UNIX networking. It's missing DNS on 53, which should probably be implemented as a stateful rule (udp from me to any 53 keep-state) -- failure to include this means that use of an external resolver has to fail over to TCP (and last I checked that isn't supported by the BSD resolver lib), NTP on 123, which should probably be implemented as stateful on 123, etc.

The problem these days is lots of people use services like Google DNS (8.8.8.8) for their DNS, and NTP Pool (pool.ntp.org) for timeservice. While you could add Google DNS or whatever happens to be in /etc/resolv.conf to the "internal" ipfw table, that wouldn't work for the NTP Pool because the entire point of pool is to spread load by semirandomly spreading the load periodically (temporarily switching to my Infrastructure @ NTP.ORG hat). Therefore the best bet there is a stateful rule, that's the safest way to deal with that traffic.

Rule 1000 does not make sense given rule 1500; it would just create log noise in the event RPC traffic came in. I see no obvious reason for it.

 

[winnielinnie]

Yeah, looks like a naive ruleset written by someone unfamiliar with the operational aspects of UNIX networking.

What is adding those rules? Surely a jail/plugin has no access to modify ipfw outside of itself (i.e, that of the TrueNAS server)?

 

[jgreco]

What is adding those rules? Surely a jail/plugin has no access to modify ipfw outside of itself (i.e, that of the TrueNAS server)?

Bearing in mind that I do a lot of work with jails, but NOT FreeNAS/TrueNAS jails ...

There is typically infrastructure to set up jail rules in the host system by the scripts that manage jails. This allows a jail to be "safely" handed off in a shared tenant environment. This could be considered part of the "jail" that isn't actually inside the jail's root directory tree.

There is *also* support in the form of VNET to do firewalling within a jail, IIRC. This generally means that someone has installed a base FreeBSD inside a jail, which is somewhat pointless IMO -- so I don't have a ton of insight into that.

I also don't know the answer to your question as to what is adding those rules, but it feels vaguely like "developer with insufficient expertise added this and it happens to work in the development environment because the development environment contains nameservers and timeservers." So I'd place a wager, a small one, on this being a TrueNAS thing, especially because it uses tables to try to define an interior zone, which is something you'd want early on in rule processing to keep performance high.

I got tired of writing UNIX firewall rulesets decades ago, so a long time ago I wrote an automated firewalling script suitable for use in the service provider world, which assumes that the world works a certain way, and handles all these details. I understand how easy it is to screw up writing rulesets by hand, especially to cover stuff like RFC1918 space, forged packets, etc. I appreciate the effort and difficulty in doing a good job of this kind of thing.

 

[sef]

Yeah, I haven't been able to figure out where the rules are added, either. I'm giving up and doing a find over the entire system now, which will ... take some time.

The only jail on the system is the Plex Plugin. It could be adding it.

My secondary FreeNAS Mini totally bit the dust, so I can't compare it any more.

 

[winnielinnie]

The only jail on the system is the Plex Plugin. It could be adding it.

I'm also running Plex in a jail (plugin), but I have no such rules. ipfw list yields only a single line.
65535 allow ip from any to any

 

[sef]

That's the ipfw set up in the jail; the mystery one -- whose source nobody has yet explained -- is in the host system.

 

[sef]

ha ha. "find / -type f -print0 | xargs -0 fgrep 'deny udp from me to any' /dev/null" consumes all swap space and died.