joelmusicman,
You are correct. But let's look at this relatively speaking.
Your config file is a whole 200kb or so. That's it. And once you've setup FreeNAS, if you've done your job and you set up everything properly, you should rarely(if ever) need to log into it again. I've used both USB sticks and full fledged SSDs. There's 1 or 2 places where it matters, tunables and sysctls. I've seen it take 2-3 minutes when deleting them(adding them takes no time at all). But, neither tunables nor sysctls are things you should be playing with as a course of business, so that argument is somewhat mute.
Also, the USB stick is mounted readonly, with the exception of the small slice for your config file. So you don't have to worry about any other writes being slow causing poor server performance. Also most if the USB stick is cached in RAM, so there's little penalty if you have to read stuff from the USB stick as it's probably already in RAM.
As for bootup, I've taken a very fast USB3 drive and a slow-ish USB2 drive and booted them on the same hardware with the same configuration. Total bootup time difference was less than 5 seconds. An SSD was only a few seconds faster than the faster USB3 drive. So when it comes down to it, speed of the USB stick really isn't that important. What is important is to make sure you use a name brand stick that is at least 4GB.