Manual create swap partition

[i_no_k]

Hello!

I am just installed FreeNAS 8.2.0 to HDD, can I now use the part of remaining free space (67G) for swap?
I do not want to use another disks space for it.

Code:

[root@freenas] ~# gpart show
=>       63  143552052  da0  MBR  (68G)
         63    1930257    1  freebsd  [active]  (943M)
    1930320         63       - free -  (32K)
    1930383    1930257    2  freebsd  (943M)
    3860640       3024    3  freebsd  (1.5M)
    3863664      41328    4  freebsd  (20M)
    3904992  139647123       - free -  (67G)

I tried:

Code:

[root@freenas] ~# gpart add -s 5Gb -t freebsd-swap da0
gpart: index '5': No space left on device

Help me please!

 

[William Grzybowski]

FreeNAS is supposed to be installed in a USB stick, the installed media cannot be expanded, this is by design, a current limitation that might change in a far feature.

By default freenas creates swap partitions ins the storage disks...

 

[i_no_k]

William Grzybowski, thank you for your answer.

I know about recommended USB installation, but this test server is pretty old and has slow USB 1.1 interfaces.
Also I don't want to partition all SCSI disks for swap - I want to use entire disks for ZFS RAIDZ pool.
May be is somehow possible to create a swap partition on the system drive, especially once remaining free space allow this?

 

[William Grzybowski]

There is no free space...

MBR is limited to 4 partitions, all of them are in use, this is a current limitation of freenas.

 

[jgreco]

So what you do is you make one of them bigger, manually. If you intend to ultimately run off of a hard disk due to slow USB 1.1, what you probably want to do is to install to a hard disk (which you've done), then also install to a USB key, which you can use to provide the needed tools, etc., to do the remainder of the work.

When you boot from the USB key, you can back up the partitions on the hard drive to another datastore (if you want/need to save them). Then you figure out what you want your gameplan to be.

Don't forget that FreeNAS wants to have two system partitions, one for the running system, and one for updates. What you could probably do is to add swap to both of those system partitions. Or you could decide that the ability to do updates isn't going to work well with a "rearchitected" system, and use the second "system" partition as swap, though that seems vaguely risky. Or you could add a second USB key for swap, and actually boot off USB as intended. Or someone could explain the purpose of slice 3, because that looks like it'd be a juicy target for swap fun. All of these require some twiddling of FreeNAS to get them to happen.

 

[i_no_k]

There is no free space...
MBR is limited to 4 partitions, all of them are in use, this is a current limitation of freenas.

But there are also extended partition exist in nature?

William Grzybowski, jgreco thank you for your answers!

I have understood that the easiest way is to use a USB key for the system and swap on datastores. Though I find it strange that usable space is lost on system drive, does not anyone tried to use it :)
Also I wanted to use the entire disk for datastores without cutting into slices - seems to be a logical way not to mix data and system.

a juicy target for swap fun.

I was thinking swap is a necessary condition for the stable FreeNAS work? My box is only 3Gb RAM.

 

[cyberjock]

It's not that the space is "lost". It's that you cannot allocate any more partitions to the drive so the space cannot be allocated.

 

[jgreco]

But there are also extended partition exist in nature?

"Extended" partitions are a DOS thing. Microsoft rapidly discovered how badly their MBR system sucked; "extended" partitions are created by creating a primary partition and then subpartitioning the space within that primary partition.

FreeBSD calls a MBR primary partition a "slice"; FreeBSD can already create 8 of its own usable partitions within a slice. When you see "ad0s1b", the "ad0" is the disk, the "s1" is slice 1 (MBR primary partition 1), and "b" is FreeBSD partition #2 within it, typically used for swap.

I have understood that the easiest way is to use a USB key for the system and swap on datastores. Though I find it strange that usable space is lost on system drive, does not anyone tried to use it :)

The expectation is not to use a large drive as the system drive. This makes it easier for FreeNAS to provide appliance-like capabilities for dealing with firmware upgrades. Speaking as someone with a very strong background in rearchitecting UNIX-based operating systems to have appliance-like designs, I note that the FreeNAS team has managed to do something that's a little tricky.

Also I wanted to use the entire disk for datastores without cutting into slices - seems to be a logical way not to mix data and system. I was thinking swap is a necessary condition for the stable FreeNAS work? My box is only 3Gb RAM.

Yes, it's a tradeoff. (ps: You only have 3 gigabits of RAM? "GB" is gigabytes. "Gb" is gigabits.) With only 3GB, you definitely want swap.

 

[i_no_k]

Guys, thanks for the explanations!

It's that you cannot allocate any more partitions to the drive so the space cannot be allocated.

FreeBSD can already create 8 of its own usable partitions within a slice.

ls /dev/da0*
/dev/da0 /dev/da0s1a /dev/da0s2a /dev/da0s4 /dev/da0s1 /dev/da0s2 /dev/da0s3

So how I can create some additional partitions without break FreeNAS? :)
System complitely test, so any experiments can be conducted.
Question is now just for my own development.

I realized that it is better to use the proposed swap location scheme and recommended drive size for system, problems with further updates generally are not wanted. In my test box I think to use standalone drive for swap and disable auto swap creation on data, system move to USB.

(ps: You only have 3 gigabits of RAM?

From dmesg:
real memory = 3221225472 (3072 MB)
avail memory = 3139182592 (2993 MB)

No simple way to add more to this HW (old Supermicro server, ECC memory, DDR1 probably).

 

[jgreco]

From dmesg:
real memory = 3221225472 (3072 MB)
avail memory = 3139182592 (2993 MB)

No simple way to add more to this HW (old Supermicro server, ECC memory, DDR1 probably).

That's three gigaBYTES, so use a capital B. 3GB, not "3Gb".

 

[jgreco]

How many times do I gotta say YOU DON'T, perhaps writing in uppercase
helps ;)

Actually, you can. Us old-timer sysadmins do this stuff on LIVE RUNNING SYSTEMS without the benefit of extra disks, from remote, now there, that is a minefield and a great way to really learn your stuff (or pay the price).

Both s1 and s2 contain a BSD partition table, and therefore resizing either one of them will allow for a swap partition to be added. The downside to this is that it complicates the upgrade process, and is potentially quite risky. So while William is wrong in saying YOU DON'T, he would be justified in saying YOU SHOULDN'T.

So here's what you can do, just to learn how this all works.

Use "fdisk" to examine the MBR of your system disk, let's call it "da1", and you have to do all this from a USB key based install of FreeNAS, because it's difficult and tricky to modify the ground underneath your feet. It'll look something like

Code:

******* Working on device /dev/da4 *******
parameters extracted from in-core disklabel are:
cylinders=967 heads=255 sectors/track=63 (16065 blks/cyl)

parameters to be used for BIOS calculations are:
cylinders=967 heads=255 sectors/track=63 (16065 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
    start 63, size 1930257 (942 Meg), flag 80 (active)
        beg: cyl 0/ head 1/ sector 1;
        end: cyl 890/ head 15/ sector 63
The data for partition 2 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
    start 1930383, size 1930257 (942 Meg), flag 0
        beg: cyl 891/ head 1/ sector 1;
        end: cyl 757/ head 15/ sector 63
The data for partition 3 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
    start 3860640, size 3024 (1 Meg), flag 0
        beg: cyl 758/ head 0/ sector 1;
        end: cyl 760/ head 15/ sector 63
The data for partition 4 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
    start 3863664, size 41328 (20 Meg), flag 0
        beg: cyl 761/ head 0/ sector 1;
        end: cyl 801/ head 15/ sector 63

Note that partitions 1 and 2 are the same size; this is part of the FreeNAS upgrade strategy. What you can do, easily, is to move partition 2 (slice 2) to the end of the disk, because presumably you have many gigs of space free there. So you do your calculations for start and size, then run "fdisk -i da1", make the changes, and if you've done it right, your partition 2 now occupies many GB at the end of the disk.

Once you've done that, you can use bsdlabel to install the BSD partition table, then you can copy over the "a" partition from slice 1, and then you can also create a partition "b" to use for swap. If you look at the stuff in /root/updatep2, you'll also get some further clues as to how to make things run from that partition. If you do that, then you can even do one single upgrade of the system (onto slice 1) without too much trouble, which is kind of cool.

You'll also need to manually twiddle /etc/fstab to make your swap work, and this may be complicated because I believe FreeNAS rewrites fstab upon reboot.

So while this isn't at all impossible, it is significantly paddling against the current, and isn't recommended. It's a substantial learning opportunity, though, and therefore I've only included the briefest outline rather than making a precise how-to.

 

[i_no_k]

How many times do I gotta say YOU DON'T, perhaps writing in uppercase helps ;)

:D

Actually, you can.

Great thanks for explanation!
Something like that I wanted that was to start, just for learn.
Will soon try to pass that way.

 

[jgreco]

No problem. With the advent of "installers" and the evolution towards "ease-of-use", users of systems like FreeNAS sometimes lose track of the bigger picture, that the system is actually based on FreeBSD, and that things possible with FreeBSD are maybe also possible with FreeNAS. I like the idea of an appliance NAS version of FreeBSD, but some of the design decisions - specifically including the assignment of swap space on data drives - are unpleasant tradeoffs. The swap thing in particular is a bit frustrating, because ultimately I'd like to be able to remove a failing drive from a FreeNAS system and replace it without a reboot, but the existence of swap on the drive may mean that the system is headed for a panic, or userland bits are headed towards their doom, when those pages cannot be swapped back in.

 

[eroteme-]

Great thanks for explanation!
Something like that I wanted that was to start, just for learn.
Will soon try to pass that way.

Hi,

I found this thread because I was trying to create a zil on my SSD I used for the FreeNAS disk and was finding harder than I should. I'm from a Linux background so the differences in FreeBSD where confusing. I found a few threads saying that 4 slices was the max; but this thread gave me a few clues (thanks jgreco!) about how to get around this problem.

This thread hasn't been updated for almost a year so I decided to post how I achieved both swap and zil on my 64GB SSD (which FreeNAS was only taking ~3GB). I've included an explanation rather than just a crib as if your going to be changing your NAS system you'll probably need to know whats going on as I suspect, as jgreco has mentioned, upgrades will probably be interesting and you'll need to know what you've done to fix your system after upgrade. If anyone sees any mistakes or any pending doom I may run into due to these changes... please let me know

The FreeNAS system I'm using is as follows:

- HP Proliant N45L Microserver
- AMD Turion(tm) II Neo duel core processor
- 4GB of memory (wish I'd had upgraded to 8GB)
- 4x 2TB Seagate Barracuda 7200rpm as Zpool
- 1x 64GB Sandisk SSD
- FreeNAS-8.3.1-RELEASE-p2-x64

1) Make sure you backup you config, this is done on the web interface and is well worth it as I had to reinstall once due to not being able to recreate partitions on the 2nd slice after deleting (if anyone knows why please let me know... There error I got was ada4s2 file exists!)
2) Now you have your backup open a shell on the system (not remote) and go into runlevel 1. This will allow you to umount the /cfg and /data partitions.

init 1

3) The root (/) filesystem is mounted read only. Remount it rw so you can create a backup directory for /cfg and /data

mount -r -o rw /

4) Mount both /cfg and /data for backup. They may already be mounted so if you get an busy device error it's probably already mounted

mount /cfg
mount /data

5) Make a backupdir in / and copy the /cfg and /data mount point data

mkdir /backupparts
cp -rp /data /backupparts/
cp -rp /cfg /backupparts/

6) Unmount /cfg and /data

umount /cfg
umount /data

7) Now for some math. You'll probably want different partition sizes than me (as you may have different memory size). You'll need these as you need to know how far to create your 3rd and 4th slice to allow your 2nd slice to grow to the required size. Have a look at you disk geom and work out from the partitions you have how big a single block is. Then work out what block offset you need for your partitions. Remember to include the ~1GB that is used for the 2nd slice as it is. Sum all these together and add to the offset given by "gpart list ada4". If you want just a zil you'll need 2 partitions in the 2nd slice.. if like me you want to create zil and swap then 3 partitions.
8) Now we start the destructive stuff, so if you have your FreeNAS system in use, make sure you REALLY want to do this. Now delete both the 3rd and 4th slice which are /cfg and /data. This _may_ change on future releases so is best to check by looking in fstab. My disk is ada4 so will be using this in my examples, your's will most likely be different so you need to find out which disk has the 4 FreeNAS slices on. If you can't find this then you probably don't want to continue with this howto. Please also read the gpart and newfs man pages.

gpart delete -i3 ada4
gpart delete -i4 ada4

9) Recreate your 3rd and 4th slice using the offset calculated from point 7 and the same original size. I'm going to use placeholders for these values. Then format the partitions giving them the same name as before so that the mount points in fstab still work.

gpart add -tfreebsd -b <offset> -s <size> -i3 ada4
gpart add -tfreebsd -b <offset> -s <size> -i ada4

newfs -L FreeNASs3 -U ada4s3
newfs -L FreeNASs4 -U ada4s4

10) IF the last step was correct you should now be able to mount the new slices as per usual. Do this then copy in the backup data. If you can't mount as per usual you'll need to find the label for data and cfg mount points in fstab and redo the steps above .

mount /cfg
mount /data
cp -rp /backuppart/data/* /data
cp -rp /backuppart/cfg/* /cfg

11) Now to resize your 2nd slice so you can partition as required. I don't know what the 2nd slice is for BUT it didn't seem to have any data in… in fact it looked like it wasn't even formatted. I tried to run FreeNAS without it and it runs fine. However, I would leave the same amount of space at the beginning of your second slice just in case. It's the same size as the main 1st slice so I think it's some sort of upgrade partition. You don't need to specify a size as gpart will do this for you and fill unto slice 3.

gpart resize -i2 ada4

12) Now you need to create a partition schema andcreate the partitions with the offsets you calculated in step 7. Remember to use the correct types to the partition your adding. I forgot to change one to swap and could not undo. I had to delete the 2nd slice and couldn't create partitions in it once recreated. This is why I had a 2nd go at it and used my very helpful backup. If anyone knows why I couldn't create partitions in a newly created 2nd slice please let me know.

gpart create -sbsd ada4s2
gpart add -t freebsd-swap -b <offset> -s <size> ada4s2
gpart add -t freebsd-ufs -b <offset> -s <size> ada4s2

13) Now to add one of the partitions as a zil to the zpool. I used the first partition I created in the 2nd slice so the geom number I used is ada4s2a. Check with all is ok with "zpool status <zpool name>"

zpool add <zpool name> log /dev/ada4s2a

14) Now to add the swap. jgreco was right in that they write the fstab out via a script in /etc/rc.d/ix-fstab on boot. If you change this script it's cobblered on boot; this means the script has to be copied over from some where and it's that copy that requires modification. The script is in /conf/base/etc/rc.d/ix-fstab. A word of warning here and this is probably where my hack goes bad. I changed the script just to output a mount point for swap to my new swap partition created on slice 2. This however is _NOT_ encrypted. The swap partitions in the zpool are and I'm not sure if I'm going to come unstuck due to this difference. I think because the zpool is public, it is encrypted for security. The FreeNAS drive is not public so I think it's ok … ? If anyone has any info on this please let me know
Once you have changed your script then reboot and check where you swap is mounted.

Thats it.. Thanks for listening

Eroteme

 

[cyberjock]

Yep.. and if you do some analysis you'll find that both the ZIL and L2ARC will perform quite poorly compared to 2 separate devices. In some cases, adding the ZIL and L2ARC will actually slow down your system because the SSD ends up too busy to really provide a system boost. Because of this, putting the ZIL and L2ARC on the same device is frowned upon in the forums.

Also, in your case, with 4GB of RAM, you're going to be extremely disappointed in the L2ARC. Having an L2ARC requires the index to the L2ARC be stored in RAM(Whoops.. you are already way to low in the RAM department). Also, only 50% of the system RAM can be assigned as the L2ARC, so you can't even use 1/2 your SSD before you run out of space for the index.

If I remember correctly both the manual and my guide has a statement like "money spent on a ZIL or L2ARC is almost always better spent on more RAM or a system that has more RAM first". There's even more reasons I won't go into here, but you really should take that statement to heart. The statement was made because its true.

 

[eroteme-]

Yep.. and if you do some analysis you'll find that both the ZIL and L2ARC will perform quite poorly compared to 2 separate devices. In some cases, adding the ZIL and L2ARC will actually slow down your system because the SSD ends up too busy to really provide a system boost. Because of this, putting the ZIL and L2ARC on the same device is frowned upon in the forums.

Also, in your case, with 4GB of RAM, you're going to be extremely disappointed in the L2ARC. Having an L2ARC requires the index to the L2ARC be stored in RAM(Whoops.. you are already way to low in the RAM department). Also, only 50% of the system RAM can be assigned as the L2ARC, so you can't even use 1/2 your SSD before you run out of space for the index.

If I remember correctly both the manual and my guide has a statement like "money spent on a ZIL or L2ARC is almost always better spent on more RAM or a system that has more RAM first". There's even more reasons I won't go into here, but you really should take that statement to heart. The statement was made because its true.

Yep.. and if you do some analysis you'll find that both the ZIL and L2ARC will perform quite poorly compared to 2 separate devices. In some cases, adding the ZIL and L2ARC will actually slow down your system because the SSD ends up too busy to really provide a system boost. Because of this, putting the ZIL and L2ARC on the same device is frowned upon in the forums.

Also, in your case, with 4GB of RAM, you're going to be extremely disappointed in the L2ARC. Having an L2ARC requires the index to the L2ARC be stored in RAM(Whoops.. you are already way to low in the RAM department). Also, only 50% of the system RAM can be assigned as the L2ARC, so you can't even use 1/2 your SSD before you run out of space for the index.

If I remember correctly both the manual and my guide has a statement like "money spent on a ZIL or L2ARC is almost always better spent on more RAM or a system that has more RAM first". There's even more reasons I won't go into here, but you really should take that statement to heart. The statement was made because its true.

Hi cyberjock,

Thanks for your reply. I did read that ZIL and L2ARC do perform poorly due to disk contention. That's why I decided not to add a L2ARC, but this is where my FreeNAS knowledge fails me; does the L2ARC exist on the FreeNAS boot disk ? If so where (data slice … arc_summary.py doesn't seem to tell me)?.

Your also right about my memory config (although I didn't post my zpool size)... I knew this when I bought the machine. It's quite an easy upgrade to 8GB (the max) but I'm finding the performance very good after adding my ZIL. My machine has never swapped lucky so don't know if the separate swap from zpool has helped. I would imaging a swapping machine to a busy zpool would be quite bad. I'll probably upgrade to 8GB when I have a bit more free money. As I only use my FreeNAS at home it's not really very stressed. Adding the ZIL and moving swap was a good little learning exercise for me which seems to have improved write to NAS performance when transferring large files.

Thanks again cyberjock! If you can let me know where the L2ARC is stored that would be great. And if I see any disk contention / speed issues I'll revert my changes and see if it improves.

Regards

Eroteme

 

[gpsguy]

Instead of upgrading to 8Gb, put 16Gb of ECC RAM in your N54L. It cost me about $150 USD. I also have a N54L.


Sent from my phone

 

[eroteme-]

Instead of upgrading to 8Gb, put 16Gb of ECC RAM in your N54L. It cost me about $150 USD. I also have a N54L.


Sent from my phone

Hi gpsguy.

Funny you should say that... I've _just_ been looking at a forum (http://homeservershow.com/forums/index.php?/topic/5557-16gb-ram-for-n54l/) which says that people has successfully used 16GB RAM in the N54L. The specs say 8GB and I'm worried that the trade off my be stability. Although the FreeNAS is a home system I still like to rely on it. The people in the forum above say they have had no problems. I'd like to get 16GB RAM if it's stable. Whats your setup ? Have you had any issues with 16GB in the N54L ?

The price difference between 8GB and 16GB is only £30 but at £120 for the 16GB I may wait to see if it comes down in price (I have a baby who needs to waist all my money on baby food he throws over my carpets which I'll need to be replace at some point ;) )

Thanks gpsguy for the tip, if you can let me know what your experience is that would be great. I'll go for 16GB if it's stable else be a wus and stick with the spec 8GB.

Kind Regards

Eroteme

 

[gpsguy]

The short answer is that I'm using the memory listed below in my N54L. It recognizes all 16Gb on every boot.

Kingston 16GB (2 x 8GB) 240-Pin DDR3 SDRAM DDR3 1333
ECC Unbuffered Server Memory Model KVR1333D3E9SK2/16G

The longer answer is that it's my ESXi 5.1 box. I'm running FreeNAS on it's predecessor (N40L) with 8Gb of ECC RAM. While 8Gb is adequate for my server (at the moment), had I known that 16Gb was possible, I probably would have gone that route instead.

 

[eroteme-]

Hi,

I decided to go for the 16GB RAM. and it worked! so I now have enough RAM for FreeNAS. Thanks gpsguy for the tip.

I also bought these:
http://www.scan.co.uk/products/lyco...ternal-mini-sas-low-profile-pcie-host-adapter
http://www.ebuyer.com/221596-startech-2-5-4bay-removable-mobile-rack-backplane-satsasbay425

This has given me an extra 4x2.5" hdd and a sata Gbs interface. I havn't got any disks for it yet so time will tell if the PSU is man enough.

Still wondering if I should use an internal USB stick for the FreeNAS OS. Cyberjock.... do you know where the L2ARC is stored?... if not on the FreeNAS partitions then I'll probably leave my SSD as is and just resize the swap partitions to match my new memory size.

Regards

Eroteme http://forums.freenas.org/members/cyberjock.22572/