Remove PERMANENTLY Spare from Pool

[Nmitsia]

Hello. I really need to remove a spare disk on a pool permanently, and create another vdev. Vdev1 has 6 data disks & 1 spare. I have available another 5 disks for Vdev2 creation but i will need 6 total thats why i want to remove permanently the spare drive as part of the vdev. Please help as no option in GUI is available, and i dont want to CLI a command that could possibly degrade my pool or even destroy any data as they are mission critical.

I dont want to replace, but remove entirely the role of spare from my vdev. Please help!

 

[sretalla]

seems SCALE hasn't allowed for that in the GUI as yet.

zpool remove poolname device

 

[Nmitsia]

Thanks for your reply. Unfortunately i get the following message:

cannot remove c254f88f-9755-4988-8891-14f214514fa6: Pool busy; removal may already be in progress

Also as i tried hard removing by hand the spare drive it automatically created a different pool called SPARE inside my RAIDZ1. Started resilvering also...

 

[Nmitsia]

Also when i press DETACH on Spare drive i get:
[EZFS_NOTSUP] Cannot detach root-level vdevs

Error: concurrent.futures.process._RemoteTraceback: """ Traceback (most recent call last): File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 273, in __zfs_vdev_operation op(target, *args) File "libzfs.pyx", line 465, in libzfs.ZFS.__exit__ File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 273, in __zfs_vdev_operation op(target, *args) File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 286, in impl getattr(target, op)() File "libzfs.pyx", line 2249, in libzfs.ZFSVdev.detach libzfs.ZFSException: Cannot detach root-level vdevs During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.9/concurrent/futures/process.py", line 243, in _process_worker r = call_item.fn(*call_item.args, **call_item.kwargs) File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 115, in main_worker res = MIDDLEWARE._run(*call_args) File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 46, in _run return self._call(name, serviceobj, methodobj, args, job=job) File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 40, in _call return methodobj(*params) File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 40, in _call return methodobj(*params) File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1382, in nf return func(*args, **kwargs) File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 282, in detach self.detach_remove_impl('detach', name, label, options) File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 289, in detach_remove_impl self.__zfs_vdev_operation(name, label, impl) File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 275, in __zfs_vdev_operation raise CallError(str(e), e.code) middlewared.service_exception.CallError: [EZFS_NOTSUP] Cannot detach root-level vdevs """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/middlewared/main.py", line 204, in call_method result = await self.middleware._call(message['method'], serviceobj, methodobj, params, app=self) File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1344, in _call return await methodobj(*prepared_call.args) File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1378, in nf return await func(*args, **kwargs) File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1246, in nf res = await f(*args, **kwargs) File "/usr/lib/python3/dist-packages/middlewared/plugins/pool.py", line 1133, in detach await self.middleware.call('zfs.pool.detach', pool['name'], found[1]['guid']) File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1395, in call return await self._call( File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1352, in _call return await self._call_worker(name, *prepared_call.args) File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1358, in _call_worker return await self.run_in_proc(main_worker, name, args, job) File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1273, in run_in_proc return await self.run_in_executor(self.__procpool, method, *args, **kwargs) File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1258, in run_in_executor return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs)) middlewared.service_exception.CallError: [EZFS_NOTSUP] Cannot detach root-level vdevs

 

[Arwen]

Your Spare drive is now in use. That "SPARE" has both the failing drive and the Spare drive. That is how it appears when a Spare drive is in use.

Now I don't know WHY your Spare drive is in use. The pool drive "sdd" does not seem to have any errors.

I will have to let someone else continue, as I don't know how to proceed.

 

[samarium]

Also when i press DETACH on Spare drive i get:
[EZFS_NOTSUP] Cannot detach root-level vdevs

My guess is that you were trying too hard and selected the wrong device, unless of course there are system log events that show an actual hardware issue which caused the spare to activate, but "no errors" doesn't support that.

Spare is not a drive, so it can't be detached, only depopulated. The drive inside spare can be removed as @sretalla said.

If you rebooted in between the original picture and the zpool remove, then the sdX device name could have changed as they can change every reboot, which is why sdX use is discouraged. I don't know why TNS is using those mutable names. For TNC they use gptids, which I also don't like but at least they stay the same across reboots. TNS command line zpool status command reports partuuids like TNC.

If you are having trouble understanding how to do something it is often a good idea to try in a disposable Virtual Machine.

I spun up my TNS VM, and added 6x 4GB virtual disks, in RAIDZ1 + 1x Spare.

Code:

# zpool status test
  pool: test
 state: ONLINE
config:

    NAME                                      STATE     READ WRITE CKSUM
    test                                      ONLINE       0     0     0
      raidz2-0                                ONLINE       0     0     0
        af5a8cf7-35ff-4730-abf1-f5ee5900743c  ONLINE       0     0     0
        6e0ac122-527b-4942-82f2-21d4c728ec83  ONLINE       0     0     0
        ea56a4c5-3782-4f39-81b0-6d3765c72055  ONLINE       0     0     0
        dc9d7341-d059-4b8f-a9db-78d4c123cad8  ONLINE       0     0     0
        4c96c072-a79b-4b62-9e00-ad1d7a185f9c  ONLINE       0     0     0
    spares
      5a3950ed-2d89-4ac3-ae46-2651bd4d4cb8    AVAIL

errors: No known data errors
# zpool status -L test
  pool: test
 state: ONLINE
config:

    NAME        STATE     READ WRITE CKSUM
    test        ONLINE       0     0     0
      raidz2-0  ONLINE       0     0     0
        sde1    ONLINE       0     0     0
        sdi1    ONLINE       0     0     0
        sdg1    ONLINE       0     0     0
        sdf1    ONLINE       0     0     0
        sdh1    ONLINE       0     0     0
    spares
      sdj1      AVAIL

errors: No known data errors
# zpool remove test sdj1
cannot remove sdj1: no such device in pool
# zpool remove test sdj
cannot remove sdj: no such device in pool
# zpool remove test 5a3950ed-2d89-4ac3-ae46-2651bd4d4cb8
# zpool status test
  pool: test
 state: ONLINE
config:

    NAME                                      STATE     READ WRITE CKSUM
    test                                      ONLINE       0     0     0
      raidz2-0                                ONLINE       0     0     0
        af5a8cf7-35ff-4730-abf1-f5ee5900743c  ONLINE       0     0     0
        6e0ac122-527b-4942-82f2-21d4c728ec83  ONLINE       0     0     0
        ea56a4c5-3782-4f39-81b0-6d3765c72055  ONLINE       0     0     0
        dc9d7341-d059-4b8f-a9db-78d4c123cad8  ONLINE       0     0     0
        4c96c072-a79b-4b62-9e00-ad1d7a185f9c  ONLINE       0     0     0

errors: No known data errors
#

 

[sretalla]

Also as i tried hard removing by hand the spare drive it automatically created a different pool called SPARE inside my RAIDZ1. Started resilvering also...

I don't think the spare drive is the drive you think it is...

You need to check in Storage | Disks and see the serial number associated with the spare (sde currently), then match that to the serial number written on the drive. it looks like you accidentally pulled the drive currently shown as sdd instead, hence the spare kicking in.

 

[Nmitsia]

Guys let me explain. As i removed the "Spare" - sde (by hardware removal), automatically RAIDZ1 created SPARE by using other unmaped HDD's that where available to system. Once i plugged back in sde resilvering started. Now resilvering stoped and im back to picture no.1 with "Spare". Then i tried what "sretalla" suggested and it worked!!!

Now i have a fully working system as it should be.
My only question is why ixsystems does not implement a GUI button for such a simple task?

Sometimes they make me feel bad things... (keep up the good work guys!)

 

[samarium]

Guys let me explain. As i removed the "Spare" - sde (by hardware removal), automatically RAIDZ1 created SPARE by using other unmaped HDD's that where available to system. Once i plugged back in sde resilvering started. Now resilvering stoped and im back to picture no.1 with "Spare". Then i tried what "sretalla" suggested and it worked!!!
View attachment 68354

Now i have a fully working system as it should be.
My only question is why ixsystems does not implement a GUI button for such a simple task?

Sometimes they make me feel bad things... (keep up the good work guys!)

The webui is a a veneer on top of the functionality if the command line. Not everything is exposed yet, and may never be exposed. You should raise a ix systems truenas jira ticket and raise the issue for enhancement.

 

[Steve the Pinkus]

I was in the very same situation as shown in this picture:

Thanks for your reply. Unfortunately i get the following message:

Also as i tried hard removing by hand the spare drive it automatically created a different pool called SPARE inside my RAIDZ1. Started resilvering also...
View attachment 68320

Indeed I found this thread while looking for a solution.
The hint was in an older (2019) thread: while apparently we cannot detach the spare drive (sde), we can detach (via the webui) the "replaced" drive (sdd), this promotes sde as a full member of the data VDEV and makes the sdd available (assuming it has no issue).

My case started with errors and erratic behavior of one disk, which activated the spare drive. It turned out the issue was not the disk but the board, which I replaced, with the result as in the picture.