Method 'get_pci_ids_for_gpu_isolation' not found in 'device'

[Procus Aquila]

Hi, I found a solution, you have to add the necessary PCI slots for your GPU under Devices.

In my case once for the GPU (GTX 1050ti) and for the audio controller.

If anyone needs step-by-step instructions, please let me know.

Hi, I cannot confirm that this works. I tried it several times and even tried to pass through the dedicated graphics card with a new VM.

Of course, I also switched the existing VM offline.
And set up a Windows 10 as a test.
In your case, the parameters seem to be different.

I also noticed that I can no longer change the RAM in the existing VM. TrueNas confirms my “change”, but it is not even applied.

I also suspect that there is a serious bug here, sorry, I'd rather wait for a clean fix before a “workaround” causes me even more problems

 

[afach_luc]

I’d appreciate a guide to the solution. Anybody else notice that once a virtual machine is created as well, you can’t edit any of the properties? The save just fails to save. Seem like a huge bug.

Step by step troubleshooting guide:

1. Create a New VM as usual
2. Add your desired GPU
3. Click next and save the VM
4. Close the error message and the window for creating the VM
5. Click edit on the newly created VM
6. No GPU should now be entered
7. click save again
8. Now click on device
9. Now add a new PCI passthrough device here
10. Now look for your graphics card (in my case Nvidia), if like me the device is Not Available on port 0000:01:00.0, first save the available device (in my case the controller for the card's audio)
11. (optional if step 10 worked completely) restart your entire system
12. (optional if step 10 worked completely) go back to the device after restarting and now add the VGA to your GPU for PCI.
13. done, now the GPU should be entered, !!!if you change the GPU settings you have to repeat all steps!!!

I hope that it works now, I would be very happy about any feedback, whether positive or negative ;^)

 

[afach_luc]

 

[afach_luc]

View attachment 74307

 

[Procus Aquila]

Step by step troubleshooting guide:

1. Create a New VM as usual
2. Add your desired GPU
3. Click next and save the VM
4. Close the error message and the window for creating the VM
5. Click edit on the newly created VM
6. No GPU should now be entered
7. click save again
8. Now click on device
9. Now add a new PCI passthrough device here
10. Now look for your graphics card (in my case Nvidia), if like me the device is Not Available on port 0000:01:00.0, first save the available device (in my case the controller for the card's audio)
11. (optional if step 10 worked completely) restart your entire system
12. (optional if step 10 worked completely) go back to the device after restarting and now add the VGA to your GPU for PCI.
13. done, now the GPU should be entered, !!!if you change the GPU settings you have to repeat all steps!!!

I hope that it works now, I would be very happy about any feedback, whether positive or negative ;^)

Thank you for the instructions, please check whether you can now also make RAM changes in the VM, e.g. RAM, CPU?

What happens then?

When I try it, the graphics unit is gone and the memory change is displayed as accepted, but this is not correct.

Don't get me wrong, the idea with your workaround may be good, but I see other problems.

Especially since I already wrote that I can create a completely new VM in the first place (more)

 

[afach_luc]

Thank you for the instructions, please check whether you can now also make RAM changes in the VM, e.g. RAM, CPU?

What happens then?

When I try it, the graphics unit is gone and the memory change is displayed as accepted, but this is not correct.

Don't get me wrong, the idea with your workaround may be good, but I see other problems.

Especially since I already wrote that I can create a completely new VM in the first place (more)

The problem with the RAM could be related to the configuration of the ports, unfortunately I don't have a solution for this.
If you want to make changes on a new machine, you may have to repeat the entire GPU process.
For me, a RAM or core change now works both in my older VMs and in the new one.

One option would be to downgrade to the previous version if it doesn't work for you.

 

[Procus Aquila]

The problem with the RAM could be related to the configuration of the ports, unfortunately I don't have a solution for this.
If you want to make changes on a new machine, you may have to repeat the entire GPU process.
For me, a RAM or core change now works both in my older VMs and in the new one.

One option would be to downgrade to the previous version if it doesn't work for you.

 

[Procus Aquila]

Then I expressed myself incorrectly.

The GPU can certainly be activated in an existing VM, as in your "workaround", but I cannot change any RAM memory here. As soon as I try this, TrueNas displays - executed. However, the GPU has been lost, so as with your workaround, I have to mount it again, then start the VM and you can see that the RAM has remained the same.

I have enough RAM memory.

I can no longer set up a completely new VM after the update to 23.10.1.
Then I only get this error and setting up the GPU after this workaround is no longer possible.

I think a downgrade with the bug is too risky, you don't know which parameters might be destroyed.

Nevertheless, I hope that TrueNas itself has created a fix soon.

 

[tatersalad]

Code:

 Error: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/plugins/vm/supervisor/supervisor.py", line 182, in start
    if self.domain.create() < 0:
       ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/libvirt.py", line 1373, in create
    raise libvirtError('virDomainCreate() failed')
libvirt.libvirtError: internal error: process exited while connecting to monitor: 2024-01-08T03:03:17.695164Z qemu-system-x86_64: -device {"driver":"vfio-pci","host":"0000:04:00.0","id":"hostdev0","bus":"pci.0","addr":"0x6"}: vfio 0000:04:00.0: failed to setup container for group 51: Failed to set group container: Operation not permitted

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 201, in call_method
    result = await self.middleware._call(message['method'], serviceobj, methodobj, params, app=self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1342, in _call
    return await methodobj(*prepared_call.args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 177, in nf
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 44, in nf
    res = await f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/vm/vm_lifecycle.py", line 54, in start
    await self.middleware.run_in_thread(self._start, vm['name'])
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1254, in run_in_thread
    return await self.run_in_executor(self.thread_pool_executor, method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1251, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/vm/vm_supervisor.py", line 68, in _start
    self.vms[vm_name].start(vm_data=self._vm_from_name(vm_name))
  File "/usr/lib/python3/dist-packages/middlewared/plugins/vm/supervisor/supervisor.py", line 191, in start
    raise CallError('\n'.join(errors))
middlewared.service_exception.CallError: [EFAULT] internal error: process exited while connecting to monitor: 2024-01-08T03:03:17.695164Z qemu-system-x86_64: -device {"driver":"vfio-pci","host":"0000:04:00.0","id":"hostdev0","bus":"pci.0","addr":"0x6"}: vfio 0000:04:00.0: failed to setup container for group 51: Failed to set group container: Operation not permitted

no dice over here, dl380p g8 with a p4000 gpu. I have tried everything I know to try. bios is set, the shared memory is disabled. helpppppp lol

 

[afach_luc]

Code:

 Error: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/plugins/vm/supervisor/supervisor.py", line 182, in start
    if self.domain.create() < 0:
       ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/libvirt.py", line 1373, in create
    raise libvirtError('virDomainCreate() failed')
libvirt.libvirtError: internal error: process exited while connecting to monitor: 2024-01-08T03:03:17.695164Z qemu-system-x86_64: -device {"driver":"vfio-pci","host":"0000:04:00.0","id":"hostdev0","bus":"pci.0","addr":"0x6"}: vfio 0000:04:00.0: failed to setup container for group 51: Failed to set group container: Operation not permitted

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 201, in call_method
    result = await self.middleware._call(message['method'], serviceobj, methodobj, params, app=self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1342, in _call
    return await methodobj(*prepared_call.args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 177, in nf
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 44, in nf
    res = await f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/vm/vm_lifecycle.py", line 54, in start
    await self.middleware.run_in_thread(self._start, vm['name'])
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1254, in run_in_thread
    return await self.run_in_executor(self.thread_pool_executor, method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1251, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/vm/vm_supervisor.py", line 68, in _start
    self.vms[vm_name].start(vm_data=self._vm_from_name(vm_name))
  File "/usr/lib/python3/dist-packages/middlewared/plugins/vm/supervisor/supervisor.py", line 191, in start
    raise CallError('\n'.join(errors))
middlewared.service_exception.CallError: [EFAULT] internal error: process exited while connecting to monitor: 2024-01-08T03:03:17.695164Z qemu-system-x86_64: -device {"driver":"vfio-pci","host":"0000:04:00.0","id":"hostdev0","bus":"pci.0","addr":"0x6"}: vfio 0000:04:00.0: failed to setup container for group 51: Failed to set group container: Operation not permitted

no dice over here, dl380p g8 with a p4000 gpu. I have tried everything I know to try. bios is set, the shared memory is disabled. helpppppp lol

Is the GPU even visible?

 

[mkurpiewski]

I was having all the same issues with VMs (Method not found error when adding GPU, and inability to change VM settings such as RAM and CPU) and was also unable to change network settings (change IP address, add bridge interface) through the WebUI. I did not try using the TrueNAS CLI, so I cannot be sure if this is a WebUI issue or if something else broke in the middleware. In any case, it seems to be affecting more than just the VM configuration. I reverted back to 23.10.0.1 for the time being and everything is back to normal.

 

[tatersalad]

Its odd that I can plug in a quadro nvs 295 and it will accept it but the p4000 it throws an absolute fit. the 295 is a cake walk to get running. How does one downgrade I would gladly do that

 

[mkurpiewski]

Systems Settings->Boot

Click on the menu button in the far right column of the 23.10.0.1 row, select Activate and then reboot.

 

[Procus Aquila]

Systems Settings->Boot

Click on the menu button in the far right column of the 23.10.0.1 row, select Activate and then reboot.

Hey, thanks for the tip. I forgot about changing the loader ;)

So you can safely change again. Many thanks

 

[Ashkaan]

Step by step troubleshooting guide:

1. Create a New VM as usual
2. Add your desired GPU
3. Click next and save the VM
4. Close the error message and the window for creating the VM
5. Click edit on the newly created VM
6. No GPU should now be entered
7. click save again
8. Now click on device
9. Now add a new PCI passthrough device here
10. Now look for your graphics card (in my case Nvidia), if like me the device is Not Available on port 0000:01:00.0, first save the available device (in my case the controller for the card's audio)
11. (optional if step 10 worked completely) restart your entire system
12. (optional if step 10 worked completely) go back to the device after restarting and now add the VGA to your GPU for PCI.
13. done, now the GPU should be entered, !!!if you change the GPU settings you have to repeat all steps!!!

I hope that it works now, I would be very happy about any feedback, whether positive or negative ;^)

This worked great for me! Thank you so much!

 

[Isma]

That's just not useful information. If you have "the same problem", add to the discussion with details of your setup, steps you've taken, results you've gotten/expectations, etc.

update without solution? tremendous error and what happens is it dragging?

 

[TrueRoux]

same error and behavior on my side with my windows VM where i can't change gpu or vm parameters ,gpu passthrough is working fine but the parameters cant be changed , hope Truenas fix it soon in the next patch !

 

[bartii]

Hi, is there an issue already created for this?

I have the same issue. I cannot assign a GPU to a VM.
A "test" downgrade to 23.10.0.1 fixed the problem.

 

[stady__]

Hey,
I have the same issue.
I cannot use my VM properly without GPU.

 

[afach_luc]

Hi, I've often read that many people still can't use their VMs with a GPU.
Have you ever tried my method?