Nvidia driver dont work in jail for hardware acceleration

[picarica]

so hello hello i am trying to get hardware acceleration working for my truenas my specs are

MB: asus a58m-k
CPU: AMD A4-6320 APU with Radeon(tm) HD Graphics
MEM: 4GB (so far 12GB coming soon)
GPU nvidia 1050 TI 4GB (i also tried radeion r7 260 but i found out it doesnt support h265 decoding)
and i am using 5 disk wich one of is booting one in raidz1

so i have installed jellyfin in jail sucesfully but i cannot get nvidia drivers working, i managed to get it installed but nvidia-smi says make sure nvidia driver is running, when i have it set in rc.conf
so how do you manage to make it work in jail? i have nowhere have seen answers only unaswered questions, how does one get hardware acceleration for jail? or can i somehow make my r7 260 work ? i need it for jellyfin streaming, i am thinking of switching to gentoo server at this points i dont know what am i doing work, dmesg doesmt even work for me so i cant check if it detected the GPU and pciconf -l also ouputs error that /dev/pci doesnt exist, whats wrong with this jail system ?

 

[picarica]

i forgot i am using truenas CORE

 

[winnielinnie]

i am using truenas CORE

i have installed jellyfin in jail sucesfully

How?

Jellyfin offers no version for FreeBSD, of which TrueNAS Core is based.

 

[picarica]

How?

Jellyfin offers no version for FreeBSD, of which TrueNAS Core is based.

i just followed this guide

jellyfin-server-freebsd/Installation_TrueNAS_GUI.md at main · Thefrank/jellyfin-server-freebsd

jellyfin-server component for freebsd. Contribute to Thefrank/jellyfin-server-freebsd development by creating an account on GitHub.

github.com

with some personal tweaks its just as easy as installing it in freebsd except video drivers are not working and hardware acceleration, thats the only problem

 

[Patrick M. Hausen]

The hardware driver needs to be loaded on the host. FreeBSD jails share a kernel and cannot load kernel extensions or access hardware directly.

If pciconf -l and whatever else needs to be done does work on the host we can try to map the /dev/pci device into the jail via devfs rules. I never used graphics cards with FreeBSD for HW acceleration, but I can give you a hand with the jail. But first you must somehow activate the hardware on the host.

 

[winnielinnie]

i just followed this guide

Unreal! I'm going to try that myself.

It looks like you can't update via pkg (repository), since no such port exists. But I guess to "update Jellyfin" you would need to re-download from their GitHub again?

EDIT: Even though my CPU (Xeon E-2144G) supports Intel QuickSync video transcoding, for some reason ffmpeg available from Fresh Ports is compiled without QSV support.

Code:

MFX=off: Intel MediaSDK (aka Quick Sync Video)

@Patrick M. Hausen don't worry about me right now. I'm just playing around with "Jellyfin on FreeBSD", not migrating to it (yet?) I don't want to hijack this thread, just adding in my personal experience as I'm interested in Jellyfin as a viable alternative on TrueNAS Core ("FreeBSD-based").

 

[picarica]

The hardware driver needs to be loaded on the host. FreeBSD jails share a kernel and cannot load kernel extensions or access hardware directly.

If pciconf -l and whatever else needs to be done does work on the host we can try to map the /dev/pci device into the jail via devfs rules. I never used graphics cards with FreeBSD for HW acceleration, but I can give you a hand with the jail. But first you must somehow activate the hardware on the host.

oh alright i have not tried but beacuse the host (non jailed shell) doesnt allow me to use pkg so i thought i had to install drivers in the jail, so what is the step to step guide? i install gpu, and then i just load it ? in rc.conf and then install nviida drivers in jail and it should be working ?|
could i also get help in installing nvidia drivers? when i try to install it in jail i get this error
pkg: fail to create temorary filefor /boot/modules/nvidia.ko: read-only file system
i can install nvidia driver in any other jail but why it wont work for this one where i havea already everything set up ?, it is also jail from qbittorrent plugin

 

[sretalla]

why it wont work for this one

t is also jail from qbittorrent plugin

There you have your answer... don't mess with Plugin jails (or even use them at all really), just do what you need in a properly created new jail.

 

[Patrick M. Hausen]

You cannot load kernel modules inside jails. The jail does not have access to the kernel. So is the nvidia.ko file all that is necessary? Then install the package inside the jail - this way it can be updated by pkg update - and then from the host do:

Code:

kldload /mnt/<pool>/iocage/jails/<jail>/root/boot/modules/nvidia.ko

If that works the next step is trying to automate that in a way that it happens before jails are started. I'd try pre init or post init tasks for that.

And then we need to find out what needs to be done so the jail can access that driver.

But first things first ...

 

[picarica]

n't mess with Plugin jails (or

b

You cannot load kernel modules inside jails. The jail does not have access to the kernel. So is the nvidia.ko file all that is necessary? Then install the package inside the jail - this way it can be updated by pkg update - and then from the host do:

Code:

kldload /mnt/<pool>/iocage/jails/<jail>/root/boot/modules/nvidia.ko

If that works the next step is trying to automate that in a way that it happens before jails are started. I'd try pre init or post init tasks for that.

And then we need to find out what needs to be done so the jail can access that driver.

But first things first ...

ok so as far as i understand, open up jail inside nvidia drivers inside them, and them just kdload them into main kernel? i am gonna try it out tommorow lets hope it works

 

[picarica]

You cannot load kernel modules inside jails. The jail does not have access to the kernel. So is the nvidia.ko file all that is necessary? Then install the package inside the jail - this way it can be updated by pkg update - and then from the host do:

Code:

kldload /mnt/<pool>/iocage/jails/<jail>/root/boot/modules/nvidia.ko

If that works the next step is trying to automate that in a way that it happens before jails are started. I'd try pre init or post init tasks for that.

And then we need to find out what needs to be done so the jail can access that driver.

But first things first ...

tried in new jail it seems to faile i get this errro when trying your command
kldload /mnt/NAS/iocage/jails/TES-DRUIVER/root/boot/modules/nvidia.ko
kldload: an error occurred while loading module /mnt/NAS/iocage/jails/TES-DRUIVER/root/boot/modules/nvidia.ko. Please check dmesg(8) for more details.
and in dmesg
linker_load_file: /mnt/NAS/iocage/jails/TES-DRUIVER/root/boot/modules/nvidia.ko - unsupported file type
and i am not even sure if the card got recognized? here is pciconf -l output

Code:

root@truenas[~]# pciconf -l
hostb0@pci0:0:0:0:    class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1410 subvendor=0x1043 subdevice=0x8526
pcib1@pci0:0:2:0:    class=0x060400 rev=0x00 hdr=0x01 vendor=0x1022 device=0x1412 subvendor=0x1043 subdevice=0x8526
ahci0@pci0:0:17:0:    class=0x010601 rev=0x40 hdr=0x00 vendor=0x1022 device=0x7801 subvendor=0x1043 subdevice=0x8622
ohci0@pci0:0:18:0:    class=0x0c0310 rev=0x11 hdr=0x00 vendor=0x1022 device=0x7807 subvendor=0x1043 subdevice=0x8622
ehci0@pci0:0:18:2:    class=0x0c0320 rev=0x11 hdr=0x00 vendor=0x1022 device=0x7808 subvendor=0x1043 subdevice=0x8622
ohci1@pci0:0:19:0:    class=0x0c0310 rev=0x11 hdr=0x00 vendor=0x1022 device=0x7807 subvendor=0x1043 subdevice=0x8622
ehci1@pci0:0:19:2:    class=0x0c0320 rev=0x11 hdr=0x00 vendor=0x1022 device=0x7808 subvendor=0x1043 subdevice=0x8622
intsmb0@pci0:0:20:0:    class=0x0c0500 rev=0x16 hdr=0x00 vendor=0x1022 device=0x780b subvendor=0x1043 subdevice=0x8622
none0@pci0:0:20:2:    class=0x040300 rev=0x01 hdr=0x00 vendor=0x1022 device=0x780d subvendor=0x1043 subdevice=0x8576
isab0@pci0:0:20:3:    class=0x060100 rev=0x11 hdr=0x00 vendor=0x1022 device=0x780e subvendor=0x1043 subdevice=0x8622
pcib2@pci0:0:20:4:    class=0x060401 rev=0x40 hdr=0x01 vendor=0x1022 device=0x780f subvendor=0x0000 subdevice=0x0000
pcib3@pci0:0:21:0:    class=0x060400 rev=0x00 hdr=0x01 vendor=0x1022 device=0x43a0 subvendor=0x1022 subdevice=0x0000
pcib4@pci0:0:21:2:    class=0x060400 rev=0x00 hdr=0x01 vendor=0x1022 device=0x43a2 subvendor=0x1022 subdevice=0x0000
ohci2@pci0:0:22:0:    class=0x0c0310 rev=0x11 hdr=0x00 vendor=0x1022 device=0x7807 subvendor=0x1043 subdevice=0x8622
ehci2@pci0:0:22:2:    class=0x0c0320 rev=0x11 hdr=0x00 vendor=0x1022 device=0x7808 subvendor=0x1043 subdevice=0x8622
hostb1@pci0:0:24:0:    class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1400 subvendor=0x0000 subdevice=0x0000
hostb2@pci0:0:24:1:    class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1401 subvendor=0x0000 subdevice=0x0000
hostb3@pci0:0:24:2:    class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1402 subvendor=0x0000 subdevice=0x0000
hostb4@pci0:0:24:3:    class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1403 subvendor=0x0000 subdevice=0x0000
hostb5@pci0:0:24:4:    class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1404 subvendor=0x0000 subdevice=0x0000
hostb6@pci0:0:24:5:    class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1405 subvendor=0x0000 subdevice=0x0000
vgapci0@pci0:1:0:0:    class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1c82 subvendor=0x1458 subdevice=0x3763
none1@pci0:1:0:1:    class=0x040300 rev=0xa1 hdr=0x00 vendor=0x10de device=0x0fb9 subvendor=0x1458 subdevice=0x3763
atapci0@pci0:2:5:0:    class=0x010400 rev=0x02 hdr=0x00 vendor=0x1095 device=0x3114 subvendor=0x1095 subdevice=0x7114
re0@pci0:4:0:0:    class=0x020000 rev=0x11 hdr=0x00 vendor=0x10ec device=0x8168 subvendor=0x1043 subdevice=0x859e

vgapci0: <VGA-compatible display> port 0xe000-0xe07f mem 0xfd000000-0xfdffffff,0xe0000000-0xefffffff,0xf0000000-0xf1ffffff irq 18 at device 0.0 on pci1 vgapci0: Boot video devic
this is what i fund in dmes,g why is it not working ?

 

[Patrick M. Hausen]

Is the jail the same FreeBSD version (13.1) as your NAS host.

 

[picarica]

yes of course 13.1

 

[PnoT]

I'm using a P2000 and core and wanted to know if you could get the Nvidia drivers loaded and working correctly. I'd rather not have to upgrade to SCALE if possible.