Need advice with creating multiple VMs and Workstations

[jbarry14]

I am pretty new to Truenas and servers. I have a small business and I am looking to create several VM PC workstations around the building. I was hoping I could use my existing Truenas Core server to do this. The idea was to have multiple Windows VMs running on a server and then having some sort of KVM extender dock to connect the peripherals to. I was trying to set this up so I did not need a physical computer and VNC on the receiving end. I have done it before where I have had a rackmount PC and used a KVM extender over IP, then having a small KVM receiver to setup a mini workstation with just peripherals. But, instead of buying several PCs, I was hoping to either use my existing server, or build a new system to accomplish this.

Thanks in advance!

My current system is running TrueNas Core 13.0-U5.3
2X - AMD EPYC 7542 32-Core Processors
256GB DDR4-3200

 

[HoneyBadger]

I was trying to set this up so I did not need a physical computer and VNC on the receiving end.

Hey @jbarry14

The only way to accomplish the result of only having multiple KVMs plugged directly into your server would be to have multiple independent GPUs and USB controllers that you could pass on a one-to-one basis into your VMs.

IMO, a far more scalable approach is to use something like a thin-client as your "receiving end" PC, and use a protocol like RDP/SPICE or similar (depending on your thin client and guest's operating system) to connect over the network to your TrueNAS VMs.

I would also suggest TrueNAS SCALE for this, if you intend to make heavy use of VMs.

Cheers!

 

[jbarry14]

Hey @jbarry14

The only way to accomplish the result of only having multiple KVMs plugged directly into your server would be to have multiple independent GPUs and USB controllers that you could pass on a one-to-one basis into your VMs.

IMO, a far more scalable approach is to use something like a thin-client as your "receiving end" PC, and use a protocol like RDP/SPICE or similar (depending on your thin client and guest's operating system) to connect over the network to your TrueNAS VMs.

I would also suggest TrueNAS SCALE for this, if you intend to make heavy use of VMs.

Cheers!

Thanks for the info. Is it possible to get a single GPU that could be split between multiple VMs? As far as a "Thin Client", are you referring to something like a raspberry pi? I wouldn't mind building a new dedicated system for this. Any recommendations on the best hardware for a truenas scale build? Or could I add a good single GPU to my current system? I have installed a Windows VM on my current server and used VNC on a raspberry pi, but it was extremely laggy, and I am assuming that was due to no graphics acceleration.

 

[probain]

Use Raspberry Pi's as the thin clients, and connect with some RDP/Spice/VNC-client. It'll be cost effective. And you can put those savings into the performance of the server instead.

Desktop performance remotely will be tricky though. Not impossible. But tricky for sure. And there will be diminishing returns.
Get som proof of concept done first. Hyper convergence is the new hotness, but it has to make sense from a workload point of view.

Do you need accelerated graphics? Do you need "workstation" performance? Or are you mostly doing lite office work, mostly browser based?

 

[jbarry14]

Use Raspberry Pi's as the thin clients, and connect with some RDP/Spice/VNC-client. It'll be cost effective. And you can put those savings into the performance of the server instead.

Desktop performance remotely will be tricky though. Not impossible. But tricky for sure. And there will be diminishing returns.
Get som proof of concept done first. Hyper convergence is the new hotness, but it has to make sense from a workload point of view.

Do you need accelerated graphics? Do you need "workstation" performance? Or are you mostly doing lite office work, mostly browser based?

Could you explain how RDP/Spice works and how I would create this? I did create a windows VM on my current server and used the built in VNC client and connected using a raspberry pi, But it was extremely laggy performance.

I would be using these workstations primarily for mostly office work, Emails, Web browsing, Microsoft office programs, etc.

 

[probain]

Could you explain how RDP/Spice works and how I would create this? I did create a windows VM on my current server and used the built in VNC client and connected using a raspberry pi, But it was extremely laggy performance.

I would be using these workstations primarily for mostly office work, Emails, Web browsing, Microsoft office programs, etc.

Ouff... That would be a lot of typing... Your question raises a little bit of a red flag for me. So please let me steer my answer into another direction. Because I don't just want to give you a bad "just google it"-reply.

I'm gonna first recommend that you'd actually do a simpler setup. If all you're gonna do is lighter office work. Where most of what is done, is either in browser. Or some larger spreadsheets? Then go with something similar to Intel NUCs. There are great alternatives. Mostly, You'll get far less headaches. FAR, FAR less! And for so much less money!
Not only because the performance of the thin client route is going to be a tricky point. You'd probably have to invest quite substantial amounts of time and money. And remember, diminishing returns are real here.

Things that you'd have to make sure works. All of which will require large amounts of research, and double checking. Followed by probably large amounts of eventual trouble shooting.
Since you're using this for a business setting. I'm raising the bar quite a bit here.

• Getting shared GPU to work.
• Being reliable enough for a business setting.
• Networking VM <-> Host
• Remote connections (RDP/VNC/Spice).
• Will you be able to use that USB-stick in the future?
• Networking speed.
• Licensing to stay legal.
• Do you have employees?
• Pool layouts for multiple desktop VMs.
• Class 1 vs Class 2 Hypervisors.
• Much much more

Now, I don't want to be a debbie downer. All of this above to me sounds like a really fun project. And I'm building something similar for my self. But my results with GPU's and Scales VMs, aren't great at the moment. Sharing a GPU between several Guests is still on my to-try-list.
I bet you too would be hopefully on a great learning journey. But I wouldn't set it up for my customers this way. And I don't want to harp on your idea. It is your business, and subsequent time. So if/when you do try, start with reasonable proof of concept before you sink tonnes of cash into new hardware.

If someone has input regarding shared GPU (Arc maybe?), then that would be fantastic.

 

[jbarry14]

Ouff... That would be a lot of typing... Your question raises a little bit of a red flag for me. So please let me steer my answer into another direction. Because I don't just want to give you a bad "just google it"-reply.

I'm gonna first recommend that you'd actually do a simpler setup. If all you're gonna do is lighter office work. Where most of what is done, is either in browser. Or some larger spreadsheets? Then go with something similar to Intel NUCs. There are great alternatives. Mostly, You'll get far less headaches. FAR, FAR less! And for so much less money!
Not only because the performance of the thin client route is going to be a tricky point. You'd probably have to invest quite substantial amounts of time and money. And remember, diminishing returns are real here.

Things that you'd have to make sure works. All of which will require large amounts of research, and double checking. Followed by probably large amounts of eventual trouble shooting.
Since you're using this for a business setting. I'm raising the bar quite a bit here.

• Getting shared GPU to work.
• Being reliable enough for a business setting.
• Networking VM <-> Host
• Remote connections (RDP/VNC/Spice).
• Will you be able to use that USB-stick in the future?
• Networking speed.
• Licensing to stay legal.
• Do you have employees?
• Pool layouts for multiple desktop VMs.
• Class 1 vs Class 2 Hypervisors.
• Much much more

Now, I don't want to be a debbie downer. All of this above to me sounds like a really fun project. And I'm building something similar for my self. But my results with GPU's and Scales VMs, aren't great at the moment. Sharing a GPU between several Guests is still on my to-try-list.
I bet you too would be hopefully on a great learning journey. But I wouldn't set it up for my customers this way. And I don't want to harp on your idea. It is your business, and subsequent time. So if/when you do try, start with reasonable proof of concept before you sink tonnes of cash into new hardware.

If someone has input regarding shared GPU (Arc maybe?), then that would be fantastic.

Yes, this all seems extremely in depth. We have about 10 Employees. They would be using these different workstations to check on project status/updates. Send emails. Create PDFs, etc. I do want as little headache as possible as we are a small business and I cant afford a lot of downtime. I just thought it would be nice to use the hardware of the server I have already built. I thought the server I built has some great hardware and I have a lot of power to spare for VMs. I did create one windows VM and connected to it using a raspberry pi and VNC, but it was very laggy, and I think that is because there is no graphics acceleration on my server. I would like to be able to use my current server or build a new server to try this.

I have researched the Intel NUC route, and maybe with the sales coming up, I could get some good deals.
Maybe the best thing for me, for now, would be to go the Intel NUC route. Get a rackmount for the NUCs and then use a KVM over IP device. Or, if there is something better or cheaper than KVM over IP.

 

[probain]

Yes, this all seems extremely in depth. We have about 10 Employees. They would be using these different workstations to check on project status/updates. Send emails. Create PDFs, etc. I do want as little headache as possible as we are a small business and I cant afford a lot of downtime. I just thought it would be nice to use the hardware of the server I have already built. I thought the server I built has some great hardware and I have a lot of power to spare for VMs. I did create one windows VM and connected to it using a raspberry pi and VNC, but it was very laggy, and I think that is because there is no graphics acceleration on my server. I would like to be able to use my current server or build a new server to try this.

I have researched the Intel NUC route, and maybe with the sales coming up, I could get some good deals.
Maybe the best thing for me, for now, would be to go the Intel NUC route. Get a rackmount for the NUCs and then use a KVM over IP device. Or, if there is something better or cheaper than KVM over IP.

They are so small. So just have them on the spot. Beneath the monitor, and your users will never be bothered by them. Or screw them under a desk, or whatever. They don't need big drives either, since you have a great NAS on your network . And you'll save your self lots of cash on KVMs. Remember, that those are a potential for headache as well. Just look at what Wendell from Level1Tech's went through to build proper ones.

Your server hardware specs do seem quite good. Nothing about your pools though. The CPU's are even far overkill for a NAS. But remember TrueNas is a storage appliance with other features built ontop of it. It will never be as performant and capable like a proper class 1 hypervisor (vmware, proxmox, xcp-ng etc.) on the same hardware.

Use your server for running the storage and stuff. And put a cheap GPU in it. You'll then have a really good platform to do experiments on. So you can learn the fundamentals of running an environment that you have in your vision for the future. The rabbit hole goes deep..... so so very deep

 

[jbarry14]

They are so small. So just have them on the spot. Beneath the monitor, and your users will never be bothered by them. Or screw them under a desk, or whatever. They don't need big drives either, since you have a great NAS on your network . And you'll save your self lots of cash on KVMs. Remember, that those are a potential for headache as well. Just look at what Wendell from Level1Tech's went through to build proper ones.

Your server hardware specs do seem quite good. Nothing about your pools though. The CPU's are even far overkill for a NAS. But remember TrueNas is a storage appliance with other features built ontop of it. It will never be as performant and capable like a proper class 1 hypervisor (vmware, proxmox, xcp-ng etc.) on the same hardware.

Use your server for running the storage and stuff. And put a cheap GPU in it. You'll then have a really good platform to do experiments on. So you can learn the fundamentals of running an environment that you have in your vision for the future. The rabbit hole goes deep..... so so very deep

I think I will just start with a few Intel NUCs to begin with. I wanted to do KVM over IP because the workstations move around quite frequently. It just made it easy to plug a small box into an ethernet jack and it was ready to go wherever you were in the building. I would be interested in building an entire new unit for scale or proxmox or something like that to run VMs in the future.

The server is definitely overkill. I did design it originally with the thought of running VMs. We have 5 designers working off of the server constantly and I am working on getting everyone on 10GB networking here. When I built the server, I installed a Intel X540T2 for 10GB. Hopefully that will work well.
My main storage pool is made up of 12 drives. 4 - 4TB Seagate Ironwolf SSDs in Z2 and 10 - 2TB Seagate Ironwolf SSDs in Z2. And two 1TB NvME drives in mirror.

Is there a good GPU that would be recommended that could possibly be split between multiple VMs and would be good for experimentation?

 

[probain]

I think I will just start with a few Intel NUCs to begin with. I wanted to do KVM over IP because the workstations move around quite frequently. It just made it easy to plug a small box into an ethernet jack and it was ready to go wherever you were in the building.

You'll still need power. And where you got power, you have your self a NUC-station easily. And the nucs will take just as much (little) space as a decent KVM any way. And you'll save your self (and your employees) tonnes of frustration

Is there a good GPU that would be recommended that could possibly be split between multiple VMs and would be good for experimentation?

I haven't found one (yet). Have a look at this post, and you'll see that it isn't quite trivial (as in PnP) to do either. But if you do manage to find something. Do let me know There are a few youtuber videos where they do such stuff. But none with Truenas as a hypervisor, IIRC.
I'm kind of hoping that Intel will do interesting things with their Arc GPU's. But it's gonna be a few years for sure before it's reasonably feasible.

 

[Patrick M. Hausen]

I did create one windows VM and connected to it using a raspberry pi and VNC, but it was very laggy, and I think that is because there is no graphics acceleration on my server. I would like to be able to use my current server or build a new server to try this.

It's because VNC while well established and open is a crappy protocol. Use RDP instead. It's Microsoft proprietary, builtin to Windows, and generally works great. You just need to install an RDP client on your Raspberry Pi and enable "remote access" in the Windows VM.

 

[Patrick M. Hausen]

Is there a good GPU that would be recommended that could possibly be split between multiple VMs and would be good for experimentation?

TrueNAS can to my knowledge not split a GPU between VMs.

Please save yourself some money and headache and try RDP first before you do anything else ...

Here's an open source client that should also run on the Raspberry Pi: https://www.freerdp.com/

 

[HoneyBadger]

I haven't found one (yet). Have a look at this post, and you'll see that it isn't quite trivial (as in PnP) to do either. But if you do manage to find something. Do let me know There are a few youtuber videos where they do such stuff. But none with Truenas as a hypervisor, IIRC.
I'm kind of hoping that Intel will do interesting things with their Arc GPU's. But it's gonna be a few years for sure before it's reasonably feasible.

GPU mdevs are a fickle beast, as shown in that link or as experienced by anyone who's tried to make it work.

Unfortunately Intel hasn't enabled SR-IOV for their consumer ARC line, only the Flex 140/170 series of datacenter GPUs which are priced in the "generally unaffordable" range for the average home user. (Perhaps their next-gen iGPUs based on ARC will support it?)

NVIDIA GPUs are available second-hand for much less that do have this support, but NVIDIA's vGPU software requires per-guest client licenses, and a whole lot of irreversible changes to the underlying SCALE install, to the point where it's more "not supported at all" let alone "not supported out of the box."

We have 5 designers working off of the server constantly

Graphic designers and content creators likely wouldn't find it acceptable, but general office workers and power users should be well-served by something like RDP to a Windows VM, as it will handle GDI acceleration to do drawing client-side vs the screen-scraping of VNC or similar protocols. I suggest trying the FreeRDP client as suggested by @Patrick M. Hausen on your rPI.

 

[NugentS]

Intel NUC's or equivalent can be screwed to the back of monitors (VESA mounts) - so the monitor and PC essesntially become one unit - allbeit with a couple of extra cables.

I will third @Patrick M. Hausen & @HoneyBadger comments. Set up a windows 10/11 VM. Turn on Remote desktop and use an RDP client to take control of the VM. This is a junior version of what Windows Terminal Server can do. But as @Patrick M. Hausen says - the law of diminishing returns applies. Companies that use Terminal Server use lots of servers (aka Server Farms) to supply lots of clients. When I last played with a terminal server the client to server limit was about 20:1 for normal office machines. Creators will need something different as they need better hardware