1 vCPU VM without unrestricted guest

[CAEguy]

My processor (W3550) supports virtualization but does not have unrestricted guest capability. According to the documentation it sounds like I should still be able to run VMs as long as they only have one vCPU but I still get the error. According to this post (https://redmine.ixsystems.com/issues/31338) it sounds like this is only true for FreeBSD VMs but the source provided doesn't talk about number of cores.

Can someone confirm under what conditions a VM can be run without unrestricted guest?

 

[dlavigne]

Were you able to figure this out?

 

[CAEguy]

I haven't done any additional work on this since I posted here, do you have any idea what the answer is?

 

[dlavigne]

What specific error do you get?

 

[CAEguy]

Here's the error I get:

Error: Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 161, in call_method
result = await self.middleware.call_method(self, message)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1109, in call_method
return await self._call(message['method'], serviceobj, methodobj, params, app=app, io_thread=False)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1049, in _call
return await methodobj(*args)
File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 664, in nf
return await f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py", line 1132, in start
await self._manager.start(vm)
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py", line 61, in start
list(done)[0].result()
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py", line 315, in run
raise CallError(f'VM {self.vm["name"]} failed to start: {output}')
middlewared.service_exception.CallError: [EFAULT] VM Ubuntu failed to start: ROM boot failed: unrestricted guest capability not available

This is while using 11.2U1, I have an identical system running U2 and can test that tonight to see if there's any change.

 

[CAEguy]

I tried in 11.2U2 and looks like it's the exact same error

Error: Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 165, in call_method
result = await self.middleware.call_method(self, message)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1141, in call_method
return await self._call(message['method'], serviceobj, methodobj, params, app=app, io_thread=False)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1081, in _call
return await methodobj(*args)
File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 664, in nf
return await f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py", line 1132, in start
await self._manager.start(vm)
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py", line 61, in start
list(done)[0].result()
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py", line 316, in run
raise CallError(f'VM {self.vm["name"]} failed to start: {output}')
middlewared.service_exception.CallError: [EFAULT] VM test failed to start: ROM boot failed: unrestricted guest capability not available

 

[CAEguy]

Hi I just wanted to see if you've had a chance to look into this at all?

 

[dlavigne]

The error indicates that the CPU isn't supported:

middlewared.service_exception.CallError: [EFAULT] VM test failed to start: ROM boot failed: unrestricted guest capability not available

Which docs have you read that indicate it should work?

 

[CAEguy]

The FreeBSD bhyve docs indicate it should work. https://www.freebsd.org/doc/handbook/virtualization-host-bhyve.html

"Hosting Linux® guests or FreeBSD guests with more than one vCPU requires VMX unrestricted mode support (UG)."

Based on this it seems most likely that one of the following is true:
1) The docs are correct but FreeNAS has added a UG check which isn't necessary for 1vCPU
2) The docs are correct but the CPU is otherwise unsupported and the error message is misleading
3) The FreeBSD docs are incorrect or outdated

But I don't know which of the three it is.

 

[dlavigne]

Might be worth asking on the freebsd-virtualization@freebsd.org mailing list and the #bhyve channel on freenode.net. If you get an answer, let us know.

 

[Ericloewe]

Unrestricted guest is necessary to boot the VM in real mode, which is necessary for traditional BIOS booting. You could boot from UEFI in protected mode, conceivably, but I'm not sure that's supported.

Without unrestricted guest, an x64 processor running in x64 mode cannot be placed into Virtual 8086 mode, necessary to run real mode software (namely the BIOS and bootcode) while maintaining memory protection for the rest of the system. Older VM software would emulate this to get around the limitation, but the bhyve devs presumably said "nah, we're not going bother, we'll just use UG, which everyone's supported for years". Which is totally reasonable, I believe. I would not particularly look forward to writing an x86 emulator whose state I then have to transfer to a real VM process that is running on my long mode process.