Corosync Qdevice in a jail?

[rvassar]

Has anyone ever looked at running a Corosync QDevice in a jail on TrueNAS Core? I'm currently parking a cluster vote on a Raspberry Pi, but it seems like I should park it on the other always-on device the cluster is dependent on, the NAS. I know I could spin up a VM, but that's a whole extra kernel worth of memory, and I'm not ready to jump to Scale where it could easily get parked in a container.

 

[sretalla]

pkg install corosync3 ...?

Never tried that myself, but seems it may be that simple.

 

[Bee Town]

Went through the process of setting this up today and eventually got it working with Proxmox

pkg install corosync3 to my knowledge doesn't include corosync-qnetd maybe-relevant-bug

I'll write up something quick below in case I'm not the only one waiting for an answer to this post but as a warning my approach was hacky I haven't retraced my steps to make sure I covered everything.

---

FreeBSD jail version: 13.x

Compilation steps

Source code: v3.0.3 release of https://github.com/corosync/corosync-qdevice wget https://github.com/corosync/corosync-qdevice/archive/refs/tags/v3.0.3.tar.gz

I installed with pkg:

• automake
• libtool
• pkgconf
• corosync3 (don't know if it's actually needed)
• bash (needed for corosync-qnetd-certutil)

in retrospect I should have used gmake instead of make since there were some incompatibilities with [BSD] make which caused sed to hang. I ended up having to run make inside qdevices/ and manually create corosync-qnetd-certutil from its template -- probably just due to me forgetting to use GNUMake.

Other than that a standard

Code:

./autogen.sh
./configure
make # probably should use gmake

Installing

I think I had to run corosync-qnetd-certutil -i once to install the prereq certs.

my /etc/rc.local to start with the jail:

Code:

#!/bin/sh
/root/corosync-qdevice/corosync-qdevice-3.0.3/qdevices/corosync-qnetd -f &

(I don't know if this is the right way to run qnetd )
Restart the jail and confirm corosync is running ps auxd

---

Getting proxmox to work
aside:this is just my use case for a qdevice, writing this down for posterity

Proxmox wants corosync-qnetd-certutil in the path when setting up so throw it in /usr/local/bin (or wherever)

Proxmox also wants to ssh onto the qdevice node to configure it so
from the jail:

Code:

# generate host key
ssh-keygen -A
# Set a root password
passwd
# Start sshd in the foreground temporarily, allow root access temporarily
/usr/sbin/sshd -D -o PermitRootLogin=yes

Then from proxmox:

Code:

pvecm qdevice setup you_ip_address_goes_here
pvecm status

remember to set up some kind of static IP for your qdevice because from that point on it will be used by proxmox

Hopefully you'll end up with a result like this:

Code:

Membership information
----------------------
    Nodeid      Votes    Qdevice Name
0x00000001          1    A,V,NMW 10.11.0.4 (local)
0x00000002          1    A,V,NMW 10.11.0.5
0x00000000          1            Qdevice