TrueScale HA

[TheUsD]

I've been digging around and maybe I haven't dug deep enough... I was looking to see who here has done a TrueNAS Scale HA.

I was gifted two Oracle FS1-2 controllers and 3, 24 2.5" disk cabinets. Unfortunately, I found out I could only use Oracle drives in said cabinets, regardless of what was controlling the cabinets. So instead, I decided to go a different route and purchase a Dell SCv200 cabinet that will accept any disk.

My next question is, if nobody has done this before, who here would be interested in seeing a TrueNAS Scale HA with two identical controllers sharing one storage cabinet?

If successful, I will be adding on either another SCv200 or SCv220 cabinet (or both!).

 

[morganL]

I've been digging around and maybe I haven't dug deep enough... I was looking to see who here has done a TrueNAS Scale HA.

I was gifted two Oracle FS1-2 controllers and 3, 24 2.5" disk cabinets. Unfortunately, I found out I could only use Oracle drives in said cabinets, regardless of what was controlling the cabinets. So instead, I decided to go a different route and purchase a Dell SCv200 cabinet that will accept any disk.

My next question is, if nobody has done this before, who here would be interested in seeing a TrueNAS Scale HA with two identical controllers sharing one storage cabinet?

If successful, I will be adding on either another SCv200 or SCv220 cabinet (or both!).

View attachment 60738

The HA version of TrueNAS SCALE is available on TrueNAS M-Series appliances.

It's quite complicated and very difficult to make it reliable and manageable without consistent hardware.

 

[TheUsD]

The HA version of TrueNAS SCALE is available on TrueNAS M-Series appliances.

It's quite complicated and very difficult to make it reliable and manageable without consistent hardware.

Just to confirm, you're saying the HA feature is not even available for users to try at their own risk, even if they have consistent equipment?

Is there any ability for someone to test this on their own?

 

[Ericloewe]

Just to confirm, you're saying the HA feature is not even available for users to try at their own risk, even if they have consistent equipment?

It is not. It's part of the business model, though that in no way invalidates what @morganL said above.

 

[morganL]

Just to confirm, you're saying the HA feature is not even available for users to try at their own risk, even if they have consistent equipment?

Is there any ability for someone to test this on their own?

For users that want to use their own hardware, we recommend clustering. It's less dependent on hardware. We don't offer HA as a software-only capability since we'd then be on the hook for supporting it..

 

[ChrisRJ]

Having lived (and suffered) with HA in the context of transaction processing, I think that iXsystems' approach makes sense. Starting with relatively simple aspects, like split-brain, this topic has so many facets where things can go wrong. We see a very simple version of this with the question of gamer components vs. enterprise ones almost every day. Imagine if suddenly we had to talk about Ethernet latency of switches (or routers if the switch is L2-only and we have VLANs).

Most companies I know actually try to avoid active-active HA solutions like the plague. Because those introduce a lot of additional complexity that, if done without sufficient experience, the availability actually suffers. It is a particularly challenging side of distributed systems, which are fascinating, but also one of the more complex topics in computer science.

For those who are interested in the underlying questions, I can recommend the book "Designing Data-Intensive Application" from Martin Kleppmann. It is not about storage per se, but covers things from the application level (rather obvious from the title). The relevance for storage is that the latter can of course be seen as an application. So things like replication, load distribution, conflict resolution, etc. apply just as well.

 

[TheUsD]

Most companies I know actually try to avoid active-active HA solutions like the plague.

I'm not sure what you mean by this as NetApp, Dell, HP and Oracle are some of the biggest storage arrays out there as well as Zadara (not as big) that do active-active HA all day long. I'm curious to know where you pulled this information from.

Though I never specified if I was doing A-A or P-A controllers. Just to specify, I would prefer A-P. The redundancy gains would be performing updates to controllers, firmware, and redundancy...

For users that want to use their own hardware, we recommend clustering.

Thanks for the direction. Can you point me to a good article and or documentation to digest?

 

[morganL]

I'm not sure what you mean by this as NetApp, Dell, HP and Oracle are some of the biggest storage arrays out there as well as Zadara (not as big) that do active-active HA all day long. I'm curious to know where you pulled this information from.

Though I never specified if I was doing A-A or P-A controllers. Just to specify, I would prefer A-P. The redundancy gains would be performing updates to controllers, firmware, and redundancy...

Thanks for the direction. Can you point me to a good article and or documentation to digest?

Clustering is managed through TrueCommand.

Clusters Screen

Option descriptions in the TrueCommand Clusters menu.

www.truenas.com

 

[TheUsD]

Clustering is managed through TrueCommand.

Clusters Screen

Option descriptions in the TrueCommand Clusters menu.

www.truenas.com

Got it.

Not super impressed with TrueCommand. It randomly, but consistently, keeps dropping connection to other TrueNAS core devices. Nothing rebooted, nothing changes in the configs, nothing changes network wise. I always have to end up deleting and rejoining them. Using same API keys and all.

 

[morganL]

Got it.

Not super impressed with TrueCommand. It randomly, but consistently, keeps dropping connection to other TrueNAS core devices. Nothing rebooted, nothing changes in the configs, nothing changes network wise. I always have to end up deleting and rejoining them. Using same API keys and all.

Please document this in a separate thread with version (both CORE and TrueCommand). It's the first I've heard of this.
If no-one can see the issue we should report--a-bug.

 

[ChrisRJ]

I'm not sure what you mean by this as NetApp, Dell, HP and Oracle are some of the biggest storage arrays out there as well as Zadara (not as big) that do active-active HA all day long. I'm curious to know where you pulled this information from.

It was not specifically about storage, but enterprise software in general. Sorry for the confusion.

To illustrate where I come from with this, here are a few more details. Overall I was referring to custom/bespoke solutions that are difficult to set up and operate. In addition there is often a strong connection to business aspects. This means that replication strategies, conflict resolution , etc. typically need individual approaches. In contrast, with files or databases it is usually possible to approach things in a more general way.

I hope that explains things.

 

[TheUsD]

Please document this in a separate thread with version (both CORE and TrueCommand). It's the first I've heard of this.
If no-one can see the issue we should report--a-bug.

Will do. Could you please remind me on how to report bugs?

It was not specifically about storage, but enterprise software in general. Sorry for the confusion.

Still unsure what you are referring to. I know plenty of HA software out there, such as Fortinet's various software's running in HA. FortiManager, FortiAnalyzer, NAC, Authenticator. Palo Alto's Panorama, the list literally goes on. Not attacking you, just not sure what you are referencing. Maybe you could help me understand by giving a direct example?

 

[ChrisRJ]

Still unsure what you are referring to. I know plenty of HA software out there, such as Fortinet's various software's running in HA. FortiManager, FortiAnalyzer, NAC, Authenticator. Palo Alto's Panorama, the list literally goes on. Not attacking you, just not sure what you are referencing. Maybe you could help me understand by giving a direct example?

Think of custom development that goes into the direction of CRM, ERP, supply chain, midlleware, etc.