How to add missing Apps - the given way vs the achievable way

[marshalleq]

Hi all, I'm hoping someone can shed some light on my confusion.

For any app that is not default app within the apps section, or not available though TrueCharts add on there is Launch Docker option. I've tried this, but since I can't edit the docker after creating one, it makes it unusable. (When I say edit, you can click edit, but the manual variables do not get presented).

So someone tells me about helm charts, which I've started looking into, I've also looked into just using docker at command line - something I used to do exclusively, but then I hear that because docker isn't really what TrueNAS uses, it's not really going to work (and it doesn't it seems). I'm not 100% sure if that was me or something else.

So would someone mind please explaining what are the reasonable options to add apps that are not in the charts? Do I have to create my own repository or charts? Will docker from the command line work 100% or is there some limitation?

Many thanks.

 

[morganL]

Is you app a single docker container or more complex?

It's best to start with the developer Notes and then ask questions that are not clear after reading those.

/scale/

 

[marshalleq]

Thanks very much for the link.

So it seems that it's not supported to use docker from CLI unless you turn off Kubernetes if I'm reading correctly. I can't use the Docker GUI because of the lack of functioning edit capability. And the docker UI doesn't have a way to import / export without using Kubernetes anyway.

So that means in the short and long term, Kubernetes is where it's at.

For Kubernetes it seems I have to create a catalog which in turn points to the apps in the form of charts.

And obviously I can't modify or add to the official catalog. The community one called TrueCharts does not support the things I want to do (because I need some proprietary stuff and stuff they deem as security issue) so this then means I must create my own. It is not surprising to me at all that this took a few days to figure out!

The golden nugget for me on the page is the example under deploying kibernetes workloads:
midclt call -job chart.release.create '{"catalog": "OFFICIAL", "train": "test", "item": "ix-chart", "values": {"image": {"repository": "plexinc/pms-docker"}, "portForwardingList": [{"containerPort": 32400, "nodePort": 32400}], "volumes": [{"datasetName": "transcode", "mountPath": "/transcode"}, {"datasetName": "config", "mountPath": "/config"}, {"datasetName": "data", "mountPath": "/data"}], "workloadType": "Deployment", "gpuConfiguration": {"nvidia.com/gpu": 1}}, "version": "2010.0.1", "release_name": "plex"}'

Though confusingly it says under this that CLI is not officially supported - so that doesn't make much sense.

And at the very bottom it says UI support for custom catalogues is planned for the next release - so I guess this is why CLI is not supported, because there's a planned GUI coming to replace it. I personally think this information needs to be surfaced a bit more plainly (assuming I've gotten it right).

I.e. to come at it from a customer / user perspective, I know I need an app, how do I get one and what are my options. Having this in developer documentation is all good for an alpha, but something we probably need to mark to improve upon?

Thanks.

 

[morganL]

Thanks very much for the link.

So it seems that it's not supported to use docker from CLI unless you turn off Kubernetes if I'm reading correctly. I can't use the Docker GUI because of the lack of functioning edit capability. And the docker UI doesn't have a way to import / export without using Kubernetes anyway.

So that means in the short and long term, Kubernetes is where it's at.

For Kubernetes it seems I have to create a catalog which in turn points to the apps in the form of charts.

And obviously I can't modify or add to the official catalog. The community one called TrueCharts does not support the things I want to do (because I need some proprietary stuff and stuff they deem as security issue) so this then means I must create my own. It is not surprising to me at all that this took a few days to figure out!

The golden nugget for me on the page is the example under deploying kibernetes workloads:
midclt call -job chart.release.create '{"catalog": "OFFICIAL", "train": "test", "item": "ix-chart", "values": {"image": {"repository": "plexinc/pms-docker"}, "portForwardingList": [{"containerPort": 32400, "nodePort": 32400}], "volumes": [{"datasetName": "transcode", "mountPath": "/transcode"}, {"datasetName": "config", "mountPath": "/config"}, {"datasetName": "data", "mountPath": "/data"}], "workloadType": "Deployment", "gpuConfiguration": {"nvidia.com/gpu": 1}}, "version": "2010.0.1", "release_name": "plex"}'

Though confusingly it says under this that CLI is not officially supported - so that doesn't make much sense.

And at the very bottom it says UI support for custom catalogues is planned for the next release - so I guess this is why CLI is not supported, because there's a planned GUI coming to replace it. I personally think this information needs to be surfaced a bit more plainly (assuming I've gotten it right).

I.e. to come at it from a customer / user perspective, I know I need an app, how do I get one and what are my options. Having this in developer documentation is all good for an alpha, but something we probably need to mark to improve upon?

Thanks.

Yes, TrueNAS SCALE is still in ALPHA state and suitable fro developers and testers. Better documentation will be available for BETA in 21.06.. that is when we start to support users (not general customers).

I assume your apps are not a single Docker container... because that is supported. For more complex apps, Helm Charts are the preferred model. Will they work for your applications?

 

[marshalleq]

Yeah - there is the problem that docker is not currently editable - that would at least mean I could get started and then that would suffice for now.

It's funny I've heard one person say the helm charts are for the more basic things only - not really what I thought but thanks you have clarified it.

Almost all of what I need is simple, but with quite a few docker variables with exception of Taiga which is a multi-component situation (it's like simpler version of Jira).

Primarily though, what I'm trying to get going now is Tdarr as this test machine is an encoding machine. I've done this through the Docker 'one time create and no edit' scenario that is current and seem to be having issues with the network mount. I can't fault find it easily because - no edit. So I go round in circles asking what is the best method. I still say it's Kubernetes.

I also need databases of course, which are unavailable in anything pre made.

And all of this really is a standard setup so that I can contribute back.

It's already very stable, it's only really code change decisions I need to worry about - official positions aside.

Great product so far!

 

[morganL]

I think you mean "the docker helm chart" is not currently editable.... if not, can you clarify?

Did you find the charts "instructions" here? https://github.com/truenas/charts

 

[marshalleq]

What I mean is when using the Launch Docker option you can create the docker (or whatever TrueNAS is actually creating behind the scenes (I heard it wasn't docker)) you can add variables , save it and run it. But, if you want to edit the variables, when you edit the docker all of the set and active variables do not show up to be edited. It's as if you were clicking Launch Docker again and starting from scratch - except it is actually working in the background. There is an open ticket for it here.

So the problem with that is when something doesn't work - you can't actually go and change anything to attempt to resolve it via the GUI to fix it. The only thing you can do is delete it and create it again and guess what might have gone wrong. Which is the situation I'm in with Tdarr.

I'm guessing most people on here have available what they need or they don't need docker variables - because this issue has been consistent across every build I've installed and hardly anyone seems to comment on it.

So I've tried using docker at the command line, but since we are apparently not actually using docker and I'd need to turn off Kubernetes to do it that way, well I assume you can see the challenge I'm having. I need to use Kubernetes in the current state of this Alpha. I need to learn charts.

Thanks for the link, I will read and begin figuring out how to solve this seemingly complex conundrum.

 

[ornias]

because I need some proprietary stuff

Thats not what we told you.

So I shall explain again:
You can submit a PR for your paid-for software, if you submit a PR that (at the very least) includes the following:
- The (non-free) App you want to have included
- A new Train "non-free" for commercial (payed for) software packages
- Update the documentation
- Update the workflows to include the new train
- Describe (in the PR description) how this Appis supposed to be maintained (for example: is there a trusted maintainer or company that is going to maintain this, do we get free keys for project maintainers to maintain the app, or is there another solution?)


We might, on a case-by-case basis, close App-Request github issues, because they are not the focus of the project at the time or do not comply to our current guidelines. (as we cannot have github issues for every Application on the internet)

However: PR's will always be evaluated, reviewed and will always get a fair chance of being included, regardless of your issue.

I think there might also still be some confusion about what an App request means:
It does not mean we are going to make your App for you, it's just a public requests if there's anyone out there to make it for you ;-)

and stuff they deem as security issue

"Deeming" makes it sound like the software is fine, which it isn't.
I can already tell you there are multiple high and critical risk CVE's in the software you requested, because none of it's libraries or its code have been updated since 2019.
While you can debate me denying this software package, you can't debate that the software itself is, in fact, unmaintained and unsecure.

(I even commented that I liked the software itself in theory. But it's not about me, it's about objective criteria ;-) )

Primarily though, what I'm trying to get going now is Tdarr as this test machine is an encoding machine. I've done this through the Docker 'one time create and no edit' scenario that is current and seem to be having issues with the network mount. I can't fault find it easily because - no edit. So I go round in circles asking what is the best method. I still say it's Kubernetes.

To be clear for everyone reading:
TrueCharts did not deny Tdarr, we actaully look forward to including tadarr. It's not one of the two Apps mentioned ealier.

It's also a hard App to implement due to having two (differently scaled) containers, but it's very much on the to-do list of some of our supporters :)

 

[marshalleq]

@ornias I do believe I understand what you're saying, but your reply above actually further justifies my position really. And for clarity that position is that you guys are doing a great job and have some great apps, but not all apps are going to be able to go through TrueCharts right now because e.g. of the position on security. Regardless of this security position, I know that running (for example) Roon inside a container on my home network, not exposed to the internet, using internal only VLANs through my firewall, with IPS / IDS active is pretty well safe. I take that as my responsibility to maintain. I don't need someone second guessing my security of my home network, but I respect your position and decision to support only apps with a minimum security requirement.

So, unless I'm misunderstanding something there (always possible) I still need to build my own.

I apologise as its probably not helping the confusion that I used the Tdarr example above but am talking more generally in other places.

Being a little more general though - I think it's fair to say that people will want to install their own stuff, and that is what I'm trying to do. It's not intended to be negative toward the great work TrueCharts has done and undoubtedly will continue to do. I will as time allows grow my understanding of all this and eventually get to the point that I can confidently become a maintainer of sorts myself, and support TrueCharts but that time has not yet arrived - I'm too new to fully understand it.

All I need right now is a pathway to get the apps that I use to function in TrueNAS.

Regarding the commentary around 'proprietary stuff' what you have listed sounds like a lot of work from the perspective of I just want my apps to work - and given there are already available docker apps, but TrueNAS is not docker it's a higher bar than I was originally expecting. Particularly: "How this Appis supposed to be maintained (for example: is there a trusted maintainer or company that is going to maintain this, do we get free keys for project maintainers to maintain the app, or is there another solution?)". I get this is important and I need to learn more about how TrueNAS works before I can answer it.

Perhaps a good place for me to start supporting TrueCharts is just with Tdarr. It's already on the Truecharts want list, I know people in the project including the developer since before it was even created, have helped write the wikis and think it's a bit simpler than it probably looks. But of course there may be something about TrueNAS Scale I don't understand when I say that.

So how about it, shall we work on that one? I will say in advance though, I'm motivated to do this, but I'm working three jobs essentially so my effort will be a bit sporadic.

 

[ornias]

So, unless I'm misunderstanding something there (always possible) I still need to build my own.

Well, what I wanted to showcase with the list of steps required for TrueCharts to get rid of the non-free apps guideline is that we are willing to work with you to get your Apps included even when it comes to modifying the guidelines. Same goes for security, you can very well make a good proposal as a PR.

Being a little more general though - I think it's fair to say that people will want to install their own stuff, and that is what I'm trying to do. It's not intended to be negative toward the great work TrueCharts has done and undoubtedly will continue to do. I will as time allows grow my understanding of all this and eventually get to the point that I can confidently become a maintainer of sorts myself, and support TrueCharts but that time has not yet arrived - I'm too new to fully understand it.

To support that self-help effort, we've just today launched a github template to deploy your own catalog with basic CI.

Perhaps a good place for me to start supporting TrueCharts is just with Tdarr

You picked the hardest app from the whole request list. There is a reason @StavrosMadK didn't get to do it already, is because we need a clear design draft for the helm chart and need to look at the best way to manage the multiple containers.

While it's possible to get an app running 1 master and 1 slave running in a few minutes, it gets harder if you want to design the App around supporting multiple slaves in such a way that you're not going to annoy users with a huge in the future.

Simply put:
We've had bad experiences with people biting off too much they can chew for their first App. We don't prohibit it ofcoarse, but know what you're getting yourself into and don't expect complete reworked code from us when your initial design isn't passing review ;-)

 

[ornias]

@marshalleq

I've just added the technical support for `non-free` and `deprecated` (usable for insecure Apps) trains to TrueCharts, though the docs/guidelines still need to be adapted.