Building containers programmatically, self-hosted

Building containers programmatically, self-hosted

1. 1. jitl

      edited 12 hours ago

      | link

      It sounds like you are describing a CI system. There are lots of CI systems that support building Docker images, and any good one you can trigger with an HTTP request and include extra info like a reference to your customer’s build description. Like, bog standard ancient things like Jenkins (yuck) would work just fine, or some more modern CI system like Buildkite (just use the api to create build jobs). Pretty much every public cloud has a build system too. CI systems generally have great UI for observability.

      If you want Very Cloud Native, there’s https://github.com/GoogleContainerTools/kaniko

   2. ocramz

      17 hours ago

      | link

      As with most cloud-native stuff : am I overthinking this? Is everybody else overthinking it? What year are we in?

      1. aae

         14 hours ago

         | link

         probably

   3. https://nixery.dev/ provides a way to generate container images from simple HTTP requests. Don’t know about self-hoisted, but maybe worth a look?

   4. option

      edited 11 hours ago

      | link

      It depends on your requirements.

      If you want a cloud-based queue anyone on earth can hit and a local daemon can subscribe to:

      • Buildkite: the daemon is already written, the cloud API is robust. There is a SaaS GUI. You self host the daemon only.
      • NATS: there is a SaaS you can use, OR you can self host the NATS server. But you need to write glue. No builder daemon.

      If you want something that you can run locally and POTENTIALLY deploy to self hosted infra:

      • Forgejo with the Actions feature: https://forgejo.org/docs/latest/admin/actions/

      The trickiest bit is…what are the fault tolerance requirements of your system? Do you want retries? Do you want a dead letter queue? Do you want pup sub or request response? People pay money for all that stuff.

      I’m practical terms, I’d look at NATS as a part of your final system design. Look at their GitHub to see the official client libraries. Read their docs to understand all their features. Join their Slack (they are friendly).

      If you literally don’t want to focus on the cloud queue to local daemon stuff right now, and want to build other parts of your system, and just want a reliable cloud API that can trigger arbitrary commands locally, then sign up for buildkite and run their daemon locally. Just keep in mind that eventually you will need to build it all yourself.

      If you want to build your own SaaS queue with your own protocol but want a sophisticated queue/job management thing, look at paying for a Faktory license and deploying that.

      Hope that helps.