Two words... Git-ops | Ramblings of someone on the internet

What is git-ops?

Git-ops is, simply put, a way to define infrastructure in a git repository. It's not so much a tool, but a way of working. I've been looking for a way to manage my homelab, and surrounding components, in a way that's reproducible, relatively easy to understand and manage, and possibly most importantly, not running on someone else's cloud...

If you want to know a bit more about what git-ops is, feel free to read more about it from the links below.

https://about.gitlab.com/topics/gitops/

https://www.atlassian.com/git/tutorials/gitops

So how to get there?

Well, the first step is the thing I've been meaning to do for ages... Finally open source my docker compose files!

https://git.kdb424.xyz/kdb424/compose

This has been loooooong overdue, and it's not fully open at this time. There's still a few private services that I run fully locally, and I'm still deciding if those will be opened, as well as one or two services that I still haven't been able to fully hide secrets (passwords, keys, ect), so those can't be opened yet. For the most part, that's everything that I run, and it's now fully open!

So how does that get us git-ops? (Forgejo runner)

Unfortunately, it doesn't yet get us git-ops. My next step was actually getting a separate (virtual) machine up and running so we can set up Forgejo runners! There are many ways to do this, but for now, I've set up a VM to confine it to, given it access to the raw docker socket (full machine control, effectively), and let it go. My current compose file is here.

I loosely followed this guide, but changed things for my setup. It's worth a read as it's slightly confusing to get set up in docker.

So now git-ops? (Renovate)

Not... exactly... We're finally going to set up something that helps us get closer though! We need to use the runner we just set up to run a project called Renovate. It's job, in my use case, is to update docker containers. Not by updating them, but by changing our compose files in a safe way. In my config I allow it to automatically deploy minor release changes, and it will open a pull request for major version changes. This allows for testing before it's put into production, and prevents a ton of breakage, while still informing you it needs changed, and offering up a potential solution to update.

So NOW git-ops?! (Dockhand)

Unfortunately, not yet, but we ARE on the last step, as of the time of writing. I've been using Dockhand for a few months now in the background, slowly deciding if it's good enough to use, or recommend. It has it's quirks, but I've mostly been set on using it for the time being. It's handled one off containers, (mostly) docker compose, and now, most importantly for git-ops, it supports remote docker compose files in git.

Thanks to, nearly, all of my stack being in git, Dockhand can pull compose files from git, reminder that Renovate also helps keep them up to date, and then deploy them via webhook every time it's merged into the main branch!

So NOOOOOW GIT-OPS?!?!

Not quite yet, but for the moment, it's a great start. I have a stack of software that keeps up to date with minor updates automatically, and a way to tell when major changes come up, and it even helps with a starting place to get upgrading! Many services can take major version updates without much of an issue, but then there's postgresql... I don't even want to get started complaining about upgrading that over time, let alone the additional changes needed going to 18... That's a rant for another day. The main piece that I'm missing for my own sanity, and part of what makes git-ops great is automated testing of proposed upgrades. I may see soon if I can set up a system to do that if I have the spare resources. I shouldn't have too much of an issue given how overkill that I went with my machine

If anyone has ideas on how to go forward from here, I'm open ears! Just keep your Nix to yourself, and I'm done with that chapter.