So we're over-complicating it again...

For those of you that have talked to me in the last several years, I've linked everyone to my original post about how I host things, and in a really vague sense, it's still the general flow. There's a VPS that hosts a web server acting as a reverse proxy over an ephemeral network into docker container(s). That said, nearly the entire tech stack has changed underneath itself.

In this post, we'll cover a bit about how it works, the common questions I get asked on why this specific tech stack, over all how it works, and what I hope to do with it into the future.

So what software runs the show these days, and why?

I'll cut to the chase, since I think this will be a long one. I'll start from the moment that https://blog.kdb424.xyz gets requested, and go from there.

Caddy

The moment that you ask for my blog, once DNS sends you to the server, Caddy is the first software that you interact with. I use Caddy as, almost exclusively, a reverse proxy, though technically it's also acting as a load balancer.

My config for Caddy at the time of this post is here

The specific section that is relevant is this.

blog.kdb424.xyz {
	reverse_proxy {
		to :8197 fry:8197 igner:8197
		lb_policy round_robin
		lb_retries 2

		health_uri /
		health_interval 5s
		health_timeout 2s
		health_status 2xx
	}
}

There are 3 machines that host this blog, all on port 8197. Depending on the health of any given one, you'll get one of them. They should all contain the same thing, and it's all static content, so it really doesn't matter which.

Caddy is incredibly simple to set up compared to Nginx/Apache, lighter on resources in my experience, and handles all of my HTTPS certs with zero effort. I don't regret moving my reverse proxy to this from Nginx, and wish I had moved sooner. If you don't like networking especially well, Caddy will be your best friend, while also still giving you the tools to do advanced things if you need the extra power.

Tailscale

Tailscale is an ephemeral network. It acts as a private network, just like you have at home. Think getting to your router at 192.168.1.1 or 10.0.0.1, and gives you a way to talk between machines that are not physically in the same network. This somewhat sounds like a VPN if you've ever used one of those, but the difference is everything is P2P. Once machines fine each other, you don't need to route through any server. Other than locating each other, which is effectively done with some magic, there's no other machines in the link, and it's fully encrypted between, so you truly can use it like a home network!

In my original post, I was using Zerotier, and was honestly quite happy with it. I didn't have much need for anything advanced, and it did the job great! I moved simply because Alpine Linux didn't package it anymore due to licensing reasons, and while I didn't dig into the reasons why, it may be worth looking into if you're looking to make the choice. Zerotier is currently treating me nicely, offers many more features that I'm not using, has great DNS that "just works TM" and gives me the ephemeral network that I enjoy.

If you don't like Zerotier/Tailscale because you want to self host, Headscale is an option, as is NetBird, though I have not tried either of these. I've had nearly zero downtime with Tailscale, and I am more than happy to move onto the next thing if Tailscale becomes something that's infeasible for any reason. The magic of good DNS means that I can change the underlying network without a hitch in my lab. I did find some success hosting a Zerotier control plane for a while, but it's been years now, and I don't know what it's like these days.

Docker container

The blog ultimately lives in a docker container. Thankfully, almost all of my docker compose files are out in the open here, so you can look at them. Because it's small enough, I'll post the entire contents.

---
services:
    blog-test:
        image: git.kdb424.xyz/kdb424/zola-blog:latest
        container_name: blog
        ports:
            - 8197:80
        restart: unless-stopped
        healthcheck:
          test: /usr/bin/curl --fail http://localhost || exit 1
          interval: 60s
          retries: 5
          start_period: 20s
          timeout: 10s

The main things to note here are that the port is 8197 as you saw in the reverse proxy, Caddy, above this section. The image is being pulled from servers run in the lab as well, hosted on Forgejo. If you are interested in that, it's out of scope for this post, but that docker compose file is here.

Docker... again?

I'm sure you've noticed that I'm linking docker compose files in a git repo, hosted on the same Forgejo instance mentioned above. I've currently got a personal and public repo. The personal mostly houses things in testing, and things with secrets that I couldn't clean up, yet.

The public repo is currently here, and all links in this blog post point to the commit at time of writing. All of these are hosted in git for a few reasons. Version control is great, Renovate-bot makes updating a lot easier, and I have a great tool called Dockhand. This gives me the ability to deploy docker containers that track a Git repository. If I, or renovate-bot, make changes to the docker-compose files, they automatically get deployed! I've personally triggered this through webhooks mostly on push events to the repo, but my blog needs something a little different.

Zola

This is the meat and potatoes of how this blog actually exists. It's currently hosted here, though you will notice that this post won't be found in that commit. That's because when this post gets committed, and pushed to the server, Woodpecker CI takes over! In this repository, the config tells the Woodpecker to build a web server that contains the blog, using the Dockerfile in the same repository. It uses buildx to build for both ARM and X86_64 machines, and deploys it to the repository. Dockhand, the section above, sees that this package is updated, and forces all of the containers that run the blog to pull the new image, and restart. The restart lasts for no more than a second between the 3 of them, thanks to the load balancer in Caddy, so 99.9% uptime even when updating.

And that's how these pages get to you today!

I'm sure that there's a ton more that I could go into, and I probably will if people continue to ask questions, which is encouraged. I could ramble about how I couldn't figure out Forgejo actions, and set up Woodpecker. Docker files for that are here btw This ended up running in it's own VM (technically it shares a space with other runners, but isolated from anything else), as I fed it the docker socket itself to simplify docker in docker. It runs it's own docker containers, all from within docker. I could talk about forgejo runner, though I still haven't done much with it outside of get renovate-bot to help keep stuff up to date. I'm sure I'll ramble more later, but for now, that's how everything gets to you! Hopefully one day, I'll do more testing before deploy, but monitoring is something I am getting ready to talk about soon. For now, have fun homelabbing!