What is nix, and why should I care?

This exact question is why I avoid nix for so long. Specifically the "what is nix?" half. Nix is not one thing, but a collection of many things when most people say "nix". I wanted to clarify what things are to the best of my knowledge and give others what I had trouble finding in one convenient location when I was not knowing anything about nix in hopes that you understand better what things are, why I'm so pedantic, and possibly enable you to give things a shot with at least a direction to go that I had such trouble finding. With that out of the way, what the heck is it?!

Nix

Nix seems to be a catch all term for anything related to core tooling, the language, ect. If you are familiar at all with ZFS, it's likewise "a filesystem" but it's also an ecosystem of tooling that you gain massive benefit from using as opposed to using traditional tooling in it's place. Nix is much like that in regards to it being a set of tools that integrate tightly with each other.

The other definition that can be applied to nix is that it's a package manager. It's a declaritive system using Nix Expression Language (covered below). I encourage you to read about the nix package manager even as there is already a great breakdown of why it's great there, and I don't need to rehash that as I can't do a better job.

Nix expression language

Nix expression language is written in a functional style. It's used to describe anything that is in the nix ecosystem. All tools are written in this one style, which allows you to have less languages to learn to create anything that you want to integrate into the system. Because of it's pure functional nature, this means that results are in most cases able to be reproduced bit for bit identical. This reduces the classic "It works on my machine", and hard to debug errors down to almost none. If this sounds similar to what Docker or other OCI containers solve to a point, you would be correct. This extends to much much more than containers though.Nix expression language great for writing build systems, making reproducable containers, creating dev environments, building operating systems that are bit for bit identical on real hardware, the list goes on.

Nixpkgs

This one is simple. It's a list of packages avaliable to the nix package manager. There's several repositories avaliable, it supports something similar to Gentoo's overlays or the AUR from Arch. You can read more about it here.

Home-Manager

I've talked about this on more than one occasion, and never fully defined what it is. home-manager is tooling that allows you to define how user's files, applications, configs, ect are all set up. As opposed to installing packages one by one like you can use nix-env to do, it brings it all together similar to how pmm does for many systems, but gives you all of the power of Nix Expression Language to be able to set up logic instead of doing manual work. It can invoke basically "anything the nix ecosystem can do", but that's the simple answer.

NixOS

NixOS is what happens if you take home manager, and give all of that power to your operating system. Have a fully redeployable operating system on real hardware that can describe not only what's installed, but how it's configured from networking, to system services, to what docker containers will be running. All within Nix Expression Language. It has integrations with home-manager to allow everything to be fully integrated while still offering you portable users. No more installing a distro and then configuring it, worrying about config file changes with updates lingering and breaking over time. It's all written in Nix, and will all work every time, exactly the same way. Even if you try to manually edit config files, they are immutable and regenerated so nothing can really damage the system from the outside as you can simply regenerate everything. Roll back to previous definitions, all without filesystem snapshotting. It's fully extensible and the only limits are what time you are willing to put into it.

Nix-Darwin

While not as integrated as NixOS, this allows you to manage many aspects of MacOS in a similar way. Nix-Darwin has integrations with home-manager so you can get most of the power without leaving all of your Mac specific apps behind.

Nix Flakes

If you ever search Nix, you'll see this being talked about constantly. For the moment, I'm avoiding it until it gets marked as stable, but it looks promising. As I am not using this experimental feature, I'll suggest you go read about it here.

Ok, but what can I do with all of that power?

I can't even begin to pretend like I can understand the full power of what is possible, but I'll list a few things that you may find interesting. These are things that I may play with and I'll document anything that I do and share my thoughts.

• Nix-Direnv: Portable dev environments
• Comma: Run ephemeral commands
• Better PMM:: Manage packages in a central, portable, place.
• Manage docker: Docker-compose, but not learning a new language
• Nixified terraform: Why learn terraform when you already know nix?
• CI/CD: All devs need CI/CD. Why not bring that in too?

Further reading

While all of this sounds insanely appealing, there is a learning curve. Nix doesn't have to be a massive time/effort investment, but it can be if you want to get the most out of it. I believe if I stopped where I am, I would still have massive gains from where I was learning nothing more. If you want to hear some thoughts from someone who's used Nix for many years, and their thoughts on it, I'll link to Hilssner's dotfiles. For those unaware of who that is, they were the creator of Doom Emacs and their content is always a joy and a laugh to read. I highly recommend reading their FAQ as it seems to cover if you should learn nix in a way that's both funny, and brutally honest, yet remaining a nix user. I'll also link their decent into madness as it's yet another good read for anyone new. While they have quite the indepth take on it, they make a lot of good points to think about if you want to take nix seriously, or just use it as a neat toy as I have been. Until the next post, I hope you enjoyed, and were entertained by hilssner even half as much as I am.

Portable dev environments

In my journey to uncruft my system, I started looking at even uncrufting my user's footprint as well. I found myself bringing in a ton of things for misc dev work. One project I may use Nim, another Python, another I need Zola for this blog, and it started adding up. I knew there had to be a better way. In comes nix-direnv. If you have done development, you may already be familiar with direnv which is great for keeping your system clean in general. I won't repeat everything when there are good links to things, but direnv is built to keep your env vars that are project specific out of your local system, and nix-direnv extends that now to packages! Instead of rambling, I'll show you a simple demo that I actually use on this blog.

You'll need two files for each project.

.envrc

use nix

shell.nix

    { pkgs ? import <nixpkgs> {}}:

    pkgs.mkShell {
      nativeBuildInputs = with pkgs; [
        gnumake
        zola
      ];
    }

You'll obviously want to install direnv and nix-direnv in whatever way you would normally. I simply included them in my home-manager file. Deperinding on how you have your shell installed, you may need to add a hook for your shell to automatically load the environment when you change into the directory. Since I don't allow nix to manage my ZSH configuration, I simply added it to my .zshrc, though it may be more readable to most people with the shown example.

# .zshrc
eval "$(direnv hook zsh)"

If you have a different shell, follow the docs for your specific shell. That should be it in terms of global setup. .envrc files won't be loaded by default for security, so you can run direnv allow . in the directory once and it will be allowed to load from that point on when you cd into the directory, and unload when you leave.

After thoughts

Not only does this allow you to keep your system cleaner by keeping env vars and packages out of the system and user's packages, it allows you to keep that portable for anyone that has a nix system with direnv and nix-direnv so there's no need to guess what packages are missing for nix as they can auto populate. If you want to adventure further into it, you can even do things like fetching remote envs, version locking to keep all users on the same dev environment, and much more, but that's outside of the scope of this. This is just meant to give you something to start with, or it may already do everything you could possibly need. Hope that helps someone understand what it does so you don't have to dig up everything like I had to.

Nix is... wild

So I decided that I was going to try nix as my attempt to remove my state from the system, and didn't realize the size of the rabbit hole that I had fallen into. I thought I was getting access to a package manager that I'd bend into shape like I had with Gentoo in the past with their world files. Turns out that Nix does that really well. I figured that I would enjoy parts of it as I've moved closer and closer to it in the years without realizing, but I didn't know what I would uncover. Before I get sidetracked, let's get this out of the way.

What even is nix?

Nix isn't an operating system. It's technically the name of a package manager, and NixOS uses that package manager. Sounds simple enough, but Nix is more of a language or an idea than either of those things. I won't pretent to have any sort of massive knowledge on nix, but there is so much out there that I wanted to make it known that you will likely be lost if you search for anything Nix related. I'll try to document what steps I'm taking and update with how I'm learning my way through it to maybe help someone out.

So what's the status?

I've started transitioning my personal state into Nix to some degree. If you want to see all of the files as they are when this was written, or as they are live, you can follow along as you please. I've started by porting out into a few modules. There's nothing special about not putting everything in one file per se, but it does allow me to not repeat things and have a more central place to edit things I want on all machines. All of this is powered by home-manager which is a nix community tool that's meant to be much like the core nix, but let you seperate users from the core system. It can also be integrated with NixOS itself, but I'll try to keep it simple for now.

what's it look like

Home-manager files are closely related to nix, but have some slight deviations, though they use nix under the hood. Here's an example from farnsworth (mac mini). I'll trim a bit to make it fit better. Mostly self comments.

    { config, pkgs, lib, ... }:

    {
      home.username = "kdb424";
      home.homeDirectory = "/Users/kdb424";

      # Set current stable
      home.stateVersion = "22.05";

      # Checks that home manager is in sync with release
      home.enableNixpkgsReleaseCheck = true;

      # Let Home Manager install and manage itself.
      programs.home-manager.enable = true;

      # pulls in other nix files
      imports = [
        ../modules/common.nix
        ../modules/mac.nix
        ../modules/commonGUI.nix
        ../modules/devel.nix

      ];

There's not really much in here. I've tried to leave the main home.nix manage only minimal things like what types of things it wants to pull in from other nix files and keep it clean. This is similar to how I managed pmm files, or using user defined sets in Gentoo/Alpine. It just lets you group things by topic and turn them all on/off in sets by chosing imports. These files are just collections of packages that look like this as a base.

    { config, lib, pkgs, ... }:

    {
      home.packages = with pkgs; [
          firefox
          nim
          ect
      ];
    }

I try to keep things organized like common tools that may be on all machines, or machines of a type, and have just them at this point to simply replace a world file with only one non standard thing in my common.nix

    { config, lib, pkgs, ... }:

    {
      home.packages = with pkgs; [
        git # shortened for readability
        vim # there's more up here in
        ect # the real file

      ] ++ lib.optionals stdenv.isDarwin [
        coreutils # provides `dd` with --status=progress
        wifi-password
        time # GNU time
      ] ++ lib.optionals stdenv.isLinux [
        iputils # provides `ping`, `ifconfig`, ...
        zsh # already on mac
        libuuid # `uuidgen` (already pre-installed on mac)
        emacs # installed with brew on Mac
      ];
    }

The use of lib.optionals will let me do an OS check and allow me to install things I determine are missing exclusively from the other. I like to try to keep everything as "linux feeling" as I can in a terminal, where mac has some things already, and Linux should likewise have them.

That's it?

Oh, not even close. That's just where I am in terms of "allowing nix to eat my files" as it can absorb even the configs related to your applications to keep them central to Nix, and easier to manage as things update and config file format drifts. They won't be silently laying around as cruft, or partial cruft if nix knows what they are, and is how they got there. I haven't decided if I want to give my dots up to Nix, but I'm playing with NixOS and it will be allowed to manage system config files wherever possible. We'll see where it goes, but that's my start into nix. Hope you enjoy reading about this other world even half as much as I'm having fun discovering it.

BONUS!

I stumbled into comma, which allows you to tell nix to fetch a command for just the line it's run on and then toss it away. This makes one off tasks, or tasks like fetching a command to fetch itself, and yes, I've tried to do that recently, insanely easy. I'm sure there's more that I haven't discovered about it's usefulness, but here it is in action.

╭─ > ~/src/zola-blog > main >
╰─ cowsay "I'm not installed"                                                                                                                           ─╯
zsh: command not found: cowsay

╭─ > ~/src/zola-blog > main >
╰─ , cowsay moo                                                                                                                                         ─╯
 _____
< moo >
 -----
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

╭─ > ~/src/zola-blog > main >
╰─ cowsay "I'm not installed"                                                                                                                           ─╯
zsh: command not found: cowsay

I changed the site!

A friend decided to ditch social media and is moving to RSS and Atom feeds, and my old blog was broken in terms of feeds. I didn't ever know how pelican worked, so I figured there wasn't a good reason to fight it if I could try something new, and here we are. The site is being generated by zola now, which has been an interesting transition. Once I got used to things being different, it seems to work a fair bit better, though I haven't noticed the speed aspect. I am on some decently fast hardware, and it's a small site, but features like broken link checking is a nice bonus. I'll open source the site soon and make sure to link to it, but you can see all of my projects on gitea now conveniently linked at the top with the new atom feed. I'm tired of staring at text, so I'm going to keep this one short, but wanted to christen the site with a new post at least. ✌️

Update: The site is fully open source here and will hopefully continue to beb updated there as my old blog was.

I'm beyond tired of computers failing

I've been on a quest lately to get computers to just "go away"™. When I've least expected it, they have alwasy broken. Taken an update, or just decided one day that it didn't want to boot, or acted in a strange way such as no audio, out of date configs, the list goes on. These days, I am not in the mood to deal with breakage at the most random of times. Things like my laptop get booted at most a few times a year, and I really don't want it to be broken on that rare time that I need it. This hasn't just applied to the laptop, but servers as well. For a long while, I was able to just avoid 99% of it by not using SystemD, but it's now even hitting my Artix systems, both the laptop, and a server that I thought I could be too lazy to switch away from because there was no SystemD. That was a mistake it turns out, and has lead me to where I am now.

Computers suck. Now what?

Let's take a step back from rage and consider that tooling has existed for years to solve a ton of this, and I'd even been using some of it for this reason. Docker. Docker has taken the almost everything that I serve, and made it trivial. It basically doesn't break unless someone stops maintaining containers, and is a voodoo box of magic that works. There's two sides of code. The side that is dirty and impure that can break, but that's quite minimal. Writing files, touching the network, ect, can't be clean and have a safe promise that it will work, but the code that actually runs the logic, it mostly can. No more operating system to worry about. There's two things that make Docker (and other OCI containers) great.

Immutability

Because systems aren't mutable, you limit the cruft to only known directories that are shared into the container. When you update it, you aren't ugrading a bunch of old packages, worrying about cruft left behind like old libraries, old programs you forgot to remove, config files that may be out of date, ect. All of it is just... Gone. When you "update", you throw everyhing away, except your limited scope that you share inside the container, meaning that the only data that can go bad, is that little data. If your image is broken, just roll back. If you want to run a completely different image based on another container, go for it. The service still runs, and can be rolled back or forward almost seamlessly, and when things do finally go wrong with your shared data, there's not much to look at to update or debug.

Declarative

One could argue that declaring the outcome as opposed to the input directly is a better solution. Docker does this. You state what you want the container to do, but never tell it how to get there. Obviously someone has to create that at one point or another, but the useful bit is what comes out. Many of the best package managers for *nix systems are this way. Portage on Gentoo, Apk on Alpine. The work on the world file and decide "how to get there", but the user controls only the world file. Directly or indirectly. That only covers packages, and not configurations, but it makes rollbacks more possible, but it doesn't completely solve the problem, and why we need both declaritive and immutable together.

So how do we get there?

Many teams have been working to solve this problem for years. Each have their own drawbacks, but I'm giving some of them a test drive to see how they do for my use. Fedora Silverblue is an immutable OS that uses ostree to build in layers, much like docker. Very limited parts of the system are mutable. This leaves almost nothing to damage when you take updates, or add/remove software as you are intended to use containers such as a toolbox or distrobox which are likewise using OCI containers that can also be rolled back if there are problems. If you somehow take a broken host system update, you can roll that back, even if unbootable as the old state is stored and can be loaded from your bootloader menu as if the update never happened. This allows for easy playing with your system with almost no chance that damage can actually happen, and you throw away whatever you don't want knowing you can easily get it back, almost never even needing a recovery disk unless the bootloader gets damaged. I've started messing with Fedora, including making my own spin on Silverblue, but running Sway that matches my dotfiles at the time it was pushed to git at least. Feel free to give it a look here if you would like to see how it works. It took me a long time to get it running, and only managed to use examples from others like Fale's Desktop work, which my work is based on.

Another option that I'm messing with is Nix and NixOS . Nix is able to run on most platforms, such as Mac/Linux and even Windows. Where as my tool pmm was able to integrate with other package managers to emulate world files, Nix can accomplish similar effect, but keeping everything within a container of sorts, exposing things to the outside system. I've started porting most of my packages out of homebrew on my Mac, and into Nix. This allows me to make it portable through Mac/Linux and, even useless to me but it can, Window. On my servers, I need to do misc tasks to manage the server, but almost none of my tooling that's nice for me to use has any bearing on the system, and only adds more cruft that can prevent the machine from working properly. If I can't use exa instead of ls on my shell, it doesn't matter. I need the host to stay stable, and running as intended. Keeping everything contained in Nix also allows me to seperate the "me" from "the system" which leaves everything more porable. You can see my progress, as well as how I'm using that here. I'm using home-manager to keep my personal configuration and tooling away from the system, while also using NixOS on my laptop to declare "how the system is supposed to look" ignoring what my user intends to use on the system. I haven't gone the full Nix route, rewriting all of my configs, docker containers, ect all in their format as I want to keep things portable, but we'll see if that becomes a solution to problems that I hope won't happen yet.

Conclusion

I don't have all the answers, and I don't know what will go wrong next, but staying stagnant was going to leave me not wanting to remember that computers even exist. Immutable and declaritive systems are not what people are used to. It's analogous to someone that's only written procedural code and going to a fuctional language. It's a complete differet mindset. Problems have to be solved in new ways, and things that didn't used to be a problem, may now be one, but at least it should act consistent with this model. We'll see if it's consistently bad, or good, but I'll take consistency at this point over anything.