Introduction

What I love the most about winter are snow flakes! So you must imagine that when I stumbled upon nix and it’s flakes I was immediately very interested.

During my studies, I often had the chance to work on big projects which tend to span over multiple repositories, each with their own set of features needed to achieve a global goal. When using the project, cloning, installing and running the differents parts of the project can be a tedious task. To facilitate the process, we used Nix to manage the dependencies, configure and build the differents part of the project, and finally run it.

In this tutorial, we will see, step by step, how to use Nix and how to manage a simple project using Nix and why it can be convenient.

Requirement

Before we begin this tutorial, be sure to:

  • Install the packet manager nix
  • Enable nix’s experimental-feature: add this line in /etc/nix/nix.conf: extra-experimental-features = nix-command flakes.
  • Have some basic knowledge with Git and Linux ecosystem.

What is Nix?

Before we start let’s ask ourselves a question: What is Nix?

Nix can designate two different things: the Nix language or the Nix package manager.

The Nix Language

Nix is a pure functional language(just like Haskell or OCaml!), meaning that all functions are considered as any other value and do not have side-effect.

Additionally it is:

  • Deterministic: An expression will always give the same results, given the same output.
  • High-level: Most of the complexity is abstracted, and you don’t have to handle memory access.
  • Lazy: Expression are only evaluated if necessary.
  • Dynamically Typed: The type of an expression is determined at runtime.
  • Declarative: The instructions are not carried out in the order in which they were written.

The language supports nine types of value:

  • String: Can be used with " for single line strings or '' for multi-line ones:
''  
Hello 
This is a multi-line 
string
''
  • Integer: 42.
  • Boolean: true or false.
  • Null: The null value.
  • Float: 42.42.
  • Path: A path is a completely different type of data than string and are written ~/Downloads for example. Unlike string, the paths are not surrounded by " so be careful to not confuse those two types or you will spend a few minute understanding why things don’t want to go your way.
  • List: A list is an ordered collection of elements and can be composed of multiple value separated with a space. Like python list, a Nix list can contain multiple different types of value within the same list. We can thus have:
1

[ 1 2 6 [ "Nix" ] ./epita/gistre/ ]
  • Set: A set is composed of couples key = value separated with a ; (be careful the last couple also needs a ;!). As it is a set, there is no notion of order (and it won’t matter as the language is declarative and not sequential). Any type of value can be present as a value in the set, you can even put other fields from the same set! (you will need to add the keyword rec before the set declaration). A typical set will have this form:
1
2
3
4
5

rec {
    hi = "hello";
    bestEvent = "Raclette GISTRE";
    secondBestEvent = bestEvent;
}
  • Function: A function takes exactly one argument and return the processed value. In it’s simplest form you can write something like that:
1

a: a + 42

Now you may ask, how do we make functions that takes multiple arguments? The answer is you can’t! You can, instead, make a function that takes one argument and returns another function that takes another argument and do that for each arguments. For example a simple add function would be:

1
2
3
4
5
6
7

nix-repl> add = a: b: a + b # Here we store the entire outer function in a variable, it's a data type like any other! 
nix-repl> addA = add 42 # Here we have the function b: 42 + b stored in addA 
nix-repl> addA 28 # We have here the second part and we finally have 42 + 28 = 70
70

nix-repl> add 42 28 # We can also directly call our function with the 2 arguments and it will direclty return the right result
70

In addition to the usual operators (boolean operators and arithmetic operator), Nix offer a few more interesting ones:

  • .: Access the value of a set.
    For example {hi = "hello"; theAnswer = 42;}.theAnswer gives 42.
  • ?: Test the existance of a value in a set.
    This time {hi = "hello"; theAnswer = 42;}.theAnswer gives true.
  • //: This operator merges two set together.
    If we take {hi = "hello";} // {theAnswer = 42;} we get {hi = "hello"; theAnswer = 42}.
  • ++: This operator performs the same thing as // but for lists.
    [1 2 3] ++ [4 5 6] gives [1 2 3 4 5 6].

Finally, we will explore a few useful control structures:

  • Variable declaration:
    In Nix you can only declare constant variables (pretty ironic I know).
    For that you can use the structure let vardecs in expression.
    As for the set each variable declaration is in the form of name = value and must be followed by a ;. All of the variable declared in the let part can be used in the in part and the entire structure will take the value of the last instruction of the in part.

  • Conditional: Like many other languages, Nix allows you to conditionally execute code. For that you can use the syntax if condition then branch1 else branch2.
    But contrary to other language (like C for example), a conditional structure must return a value and thus you cannot omit the else clause.

  • Assertion: When coding it is very useful to detect error and stoping the program when encountering one.
    In Nix we use the structure assert condition; expression.
    If the condition is false, the program will stop and it will raise an error.
    If it’s true, the structure will take the value of the expression after the ;.

We can discover a lot more functionalities and fun things in the Nix language but we only need to cover the basics in this tutorial. If you want to find more about the possibilities of the language, I can only advise you to read the nix course of Litarvan where you will find more detailed informations.

The Nix package Manager

Nix can also designate the package manager, which double as a build system.

Nix is closely linked to nixpkgs, which is a github repository containing the recipes for almost every package you will ever need (except of course some very specific ones you need during GISTRE’s courses).

I say recipes because it does not contain the sources of the package you want to build but often a default.nix file which will describe where to find the sources and how to build them with which dependencies. The advantage is that when requesting the package from nixpkgs, the package will be built in an environment with only the right dependencies in their right version (which can be different from the one on your system).

Obviously, the process of building from source can be very long. To prevent that, nix offers a cache with prebuilt binaries for most of the package you can find in nixpkgs (except those who are under unfree license) stored at https://cache.nixos.org/. If you need, you can even create your own cache with your own binaries.

Nixpkgs additionally offer a large standard library of nix function you can use to better create your packages, operate on your sets or lists, manipulate your nix expressions and many other cool features. We will see some of them during this tutorial but if you want to learn more about nixpkgs here

Derivations

In the previous section I talked about recipes and packages. Actually, in Nix there is no notion of “packages”, we call them derivations. A derivation is not only a “nix-styled package”, it can also contain any types of files (generated files, configuration files, patch files to name a few). Once they are built, derivations are stored in the nix store which is a read-only directory where all of your derivations are stored using the format <hash>-name. As the directory is read-only if you want to modify something, you need to either rebuild the derivations if it’s yours or, if it’s not, you will need to recreate the derivation configuration locally and then rebuild it.

Now you may ask, why do we have to go through the trouble of all of that?
The answer is simple: reproducibility.
Derivations are build in isolated environments with only the dependencies we want inside. This way, we can be sure that every time we build a derivation, we can be certain that we will have the same results.

We will now see how to build a short derivation. For that we will need a function from nixpkgs called mkDerivation (nixpkgs.stdenv.mkDerivation). This function is a wrapper around the nix builtin derivation, and prefills for you some of the tedious fields need by derivation.

Let’s study the following nix file where we create a derivation that exposes two simple executables.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

let 
    nixpkgs = import <nixpkgs> { };
in 
    nixpkgs.stdenv.mkDerivation {
        pname = "hello-gistre"; # Name of the derivation
        version = "1.0.0"; # Version of the derivation

        buildInputs = [ nixpkgs.lolcat ]; # Other derivations needed by this one

        src = nixpkgs.fetchgit { # Fetch the sources from a git repository
            url = "https://github.com/Matheo-Rome/helloNix";
            rev = "779ce0e5577115e188bcaf9961ffb33e829d871c";
            hash = "sha256-gCSoB+VqWJJqKjNliaJ33ovuAz1/sjuFLFCUqFVaKQY=";
        };

        buildPhase = '' 
            make
        ''; # Build the sources
        installPhase = ''
            mkdir -p $out/bin
            make install PREFIX=$out
            echo -e "#!/bin/sh\n$out/bin/hello | ${nixpkgs.lolcat}/bin/lolcat" > $out/bin/hello_colored
            chmod +x $out/bin/hello_colored
        ''; # Install the executables
    }

As I just said, we use mkDerivation to create our derivation. This function takes a set as an argument which contains all the information required to build the derivation:

  • pname: The name of the derivation.
  • version: The version of the derivation.
  • buildInputs: A list of other derivations this one depends on. Here, we include lolcat to print a colorful hello message.
  • src: The sources of the derivations. Here, we use another function from the standard library fetchgit which does exactly that: clone the sources from a git repository. fetchgit takes another set with three keys:
    • url: The URL of the git repository. Here it points on the main branch of this repository.
    • rev: The sha of the commit (the weird alphanumerical string at the end of the URL when you select a commit on github/gitlab).
    • hash: Used to check the integrity and consistency of the fetched repository.
      If you don’t know the hash value, you can, for now, just put a random valid hash like sha256-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxw=.
  • buildPhase: A string of shell commands to build our sources. Here we use make to create our executable.
  • installPhase: A string of shell commands to install our executable on our system. Here we install the generated executable inside the nix-store directory. For that we use the $out variable, set by nix which contains the path to the created store. Finally we create a second executable, this time a shell script who will call our first executable and pipe it into lolcat. We use the variable ${nixpkgs.lolcat} to point exactly where the lolcat binary is as it will not be available in the PATH (except of course if you installed lolcat in your system before).

Once we have this file, all we have to do is build our new derivation! For that we use the command nix-build:

1

sh$ nix-build hello.nix

The first time your run this command you will certainly have the following error:

1
2
3
4

error: hash mismatch in fixed-output derivation '/nix/store/wgcrqfdp9x5j4cbhlq8q0qb652hs1af0-helloNix-779ce0e.drv':
         specified: sha256-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxw=
            got:    sha256-gCSoB+VqWJJqKjNliaJ33ovuAz1/sjuFLFCUqFVaKQY=
error: 1 dependencies of derivation '/nix/store/njhhw7aqjg8p2y8h1hwkwkq6mikx0z29-hello-gistre-1.0.0.drv' failed to build

This will give you the right hash to use in the fetchgit.hash. Once you rebuilded your derivation, you should find yourself with a symlink to the nix-store named result. Inside there will be your two executables and if you execute hello_colored you should see “Hello GISTRE!” colored using lolcat even if lolcat is not installed on your system.

Nix flakes

Another very useful nix feature are the nix flakes.

They allow to manage projects by managing their inputs (dependencies) and outputs. As everything in Nix they allow good reproducibility and easy dependencies management but they offer some very cool additionals features.

If you type in your terminal:

1

nix flake init

You will find yourself with your very own flake!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

{
  description = "A very basic flake";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
  };

  outputs = { self, nixpkgs }: {

    packages.x86_64-linux.hello = nixpkgs.legacyPackages.x86_64-linux.hello;

    packages.x86_64-linux.default = self.packages.x86_64-linux.hello;

  };
}

Let’s review this flake step by step:

  • description: A string describing what the flake is doing/used for.
  • inputs: A set containing all the dependencies of the flake.
    Each of those dependencies is a git repository containing a flake and is represented by another set containing information on where to find the repository you need. You can also use repositories without flake as an input, but you will need to add the tag flake = false; in the set of this specific input.
    Here we only give the URL with the additionals information passed as query parameters but you can also fill the fields type, owner, repo, rev, ref.
    An equivalent to the first notation would be something like:
inputs = {
    nixpkgs = {
        type = "github";
        owner = "nixos";
        repo = "nixpkgs";
        ref = "nixos-unstable";
    };
};
  • outputs: It is defined as a function that takes a set containing the flake itself (self) and all the inputs (here we only have nixpkgs) and returns a set containing everything our flake exposes. Here for example the flake exposes two derivations: default and hello. The first one takes the hello derivation from the legacyPackages (here, legacy doesn’t mean deprecated, it’s just a naming convention) and stores it. The second one takes the first one and define it as the default output of the flake (if we call nix build without any arguments it will build this derivation). You can also note that the architecture and the os are specified. This is because most of the derivations are stored in a subset depending on which system you are using. Moreover the building strategy will be different for each system (Linux will use gcc and Darwin clang by default).

Once our flake is setuped we can use nix-build to build our desired derivations. We can also use the nix-run command to directly build and run the executable in the derivation if there is one:

1
2

sh$ nix-run .#hello
Hello, world!

Another nice feature of flakes are devShells.
They allow to create a development shell with specific derivations (either from nixpkgs or from your own flakes). Those derivation will be made available inside this shell while not being accessible from you whole system. This can be very useful when working on specific fields with technologies you don’t necessarily need to have on your whole os.
To declare a devShell you can add in the followings lines in the output section of your flake:

1
2
3

devShells.x86_64-linux.default = nixpkgs.legacyPackages.x86_64-linux.mkShell {
  buildInputs = [ nixpkgs.legacyPackages.x86_64-linux.lolcat self.packages.x86_64-linux.hello ];
};

Now when running the command nix develop, you will enter a shell where you can access all the derivation’s binaries easily.

Writting everything in your flake.nix can make it difficult to read. To avoid that you can use the keyword import to separate your code into subfiles. You juste need to keep in mind that flakes and git are closely related as the flake will only “see” file that are tracked by git. So if you ever have an error where a flake seems to not see a file, don’t forget to check if the file has been committed at least one before.

Also when working with a flake which depends on another of your repository, you will need to use the command nix flake update to pull the modification you could have done on the others repository in your current one.

Finally, flakes works with a system of locking where each input version is stored in a flake.lock to maintain the good reproducibility and easily see which version of an input you are currently using.

Building your first project

Now that we’ve seen all of that we can assemble everything we just saw. For that we will use the repository from before but on another branch where a nix flake is setup to build a library. You can find it here.

First, we will retake our flake from before and modify it’s inputs to add the repository as a dependency:

1
2
3
4

inputs = {
  nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
  helloNix.url = "github:Matheo-Rome/helloNix?ref=add_flake_to_hello";
};

The helloNix repository will provide a libhello.a and a hello.h. In our current repository, we will create a directory greeter with a main.c and a Makefile:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

#include <stdio.h>
#include "hello.h"

int main(int argc, char **argv)
{
    if (argc != 2)
    {
        printf("Need someone to greet!\n");
        return 1;
    }

    hello(argv[1]);
    return 0;
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

CC=gcc 
CFLAGS=-Wall -Wextra -Werror -pedantic -std=c99 -Wvla
PREFIX=/usr/bin
LIB_PATH=/usr/
TARGET=greeter

all: $(TARGET)

$(TARGET):
	$(CC) -o $@ main.c -I$(LIB_PATH)/include  -L$(LIB_PATH)/lib/ -lhello

install: 
	install -m 0755	$(TARGET) $(PREFIX)/bin

clean: 
	$(RM) *.o $(TARGET) *.a

Those two file are pretty standard and just build a binary with the library at LIB_PATH.

Now that we have all of our files, we can build our derivation that will link the static library and our source code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

greeter = pkgs.stdenv.mkDerivation {
  pname = "greeter"; # Name of the derivation
  version = "1.0.0"; # Version of the derivation

  buildInputs = [ helloNix.packages.${system}.libhello ];
  src = ./greeter; # get all the files in the hello directory

  buildPhase = '' 
    make LIB_PATH=${helloNix.packages.${system}.libhello}
  ''; # Build the final binary 
  installPhase = ''
    mkdir -p $out/bin
    make install PREFIX=$out
  ''; # Install the binary
};

Here we create the binary by using our libhello as a build input and then using the variable ${libhello.packages.${system}.libhello} to get it’s path from the nix store.

Once this is done, we can create an devShell with our new derivation inside or add an apps section to indicate that this binary destined to be executed.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

packages.${system}.greeter = greeter;

devShells.${system}.greeter = pkgs.mkShell {
  buildInputs = [ self.packages.${system}.greeter ];
};

apps.${system}.greeter = {
  type = "app";
  program = "${self.packages.${system}.greeter}/bin/greeter";
};

The apps sections allows us to directly call our binary with nix run:

1
2

sh$ nix run .#greeter Gistre
Hello Gistre!

And of course you can still use nix develop to access the binary directly without nix run if needed.

At the end your flake should look something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

{
  description = "A very basic flake";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
    libhello.url = "github:Matheo-Rome/helloNix?ref=add_flake_to_hello";
  };

  outputs = { self, nixpkgs, libhello }: 
    let 
      system = "x86_64-linux";
      pkgs = nixpkgs.legacyPackages.${system};

      greeter = pkgs.stdenv.mkDerivation {
        pname = "greeter"; # Name of the derivation
        version = "1.0.0"; # Version of the derivation

        buildInputs = [ libhello.packages.${system}.libhello ];
        src = ./greeter; # get all the files in the hello directory

        buildPhase = '' 
            make LIB_PATH=${libhello.packages.${system}.libhello}
        ''; # Build the final binary 
        installPhase = ''
            mkdir -p $out/bin
            make install PREFIX=$out
        ''; # Install the binary
      };
    in {
      packages.${system}.greeter = greeter;

      devShells.${system}.greeter = pkgs.mkShell {
        buildInputs = [ self.packages.${system}.greeter ];
      };

      apps.${system}.greeter = {
        type = "app";
        program = "${self.packages.${system}.greeter}/bin/greeter";
      };
    };
}

Conclusion

And that’s all you need to start manage your projects with Nix! It may seem like a bit of a learning curve at first, but once you get the hang of it, it really simplifies managing projects with multiple dependencies. For me, it’s been a game-changer, especially when working with projects spanning across different repositories. Plus, once everything is set up, you can easily share it with others and get consistent builds across different environments.

So, if you’re tired of dependency problems or just curious about a new way to manage your projects, give Nix a shot!

Bibliography