/ nixos

Share scripts that have dependencies with Nix

Writing and sharing scripts is essential to productivity in development teams. But if you want the script to work everywhere, you can only depend on the tools that are everywhere. That usually means not using the newer features of a language, and not using libraries. In this article, I explain how I use Nix to write scripts that can run everywhere, regardless on what they depend on.

In my work as a software engineer, I often write small scripts to help with administrative actions or provide some nice view of what we are working on. From time to time, one of these scripts is useful enough that I want to share it with my colleagues.

But sharing the scripts comes with a challenge. I write my scripts in Python because it is a great language for that kind of things. But the best parts are outside of the standard library. As a result, to share my scripts, I also have to write a documentation explaining how to get the right version of python, create a python virtual environment, activate it, install the dependencies, and run the script. That's not worth the hassle, and my colleagues won't bother following the complex set of instructions for a script.

Recently, I've started using Nix to make self-contained scripts, and it's been working pretty well.

A script that can download its dependencies

For example, some of the things I like using in python scripts are:

  • the new string formatting syntax in Python 3.6,
  • the requests package for anything that touches HTTP
  • docopt for argument parsing.

My scripts will usually look like:

#! /usr/bin/env nix-shell
#! nix-shell -i python -p python36 python36Packages.requests2 python36Packages.docopt -I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixos-17.03.tar.gz
# You'll need nix to automatically download the dependencies: `curl https://nixos.org/nix/install | sh`
"""
Some script

Usage:
  some_script.py <user> <variant> [--api=<api>]
  some_script.py -h | --help

Arguments:
  <user>        format: 42
  <variant>     format: 2

Options:
  -h --help     Show this screen.
  --api=<api>   The api instance to talk to [default: the-api.internal]
"""
import requests
from docopt import docopt

def doit(user, variant, api):
    # The actual thing I want to do
    pass

if __name__ == '__main__':
    arguments = docopt(__doc__)
    doit(arguments['<user>'], arguments['<variant>'], arguments['--api'])

Now, anybody can run the script with ./some_script.py, which will download the dependencies and run the script with them. They'll need to install nix if they don't have it yet (with curl https://nixos.org/nix/install | sh).

What actually happens there?

The first line of the script is a shebang. It tells the system to run the script with nix-shell, one of the tools provided by Nix.

The second line specifies how to invoke nix-shell:

  • -i python: this argument means we want nix-shell to run the script with the python command (-i stands for interpreter).
  • -p python36 python36Packages.requests2 python36Packages.docopt: this argument is the list of the nix packages that my script is going to use. Here, I want python 3.6, and the python packages requests and docopt. I usually use the Nix packages page to find the name of the dependencies I want.
  • -I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixos-17.03.tar.gz: the version of nix I want to use for this script. This ensures people running the script in the future will get compatible versions of the dependencies.

When I run the script, nix-shell starts by downloading https://github.com/NixOS/nixpkgs-channels/archive/nixos-17.03.tar.gz if it has not done so recently. This is the definition of all the nix packages. Then, nix-shell looks into this file to know how to download the packages I want and their dependencies. Finally, it runs my script in an environment where the dependencies are available.

Try it now

This technique allows coworkers to share scripts written in different technologies without the cost of teaching everybody how to install dependencies and run scripts in each language.

Try it yourself right now with the example script from earlier. Make sure you have Nix (or install it with curl https://nixos.org/nix/install | sh), get the script, and run it without caring about the dependencies!

Georges Dubus

Georges Dubus

Researcher among the engineers, engineers among the researchers. Python and Haskell, among other things.

Read More