Writing and sharing scripts is essential to productivity in development teams. But if you want the script to work everywhere, you can only depend on the tools that are everywhere. That usually means not using the newer features of a language, and not using libraries. In this article, I explain how I use Nix to write scripts that can run everywhere, regardless on what they depend on.
In my work as a software engineer, I often write small scripts to help with administrative actions or provide some nice view of what we are working on. From time to time, one of these scripts is useful enough that I want to share it with my colleagues.
But sharing the scripts comes with a challenge. I write my scripts in Python because it is a great language for that kind of things. But the best parts are outside of the standard library. As a result, to share my scripts, I also have to write a documentation explaining how to get the right version of python, create a python virtual environment, activate it, install the dependencies, and run the script. That's not worth the hassle, and my colleagues won't bother following the complex set of instructions for a script.
Recently, I've started using Nix to make self-contained scripts, and it's been working pretty well.
A script that can download its dependencies
For example, some of the things I like using in python scripts are:
- the new string formatting syntax in Python 3.6,
- the requests package for anything that touches HTTP
- docopt for argument parsing.
My scripts will usually look like:
#! /usr/bin/env nix-shell
#! nix-shell -i python -p python36 python36Packages.requests2 python36Packages.docopt -I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixos-17.03.tar.gz
# You'll need nix to automatically download the dependencies: `curl https://nixos.org/nix/install | sh`
some_script.py <user> <variant> [--api=<api>]
some_script.py -h | --help
<user> format: 42
<variant> format: 2
-h --help Show this screen.
--api=<api> The api instance to talk to [default: the-api.internal]
from docopt import docopt
def doit(user, variant, api):
# The actual thing I want to do
if __name__ == '__main__':
arguments = docopt(__doc__)
doit(arguments['<user>'], arguments['<variant>'], arguments['--api'])
Now, anybody can run the script with
./some_script.py, which will download the dependencies and run the script with them. They'll need to install nix if they don't have it yet (with
curl https://nixos.org/nix/install | sh).
What actually happens there?
The first line of the script is a shebang. It tells the system to run the script with
nix-shell, one of the tools provided by Nix.
The second line specifies how to invoke
-i python: this argument means we want
nix-shellto run the script with the
-istands for interpreter).
-p python36 python36Packages.requests2 python36Packages.docopt: this argument is the list of the nix packages that my script is going to use. Here, I want python 3.6, and the python packages requests and docopt. I usually use the Nix packages page to find the name of the dependencies I want.
-I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixos-17.03.tar.gz: the version of nix I want to use for this script. This ensures people running the script in the future will get compatible versions of the dependencies.
When I run the script,
nix-shell starts by downloading
https://github.com/NixOS/nixpkgs-channels/archive/nixos-17.03.tar.gz if it has not done so recently. This is the definition of all the nix packages. Then, nix-shell looks into this file to know how to download the packages I want and their dependencies. Finally, it runs my script in an environment where the dependencies are available.
Try it now
This technique allows coworkers to share scripts written in different technologies without the cost of teaching everybody how to install dependencies and run scripts in each language.
Try it yourself right now with the example script from earlier. Make sure you have Nix (or install it with
curl https://nixos.org/nix/install | sh), get the script, and run it without caring about the dependencies!