Guix and Spack, like conda-forge and in contrast to PyPI, are source-based package managers. We spend some time and energy so that it is now possible to install Fluiddyn packages with Guix and Spack. These package managers are quite different than conda and pip/PyPI. Unlike pip, they are generalist and not restricted to Python packages.
PyPI is very convenient for Python users. With tools like venv, pip, pipx, PDM or uv, it is very simple to produce in few second Python environments relativement reproducible. PyPI is not a source-based index, meaning that one can upload uncontrolled wheels on PyPI but it makes sense for Python and most packages are either pure Python or adopt a controlled build process (using Github Actions and cibuildwheel). Non Python dependencies can be controlled with other tools (modules, conda or Linux distributions with or without Docker/Apptainer as shown here ). Overall, this solution makes sense and is not that bad. Of course, there are bad cases, like the Pytorch package in PyPI described in this interesting note , but I would argue that this is a very special case.
Conda is convenient because it is also generalist and produce virtual environments
compatible with Python virtual environments, with a standard structure with
lib/python3.13
,
bin
and
share
. Conda-forge is great because it is source-based.
However, Conda-forge is based on conda, which is not “source-based” and it is not
oriented at all on bit-to-bit reproduction. But using an index containing shared builds
also makes it much faster than Guix and Spack.
Unlike conda, Guix and Spack produce environments which are not based on the standard structure on disk. Each package is installed on its own directory, which can potentially be reused for other environments. Guix uses the Guile programming language (not far from LISP) for its package descriptions while Spack uses Python. Guix is used at GRICAD (Grenoble) and Spack is very popular for supercomputing.
I had a good experience with Spack. First, it was possible and not too difficult to create my own Spack packages to install Fluidsim and co. Then I was able to add packages in the main Spack repository (for example https://github.com/spack/spack/pull/46236 ). A negative aspect is the duration of the first builds, which are really long. It’s possible to create modules from a list of Spack packages which could be very convenient for us on supercomputers.
I have to admit that our experience with Guix is less positive. We finally have packages
in
https://gricad-gitlab.univ-grenoble-alpes.fr/bouttiep/gricad_guix_packages/-/tree/master/common
and we are able to run simulations (see
https://foss.heptapod.net/fluiddyn/fluidsim/-/tree/branch/default/doc/examples/clusters/gricad
)
but it was complicated and we had quite a lot of problems, with not clear error messages.
Therefore, we would not have been able to use Guix without the great help of GRICAD staff
(thanks Benjamin Arrondeau and Pierre-Antoine Bouttier). Moreover, we experienced
annoying breakage of a working setup after a
guix
pull
, which is not exactly what we
would expect from a tool so focussed on reproducibility.