Software Environment¶

⚠ Disk space disclaimer

Your home directory is subject to the CPPM lab quota (see Disk Space). Software environments (Conda, Pixi, Guix profiles…) can easily reach tens of gigabytes and will quickly fill your home directory.

We strongly recommend storing your environments on /datadec, for example: bash /datadec/cppm/<username>/envs/ and configuring each tool to use that path (see instructions below).

Conda (Miniforge / Micromamba)¶

Conda is a popular package and environment manager widely used in data science and scientific computing.

We recommend installing Miniforge, a minimal Conda installer that defaults to the conda-forge channel:

wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh -p /datadec/<username>/miniforge3

💡 Consider using Micromamba instead

Micromamba is a fast, standalone reimplementation of Conda with no base environment and a much smaller footprint. It is a drop-in replacement for most conda commands and is generally faster at solving environments.

To avoid filling your home directory, configure Conda to store environments and packages on /datadec:

conda config --add envs_dirs /datadec/<username>/envs
conda config --add pkgs_dirs /datadec/<username>/pkgs

For a full tutorial on using Conda environments on the DEC, refer to: 👉 CPPM Conda environments tutorial

Pixi¶

Pixi is a modern, fast package manager built on top of the Conda ecosystem (it uses the same channels and package format) but with a project-centric workflow similar to Cargo or Poetry.

Why Pixi over Conda?¶

Feature	Conda	Pixi
Environment definition	`environment.yml` (separate file)	`pixi.toml` (co-located with project)
Lockfile	Optional (`conda-lock`)	Built-in, automatic
Reproducibility	Limited	First-class: exact lockfile committed with the project
Speed	Moderate	Very fast (written in Rust)
No base environment	No	Yes — isolated by design

Pixi ties the environment to the project directory, making it easy to share and reproduce the exact same environment across machines.

Getting started¶

Install Pixi (no admin rights required):

curl -fsSL https://pixi.sh/install.sh | bash

To store the Pixi cache on /datadec instead of your home directory:

export PIXI_CACHE_DIR=/datadec/<username>/.pixi/cache

Add this line to your ~/.bashrc to make it permanent.

Initialise a project and add dependencies:

pixi init myproject
cd myproject
pixi add python numpy matplotlib
pixi run python script.py

For full documentation, refer to 👉 pixi.sh.

Guix¶

GNU Guix is a functional package manager and an advanced distribution system for the GNU system. It provides reproducible, user-controlled, and transparent package management with no need for administrative privileges.

If you are not familiar with Guix, we had a seminar at CPPM by Konrad Hinsen — watch it here. There is also a practical introduction on the gitlab here.

Guix vs. Conda¶

Reproducibility: bit-for-bit reproducible environments, down to the binary level.
No base environment: users install software without affecting others.
Transparency: every build is auditable and modifiable.
Time travel: reproduce an environment from months ago exactly.

Installing packages¶

guix install htop

Search for packages:

guix package --search=python
guix package -A python          # list all available versions
guix install python@3.8         # install a specific version

Managing environments¶

guix shell python jupyter                        # ephemeral environment
guix shell --pure python jupyter                 # clean environment variables
guix shell --container python jupyter            # fully isolated container

Export and reproduce an environment:

guix shell --pure python --export-manifest > manifest.scm
guix describe -f channels > channels.scm
guix time-machine --channels=channels.scm -- shell --manifest=manifest.scm

Creating a Singularity container¶

guix pack -f singularity python jupyter

This produces a Singularity image that can run on any cluster with Singularity, without requiring Guix.

Combining Guix with pip or rustup¶

Guix packages may not always be as up-to-date as PyPI or crates.io. You can combine both:

guix shell python python-pip
pip install pytorch          # pip packages are not managed by Guix

guix shell rust
rustup install nightly       # rustup packages are not managed by Guix

Note: packages installed via pip or rustup outside of Guix are not reproducible through Guix mechanisms.

For more information, refer to the Guix documentation.