Python

The Python Stack

The python management stack

We use various tools to manage our python installs and projects in order to make life a bit easier. Below you’ll find instructions on how to install and use these.

Python versions

At the time of writing, Ubuntu provides Python 3.12 by default. If you need an older or newer interpreter because of what a specific project you work on uses, there’s two ways to do that.

If you’re on Ubuntu, the easiest way is to manually install any Python versions you’ll need later on. On other operating systems, pyenv might be for you.

Manual installs

In case your Ubuntu doesn’t provide the specific version you’re looking for, add the PPA first:

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install python3.13-dev python3.13-distutils

pyenv

Pyenv is a version manager for python. It works on macOS/Linux, and on Windows inside WSL. Installation can be done via brew. Windows outside WSL is available as a fork.

First install the suggested build environment:

# macOS:
$ brew update; brew install openssl readline sqlite3 xz zlib tcl-tk
# Linux (debian based):
$ sudo apt update; sudo apt install build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev curl git libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev

Then install pyenv itself:

brew update; brew install pyenv

Once it’s installed you need to add it to your path (replacing zshrc with bashrc if not using zsh):

echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.zshrc
echo '[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.zshrc
echo 'eval "$(pyenv init -)"' >> ~/.zshrc

Once you’ve installed it you can install various python versions via the CLI:

pyenv install 3.12

If you need one of those python versions to be your CLI version, you can use:

pyenv global 3.12

pipx

pip is a general Python package installer. It can be used to install libraries or CLI applications with entrypoints.
pipx is a specialized package installer. It can only be used to install packages with CLI entrypoints.
pipenv is a CLI app used to develop applications and libraries

sudo apt-get install pipx

To make sure everything you install using pipx can be executed as you would expect, make sure ~/.local/bin is added to your PATH. Then, reload the shell for the changes to take effect:

pipx ensurepath

Pipenv

Pipenv is used for managing dependencies and virtual environments. Follow instructions on the Pipenv website to install it for your system. For Ubuntu, it’s:

pipx install pipenv

Conventions

Be sure to check out our Python conventions.

Tricks

Dependency bumping

Updating dependencies using pipenv might not always succeed. This might:

rm Pipfile.lock
pipenv lock --clear

Debugging

The following tools are very convenient for development:

Name	Description
pdb++	A pre-configured pdb including tab completion and syntax highlighting.
pytest-watch	Run pytest when a file is changed.
sphinx-autobuild	Build sphinx documentation on change and serve the rendered docs on a local HTTP server.

Python supports a large variety of environment variables that facilitate tracking down issues. Noteworthy are the following:

Name	Compatibility	Description
PYTHONTRACEMALLOC	>= Python3.4	Trace Python’s memory allocations. The value of this variable is the maximum number of frames stored in the traceback of a trace.
PYTHONASYNCIODEBUG	>= Python3.4	Set to a non empty string to enable the debug mode of the asyncio module. NB: It is not recommended to enable this by default as the large number of traces easily flood the log and may even result in reaching the maximum limit of the GitLab’s configured maximum logfile size.
PYTHONTHREADDEBUG	—with-pydebug	If set, Python will print threading debug info.
PYTHONDUMPREFS	—with-pydebug	If set, Python will dump objects and reference counts still alive after shutting down the interpreter.
PYTHONMALLOCSTATS	—with-pydebug	If set, Python will print memory allocation statistics upon every object creation and upon shutdown.

More info on Python’s available environment variables can be found in the official Python documentation - Chapter1: Command line and environment.