Python is a widely used and easy-to-learn programming language that has found broad application thanks to numerous extensions (in the form of modules and packages). However, as Python and its extensions are constantly being further developed, problems can arise when sharing code. If, for example, a colleague uses a different version of Python or an installed extension, it is possible that the code that works perfectly on your own computer will produce errors on your colleague's computer. If you are also working on different projects at the same time, different extensions can interfere with each other, e.g. if module A requires module B with version 1.3 for project 1, but module B with version 2.1 is to be used for project 2. If all extensions are installed with pip, this can "fill up" the system version of Python and make working on different projects more difficult. To remedy this problem, you can use a Python version manager (with pyenv) and a virtual environment (with venv or virtualenv). How to install these tools under macOS is described in this article.

Pyenv

Pyenv (Github) is a small program that manages Python versions. It is installed either by a script as follows:

curl https://pyenv.run | bash

or using Homebrew (my preferred way to install programs) as follows:

brew update
brew install pyenv

After the installation you have to update the shell start script so that Pyenv and the required environment variables are loaded. On macOS, zsh is used as a shell, which is why the file .zprofile in the user folder (i.e. ~/.zprofile) must be extended as follows:

if [ -e /opt/homebrew/bin/pyenv ]; then
    export PYENV_ROOT="$HOME/.pyenv"
    eval "$(pyenv init --path)"
fi

How to change the shell start scripts for other operating systems and shell environments is well described in the Pyenv Github repository.

If you restart the terminal (or load the changes with source ~/.zprofile), you can enter the command pyenv and check with the argument --version, for example, whether the latest version of Pyenv has been installed correctly (as of October 2024):

$ pyenv --version
pyenv 2.4.16

For installing a specific Python version, you can use the following command to list all available versions first:

pyenv install --list

In my case, there were many different versions that can be installed. To install Python 3.13.0, for example, use the following command:

pyenv install 3.13.0

After successful installation, you can use pyenv versions to check which versions are now available for selection:

$ pyenv versions
* system
  3.13.0

You can see that the new version is installed, but if you enter python --version in the terminal, you will see that the system version is still being used. This can be changed with the commands pyenv local and pyenv global. The keyword local instructs Pyenv to use a specific Python version in the current folder, while global specifies a Python version in general.

For testing purposes, we can execute the following command in the ~/Developer/python-test folder:

pyenv local 3.13.0

and then query the versions:

$ pyenv versions
  system
* 3.13.0 (set by /Users/maxi/Developer/python-test/.python-version)

The asterisk indicates that the most recently installed version of Python has now been selected. This is due to the hidden file .python-versions in the python-test folder. If you now check the Python version, you will see that this version is now also installed:

$ python --version
Python 3.13.0

The following command shows where the Python interpreter is located:

$ which python
/Users/maxi/.pyenv/shims/python

You can see that Python is stored in a folder created by Pyenv. If several versions of Python are now installed as above, Pyenv saves all versions in this folder and by specifying a version using local or global it refers to the corresponding version.

However, this also means that two projects using the same Python version will also be provided with the same extensions (i.e. modules and packages). If you install an extension using pip, this extension is installed in the .pyenv folder in the user folder (i.e. ~/.pyenv) for the currently selected Python version and is always available when the corresponding Python version is used.

It is possible to remove the Python version with the keyword uninstall and reinstall it cleanly with install. However, this is inconvenient if you have to switch between different projects and always have to reinstall Python. So we come to the second tool: venv

Venv

A virtual environment created with venv can be thought of as a local copy of the Python version currently in use, which manages all extensions locally and thus leaves the original Python installation unchanged. The module is shipped with modern versions of Python, but for older versions, a comparable module named virtualenv can be installed (for each freshly installed Python version) with the following command:

pip install virtualenv

The following commands for the module venv mutually apply to virtualenv.

And now you can create a virtual environment in the ~/Developer/python-test folder, for example, as follows:

python -m venv .venv

Thereby, the module venv (characterized by the -m in the command) creates a folder called .venv (which is also hidden due to the point preceding the name), which provides the currently activated Python version locally. Activate this virtual environment with:

source .venv/bin/activate

and you can now use pip to install extensions locally, i.e. without changing (or "filling up") the original Python version in the .pyenv folder or the system version. All extensions are only installed locally in the .venv folder.

If you want to leave the virtual environment, it is deactivated as follows:

deactivate

You can delete and recreate the .venv folder without hesitation and are therefore flexible from project to project with regard to the installation of different extensions.

Last tip

If you have installed several extensions in a virtual environment and would like to share your code, the following approach is recommended. Use the following command to specify all modules and packages in a file:

pip freeze > requirements.txt

so that a colleague (after setting up and activating the virtual environment with the corresponding Python version) can quickly install all dependencies and execute the code:

pip install -r requirements.txt

This means you don't have to share the .venv folder with all extensions (especially not on Github). Instead, you only need to specify which Python version to use (e.g. in the readme.md file or through the .python-version file) and you need to specify the dependencies through the requirements.txt file. This means that everything is ready to execute the code.