Python Developing
15. Feb. 2024
Python is a widely used and easy-to-learn programming language that has found broad application thanks to numerous extensions (in the form of modules and packages). However, as Python and its extensions are constantly being further developed, problems can arise when sharing code. If, for example, a colleague uses a different version of Python or an installed extension, it is possible that the code that works perfectly on your own computer will produce errors on your colleague's computer. If you are also working on different projects at the same time, different extensions can interfere with each other, e.g. if module A requires module B with version 1.3 for project 1, but module B with version 2.1 is to be used for project 2. If all extensions are installed with pip
, this can "fill up" the system version of Python and make working on different projects more difficult. To remedy this problem, you can use a Python version manager (with pyenv
) and a virtual environment (with virtualenv
). How to install these tools under macOS is described in this article.
Pyenv
Pyenv (Github) is a small program that manages Python versions. It is installed either by a script as follows:
curl https://pyenv.run | bash
or using Homebrew (my preferred way to install programs) as follows:
brew update
brew install pyenv
After the installation you have to update the shell start script so that Pyenv and the required environment variables are loaded. On macOS, zsh
is used as a shell, which is why the file .zshrc
in the user folder (i.e. ~/.zshrc
) must be extended as follows:
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.zshrc
echo '[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.zshrc
echo 'eval "$(pyenv init -)"' >> ~/.zshrc
How to change the shell start scripts for other operating systems and shell environments is well described in the Pyenv Github repository.
If you restart the terminal (or load the changes with source ~/.zshrc
), you can enter the command pyenv
and check with the argument --version
, for example, whether the latest version of Pyenv has been installed correctly:
$ pyenv --version
pyenv 2.3.35
For installing a specific Python version, you can use the following command to list all available versions first:
pyenv install --list
In my case, there were 743 different versions that can be installed. To install Python 3.12.1, for example, use the following command:
pyenv install 3.12.1
After successful installation, you can use pyenv versions
to check which versions are now available for selection:
$ pyenv versions
* system
3.12.1
You can see that the new version is installed, but if you enter python --version
in the terminal, you will see that the system version is still being used. This can be changed with the commands pyenv local
and pyenv global
. The keyword local
instructs Pyenv to use a specific Python version in the current folder, while global
specifies a Python version in general.
For testing purposes, we can execute the following command in the ~/Developer/python-test
folder:
pyenv local 3.12.1
and then query the versions:
$ pyenv versions
system
* 3.12.1 (set by /Users/maxi/Developer/python-test/.python-version)
The asterisk indicates that the most recently installed version of Python has now been selected. This is due to the hidden file .python-versions
in the python-test
folder. If you now check the Python version, you will see that this version is now also installed:
$ python --version
Python 3.12.1
The following command shows where the Python interpreter is located:
$ which python
/Users/maxi/.pyenv/shims/python
You can see that Python is stored in a folder created by Pyenv. If several versions of Python are now installed as above, Pyenv saves all versions in this folder and by specifying a version using local
or global
it refers to the corresponding version.
However, this also means that two projects using the same Python version will also be provided with the same extensions (i.e. modules and packages). If you install an extension using pip
, this extension is installed in the .pyenv
folder in the user folder (i.e. ~/.pyenv
) for the currently selected Python version and is always available when the corresponding Python version is used.
It is possible to remove the Python version with the keyword uninstall
and reinstall it cleanly with install
. However, this is inconvenient if you have to switch between different projects and always have to reinstall Python. So we come to the second tool: virtualenv
Virtualenv
A virtual environment created with virtualenv
can be thought of as a local copy of the Python version currently in use, which manages all extensions locally and thus leaves the original Python installation unchanged. The module must first be installed (for each freshly installed Python version) with the following command:
pip install virtualenv
And now you can create a virtual environment in the ~/Developer/python-test
folder, for example, as follows:
python -m virtualenv .venv
This creates a hidden folder called .venv
, which provides the currently activated Python version locally. Activate this virtual environment with:
source .venv/bin/activate
and you can now use pip
to install extensions locally, i.e. without changing (or "filling up") the original Python version in the .pyenv
folder or the system version. All extensions are only installed locally in the .venv
folder.
If you want to leave the virtual environment, it is deactivated as follows:
deactivate
You can delete and recreate the .venv
folder without hesitation and are therefore flexible from project to project with regard to the installation of different extensions.
Last tip
If you have installed several extensions in a virtual environment and would like to share your code, the following approach is recommended. Use the following command to specify all modules and packages in a file:
pip freeze > requirements.txt
so that a colleague (after setting up and activating the virtual environment with the corresponding Python version) can quickly install all dependencies and execute the code:
pip install -r requirements.txt
This means you don't have to share the .venv
folder with all extensions (especially not on Github). Instead, you only need to specify which Python version to use (e.g. in the readme.md
file or through the .python-version
file) and you need to specify the dependencies through the requirements.txt
file. This means that everything is ready to execute the code.