When I began coding in Python, one of the more confusing issues for me was how to manage appropriately the packages that I installed on my computer. My usual workflow was that when I needed a new package, I installed it in the default Python system without understanding what I was doing. Consequently, my Python began to raise errors due to the incompatibility issues between packages. Also, my codes didn’t work in my colleagues’ machines because they didn’t have identical versions of the packages that I used. Some months later, I discovered this mysterious concept of virtual environments. When I find out the utility and how to use them, it entirely improved my coding experience.
A virtual environment is basically an isolated setting where you can specify all the features related to dependencies and their versions to develop a particular project. In simple words, it is installing a new version of a software (said Python, Rstats, Javascript) in your computer that does not share anything with the default version or with other environments. In this context, virtual environments allow you to:
There are two well-defined and documented ways of creating virtual environments for Python: virtualenv
and conda
. In one hand, we have virtualenv
, an environment manager that allows us to create and control our environments. The easiest way of installing packages is through pip
. For Stata users, this is equivalent to ssc
. On the other hand, we have conda
, both an environment manager and a package manager.
In this post, I will teach you how to create environments with both tools and take advantage of this amazing tool!
venv
.conda
.venv
.Congrats!! You just created an environment. If you are using Anaconda, your terminal probably will look like this:
Now, we can begin installing packages in our virtual environment. For illustration purposes, we will use one of the most critical packages to perform scientific computing: NumPy
.
There are many variations and commands to install packages using pip
. Here is the link to the documentation.
It is common to install multiple packages for a project. In addition to numpy
, let’s imagine you need to work with dataframes (pandas
) and graphs (NetworkX
). You can specify a requirements.txt
file to manage all the packages that you will require in one place.
Install all the packages in requirements.txt
using the following command:
Finally, to deactivate the environment or delete it:
conda
.We already have some idea of how to use venv
together with pip
to manage environments and packages. An alternative and widely used form to achieve the same is use conda
. As we discussed earlier, conda
is both an environment and package management system, so you can create environments and install packages just with conda
. Depending on your operative system, click here to install conda
.
By default, all your environments will live in a directory inside your conda
directory. For example, in my machine the environment was saved in /Applications/anaconda3/envs/your-env-name
. This approach is different for the one followed by venv
, because the latter creates the environment in the same folder of the project.
As a package manager, conda
installed packages from Anaconda Repository by default. You can also install packages from third parties repositories (Conda-Forge is the most popular one) and also from pip
. This is how it works!
One thing that I found amazing about managing environments with conda
is that you can specify every aspect of the configuration in a single .yml
file. For example, let’s assume that you have a environment.yml
configuration:
name: your-env-name
channels:
- conda-forge
- defaults
dependencies:
- python=3.7
- pytest
- ipykernel
- ipython
- pip:
- autopep8==1.5.4
- dropbox==10.4.1
- paramiko==2.7.2
- redis==3.5.3
- twilio==6.41.0
With this file you can create environments using:
Also, if you want to save the specification of an environment in a .yml
you can do it!
For more specific details, read the docs!
There are several packages that I always want to have in any environment I use. Here is the specification :)!
name: null
channels:
- conda-forge
- defaults
dependencies:
- python>=3.7 # Specify your desire version
- ipython # To develop and run my code
- pytest # To perform testing to the code
- autopep8 # Code formatter for the PEP8 guidelines
- mypy # Static typing
- flake8 # For linting (spell and grammar checker of python code)
prefix: path/to/env/conda-env
With venv
, this workflow is like:
Using virtual environments will help you avoid many headaches for yourself and your colleagues produced by unknown errors related to incompatibility issues. If you like this post or you have some comments, contact me on Twitter! Go ahead and start using environments for your projects!