After searching and not finding, I must ask here:
How does conda env
work under the hood, meaning, how does anaconda handle environments?
To clarify, I would like an answer or a reference to questions like:
What is kept in the envs/myenv
folder?
What happens upon activate myenv
?
What happens upon conda install ...
?
Where can i find such information?
Conda envs
Basically, conda
environments replicate the structure of your system, meaning it will store /bin
, /lib
, /etc
, /var
, among other directories. This is more obvious for unix systems, but the same concept is true under windows (DLLs
, libs
, Scripts
, ...).
More details in the official documentation.
Conda install
The idea is that conda install PACKAGE
will fetch a precompiled package from a channel
(a conda packages repository), and install it under this system-like structure. Instead of relying on system dependencies, conda
will install all dependencies of this package under the environment structure, using only conda packages.
Thus installing the same package at a given time point under different systems should result in reliably identical installs.
This is a way to standardize binaries, and it is only achieved by precompiling every package against given versions of libraries, which are shipped as dependencies of the conda environment. For instance, conda-forge
and bioconda
channels rely on cloud-based CI/CD pipelines to compile all packages on identical and completely clean system images.
Conda also stores metadata about these packages (version, build number, dependencies, license,...) so it is able to solve pretty complex dependency trees and avoid packages/libraries incompatibilities. It is the Solving...
step each time you execute conda install
.
Conda activate
Then when you conda activate ENV
, conda prepends the environment root $CONDA_PREFIX/bin
to PATH
, so that all executables installed in the environment will be found by the system (and will overload system-wide install of the same executable).
You can imagine it like temporarily replacing the system executables with those from the environment.
More
This a very basic explanation, not 100% accurate, and certainly not complete. If you want to learn more, go read the documentation, experiment with conda
, and maybe have an in-depth look to how Conda-forge and Bioconda do build packages, as everything is hosted on github.