Packages and libraries

How es-python finds your installed Python packages, how to install new ones, and which copy of a library your program imports when es-python bundles its own.

Your packages are shared

es-python uses the packages you already installed for your regular Python, so there is no separate package manager to learn and no second copy of every dependency. Both interpreters must be the same version (for example both 3.12) for this to apply.

pip install requests               # your normal pip
es-python -c "import requests"     # already importable

es-python reads your regular Python's site-packages directories, which the installer recorded on the machine.

Installing packages

You can install with either interpreter; both see the result. es-python -m pip defaults to a per-user install, so packages land in ~/.local, which your regular Python of the same version reads too.

es-python -m pip install requests

Your regular pip and python are not changed. The per-user default is skipped inside a virtual environment, when you run es-python with -s, and when you already chose a destination with the PIP_USER, PIP_TARGET or PIP_PREFIX environment variables.

Virtual environments

A virtual environment is self-contained by design, so inside one es-python does not share your regular packages; it uses the environment's own. Create the environment with es-python so that python inside it is es-python:

es-python -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt

Inside a virtual environment, your environment's own copy of a bundled library may be the one that imports. es-python tells you when that happens; see Coverage gaps.

Bundled libraries

es-python comes with its own copies of some popular third-party libraries, and watches the risky operations most of them perform. For those watched libraries, es-python puts its copy ahead of yours, so your program imports the watched copy even after you pip install the same library yourself.

This matters. es-python sees the query text your program sends to the database through the bundled database drivers; if your own copy of a driver imports instead, es-python does not watch that library, even with every source switched on.

The bundled libraries include the common database drivers (psycopg2, psycopg 3, mysqlclient, mariadb, oracledb), python-ldap, and a number of parsers and network libraries (for example uvloop, PyYAML and msgpack). On Python 3.9, psycopg 3 and mysqlclient are not included. They behave like the regular releases of the same libraries.

See which copy wins

To see, on a particular machine, every bundled library, whether es-python watches it, the version included, and whether es-python's copy or yours is the one that imports, run:

es-python -m _eyalsec_bridge

It ends with a warning that names any watched library that is not being watched here, or with a line saying none are disabled.

One side effect to know about: pip show psycopg2 can report your own copy in ~/.local, while import psycopg2 loads the bundled one. The import is what counts.

Prefer your own copy of one library

If you need your own version of a specific library (for example, because of a version conflict), name it in ES2_PREFER_APP_LIBS. Your copy of those libraries wins; every other bundled library is unchanged. Use a pip name (pyyaml) or an import name (yaml), separated by colons.

ES2_PREFER_APP_LIBS=psycopg2 es-python app.py
ES2_PREFER_APP_LIBS=pyyaml:msgpack es-python app.py

If a library you name is one es-python watches, it is not watched for that run.

Prefer your own copy of every library

ES2_NO_BUNDLED_LIBS=1 makes your copy of every bundled library win. es-python then watches none of the bundled libraries, so reach for the per-library ES2_PREFER_APP_LIBS first: one version conflict is a reason to prefer your copy of that library, not to switch off the rest.

ES2_NO_BUNDLED_LIBS=1 es-python app.py

Coverage gaps

Whenever your own copy of a watched library is the one that imports, the machine tells EyalSec, whether that came from ES2_PREFER_APP_LIBS, ES2_NO_BUNDLED_LIBS, a virtual environment, PYTHONPATH, or pip install --target. That way a library that is not being watched never looks like one that found nothing.

Libraries es-python does not watch are handed over silently, because nothing is lost. cryptography is one of them, and your copy of it wins by default, so a security update you install is never replaced by an older bundled one.

Turn package sharing off

Set ES2_NO_SYSTEM_SITE=1 to run es-python without your regular Python's installed packages. It then stops reading your regular Python's site-packages directories. Your per-user packages in ~/.local are still seen, as with any Python; add the standard -s option to leave those out too.

ES2_NO_SYSTEM_SITE=1 es-python app.py

Use this when a package compiled for your regular Python misbehaves under es-python. Inside a virtual environment the sharing is already off.