“venv” in Python 2.7 and how it Simplifies Life

Virtual environments, specifically ‘venv’ which we backported from Python 3.x, are a technology that enables the creation of multiple, lightweight, independent Python environments. Each virtual environment appears to be a self-contained Python installation, but loads the Python standard library and other common resources from a common base Python installation. Optionally, a virtual environment can also load packages from its base Python environment, whether that’s Canopy Core itself or another virtual environment.

What makes virtual environments so interesting? Well, they reduce disk space by not having to duplicate the full Python environment each time. But more than that, making Python environments far “lighter” enables several interesting capabilities.

First, the most common use of virtual environments is to allow separate projects to run in separate environments with different packages requirements. Each Python application runs in a separate virtual environment so package updates needed for one application don’t break the others. This model has long been used by web developers as well as a few scientific software developers.

The second case is specifically enabled by Canopy. Sharp-eyed readers will have noted in the first paragraph that we said that a virtual environment can have Canopy Core or another virtual environment as the base. But virtual environments can’t be layered, right? Now they can.

We have extended venv to support arbitrary numbers of layers, so we can do something like this:

'venv' in Canopy

‘venv’ in Canopy

‘Project1’ can be created with the following Canopy command:

canopy_cli  setup  ./Project1

Canopy constructs Project1 with all of the standard Canopy packages installed, and Project1 can now be customized to run the application. Once we’ve got Project1 working with a particular Python configuration, what if we want to see if the application works with the latest version of NumPy? We could update and potentially break the stable environment. Or, we can do this:

./Project1/bin/ venv  -s  ./Project1_play

Now ‘Project1_Play’ is a virtual environment which has by default all of Project1’s packages and package versions available. We can now update NumPy or other packages in Project1_play and test the application. If it doesn’t work, no big deal, we just delete it. We now have the ability to rapidly experiment with different (safe) Python environments without breaking our stable working area.

Canopy makes use of virtual environments to provide a protected Python environment for the Canopy GUI application to run in, and to provide one or more User Python environments which you can customize and run your own code in. Canopy Core is the base for each of these virtual environments, providing the core Python components and several common, large packages such as Qt and PySide. This structure means that the Canopy GUI can be updated without impacting your code, and any package updates you install won’t destabilize the Canopy GUI.

Canopy Core can be updated if you want, such as to move to a new version of Python, and each of the virtual environments will be updated automatically as well. This eliminates the need to install a new Python environment and then re-install any third-party packages into that new environment just to update Python.

For more information on how to set up virtual environments with Canopy, check the online docs, or get Canopy v1.1 and try it out.

Our next post will detail how to use Canopy and virtual environments to set up multi-user networks and cluster environments.

7 thoughts on ““venv” in Python 2.7 and how it Simplifies Life

  1. Pingback: Enthought: “venv” in Python 2.7 and how it Simplifies Life | The Black Velvet Room

    1. avatarJason McCampbell

      Virtualenv is a great package, but it’s separate from Python itself. As such, it needs to use messy tricks such as maintaining symbolic links to the Python standard library files. venv is basically virtualenv added to Python itself which removes these workarounds and makes it a cleaner solution. Venv also ends up being more extensible because of this. As a virtualenv user, venv should feel very familiar, though.

      Reply
  2. avatarJeff Weakley

    I’m new to Canopy but love that you’ve put this together in one place. I’m a little confused though… when I installed Canopy Express, is it automatically in it’s own virtual environment or do I have to install it in one???
    thanks

    Reply
    1. avatarbmurphy

      Hi Jeff, the first time you launch the Canopy application after installation, it sets up the User virtual environment for you. You can then start the Canopy Editor and that “IDE” will include an IPython console that uses that virtual environment. This blog post is only really needed for folks who want to configure their own Python environments in more complex ways, and from the command line. For those of us more used to something like MATLAB (including me) the Canopy desktop is plenty and more intuitive.

      Reply
  3. Pingback: Installing and Managing a Central Python Install with Enthought Canopy v1.1 | Enthought[s]

  4. avatarAdam

    If you’re planning on creating and releasing your own open source stuff in Python, venv is a must. I wrote up a (probably over – simplified) article on venv. See my website link.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

Please leave these two fields as-is:

Protected by Invisible Defender. Showed 403 to 102,945 bad guys.