Category Archives: Enthought Python Distribution

Enthought Canopy v1.2 is Out: PTVS, Mavericks, and Qt

Author: Jason McCampbell

Canopy 1.2 is out! The release of Mac OS “Mavericks” as a free update broke a few features, primarily IPython, so we held the release to try to make sure everything worked. That ended up taking longer than we wanted, but 1.2 is finally out and adds support for Mavericks. There is one Mavericks-specific, Qt font issue that we are working on correcting which causes the wrong system font to be selected so UI’s look less-nice than they should.

Enthought Canopy integrated into PTVS

Enthought Canopy integrated into PTVS

The biggest new feature is integration with Microsoft’s Python Tools for Visual Studio (PTVS) package. PTVS is a full, professional-grade development IDE for Python based on Visual Studio and provides mixed Python/C debugging. The ability to do mixed-mode debugging is a huge boon to software developers creating C (or FORTRAN) extensions to Python. Canopy v1.2 includes a custom DLL that allows us to integrate more completely with PTVS and solves some issues with auto-completion of Python standard library calls.

Beyond PTVS, we have added the Qt development tools, such as qmake and the UIC compiler, to the Canopy installation tree. These tools are available on all platforms now and enable Qt developers to access them from Canopy directly rather than having to build the tools themselves.

Canopy 1.2 includes a large number of smaller additions and stability improvements. Highlights can be found in the release notes and we encourage all users to update existing installs. As always, thanks for using Canopy and please don’t hesitate to drop us a note letting us know what you like or what you would like to see improved. You can contact us via the Help -> Suggestions/Feedback menu item or by sending email to canopy.support@enthought.com.

And you can download Canopy from the Enthought Store page.

Enthought Tool Suite Release 4.4 (Traits, Chaco, and more)

Authors: The ETS Developers

We’re happy to announce the release of multiple major projects, including:

  • Traits 4.4.0
  • Chaco 4.4.1
  • TraitsUI 4.4.0
  • Envisage 4.4.0
  • Pyface 4.4.0
  • Codetools 4.2.0
  • ETS 4.4.1

These packages form the core of the Enthought Tool Suite (ETS, http://code.enthought.com/projects), a collection of free, open-source components developed by Enthought and our partners to construct custom scientific applications. ETS includes a wide variety of components, including:

  • an extensible application framework (Envisage)

  • application building blocks (Traits, TraitsUI, Enaml, Pyface, Codetools)

  • 2-D and 3-D graphics libraries (Chaco, Mayavi, Enable)

  • scientific and math libraries (Scimath)

  • developer tools (Apptools)

You can install any of the packages using Canopy‘s package manager, using the Canopy or EPD ‘enpkg \’ command, from PyPI (using pip or easy_install),  or by building them from source code on github. For more details, see the ETS intallation page.

Contributors

==================

This set of releases was an 8-month effort of Enthought developers along with:

  • Yves Delley
  • Pieter Aarnoutse
  • Jordan Ilott
  • Matthieu Dartiailh
  • Ian Delaney
  • Gregor Thalhammer

Many thanks to them!

General release notes

==================

  1. The major new feature in this Traits release is a new adaptation mechanism in the “traits.adaptation“ package.  The new mechanism is intended to replace the older traits.protocols package.  Code written against “traits.protocols“ will continue to work, although the “traits.protocols“ API has been deprecated, and a warning will be logged on first use of “traits.protocols“.  See the ‘Advanced Topics’ section of the user manual for more details.

  2. These new releases of TraitsUI, Envisage, Pyface and Codetools include an update to this new adaptation mechanism.

  3. All ETS projects are now on TravisCI, making it easier to contribute to them.

  4. As of this release, the only Python versions that are actively supported are 2.6 and 2.7. As we are moving to future-proof ETS over the coming months, more code that supported Python 2.5 will be removed.

  5. We will retire chaco-users@enthought.com since it is lightly used and are now recommending all users of Chaco to send questions, requests and comments to enthought-dev@enthought.com or to StackOverflow (tag “enthought” and possibly “chaco”).

More details about the release of each project are given below. Please see the CHANGES.txt file inside each project for full details of the changes.

Happy coding!

The ETS developers

Traits 4.4.0 release notes

=====================

The Traits library enhances Python by adding optional type-checking and an event notification system, making it an ideal platform for writing data-driven applications.  It forms the foundation of the Enthought Tool Suite.

In addition to the above-mentioned rework of the adaptation mechanism, the release also includes improved support for using Cython with `HasTraits` classes, some new helper utilities for writing unit tests for Traits events, and a variety of bug fixes, stability enhancements, and internal code improvements.

Chaco 4.4.0 release notes

=====================

Chaco is a Python package for building efficient, interactive and custom 2-D plots and visualizations. While Chaco generates attractive static plots, it works particularly well for interactive data visualization and exploration.

This release introduces many improvements and bug fixes, including fixes to the generation of image files from plots, improvements to the ArrayPlotData to change multiple arrays at a time, and improvements to multiple elements of the plots such as tick labels and text overlays.

TraitsUI 4.4.0 release notes

======================

The TraitsUI project contains a toolkit-independent GUI abstraction layer, which is used to support the “visualization” features of the Traits package. TraitsUI allows developers to write against the TraitsUI API (views, items, editors, etc.), and let TraitsUI and the selected toolkit and back-end take care of the details of displaying them.

In addition to the above-mentioned update to the new Traits 4.4.0 adaptation mechanism, there have also been a number of improvements to drag and drop support for the Qt backend and some modernization of the use of WxPython to support Wx 2.9.  This release also includes a number of bug-fixes and minor functionality enhancements.

Envisage 4.4.0 release notes

=======================

Envisage is a Python-based framework for building extensible applications, providing a standard mechanism for features to be added to an

application, whether by the original developer or by someone else.

In addition to the above-mentioned update to the new Traits 4.4.0 adaptation mechanism, this release also adds a new method to retrieve a service that is required by the application and provides documentation and test updates.

Pyface 4.4.0 release notes

======================

The pyface project provides a toolkit-independent library of Traits-aware widgets and GUI components, which are used to support the “visualization” features of Traits.

The biggest change in this release is support for the new adaptation mechanism in Traits 4.4.0. This release also includes Tasks support for Enaml 0.8 and a number of other minor changes, improvements and bug-fixes.

Codetools release notes

====================

The codetools project includes packages that simplify meta-programming and help the programmer separate data from code in Python. This library provides classes for performing dependency-analysis on blocks of Python code, and Traits-enhanced execution contexts that can be used as execution namespaces.

In addition to the above-mentioned update to the new Traits 4.4.0 adaptation mechanism, this release also includes a number of modernizations of the code base, including the consistent use of absolute imports, and a new execution manager for deferring events from Contexts.

PyQL and QuantLib: A Comprehensive Finance Framework

Authors: Kelsey Jordahl, Brett Murphy

Earlier this month at the first New York Finance Python User’s Group (NY FPUG) meetup, Kelsey Jordahl talked about how PyQL streamlines the development of Python-based finance applications using QuantLib. There were about 30 people attending the talk at the Cornell Club in New York City. We have a recording of the presentation below.

FPUG Meetup Presentation Screenshot

QuantLib is a free, open-source (BSD-licensed) quantitative finance package. It provides tools for financial instruments, yield curves, pricing engines, creating simulations, and date / time management. There is a lot more detail on the QuantLib website along with the latest downloads. Kelsey refers to a really useful blog / open-source book by one of the core QuantLib developers on implementing QuantLib. Quantlib also comes with different language bindings, including Python.

So why use PyQL if there are already Python bindings in QuantLib? Well, PyQL provides a much more Pythonic set of APIs, in short. Kelsey discusses some of the differences between the original QuantLib Python API and the PyQL API and how PyQL streamlines the resulting Python code. You get better integration with other packages like NumPy, better namespace usage and better documentation. PyQL is available up on GitHub in the PyQL repo. Kelsey uses the IPython Notebooks in the examples directory to explore PyQL and QuantLib and compares the use of PyQL versus the standard (SWIG) QuantLib Python APIs.

PyQL remains a work in progress, with goals to make its QuantLib coverage more complete, the API even more Pythonic, and getting a successful build on Windows (works on Mac OS and Linux now). It’s open source, so feel free to step up and contribute!

For the details, check out the video of Kelsey’s presentation (44 minutes).

And here are the slides online if you want to check the links in the presentation.

If you are interested in working on either QuantLib or PyQL, let the maintainers know!

Installing and Managing a Central Python Install with Enthought Canopy v1.1

Author: Jason McCampbell

In the last post we talked about virtual environments and how we have back-ported venv from Python 3 and extended it in Canopy 1.1. This post will now walk through how we use virtual environments to provide new options to organizations and workgroups who want to install Canopy on a multi-user network and how Canopy provides a flexible Python environment on large compute clusters without sacrificing performance.

Multi-user Network Installs

In a standard, single-user installation, Canopy creates two virtual environments, System and User. System is used for running the GUI itself and User is the main Python environment for running user code. The package set in User is completely under the user’s control (ie, won’t break the GUI).

With the 1.1 release, Canopy supports the creation of shared versions of the System and User virtual environments. These virtual environments, referred to as Common System and Common User, can be centrally managed, providing an easy means of managing a consistent set of package versions and dramatically reducing disk usage by having shared copies of the packages. Each individual user’s System and User virtual environment are layered on top of the common installs as shown below.

Canopy venv layout

In this case, Canopy Core and the two virtual environments “Common System” and “Common User” are installed in a central networked disk. Typically, all of the standard packages would be installed in “Common User”, making them available to all users. When each user first starts Canopy, the per-user virtual environments “User’s System” and “User’s User” are automatically created. Users have the freedom to install new packages and alternate package versions in their own virtual environments while still benefitting from the centrally managed package set.

To set up this structure, after installing Canopy, an administrator first runs Canopy and creates the System (“Common System”) and User (“Common User”) virtual environment in the desired location as one would in a single-user environment. Changes to the package set in User can be made by this administrative user. To make these environments available to all users, the following command is run, again as the administrative user:

canopy_cli –common-install

This writes a file named ‘location.cfg’ to Canopy Core. Now whenever a user starts Canopy, the per-user environments will be layered on top of the common environments.

The initial setup of the virtual environments, by default, uses the Canopy GUI, which is not always available or desired. To address these cases, Canopy now supports a new switch “–no-gui-setup’. See the Canopy Users Guide for more details.

Cluster Installs

Large compute clusters are an interesting special case of the multi-user network because a large number of nodes may be requiring the same resources at the same time. Starting a 1000-node job where a large number of files are required from a networked disk can increase startup time substantially, wasting precious time on an expensive cluster. Ideally, most or all of the files will be local to each node.

We can use a modified version of the multi-user setup above to address this. After installing Canopy on each node, we want to create the System and User virtual environments with all of the standard packages installed. Running the GUI to install to 1000+ machines is … inefficient… so we will use the non-GUI setup option (assuming Canopy is installed in /usr/local/Canopy on each machine):

ssh node1 /usr/local/Canopy/bin/canopy_cli –no-gui-setup –install-dir /usr/local/Canopy –common-install

Running this command once for each node in the cluster results in the virtual environments being installed to /usr/local/Canopy/Canopy_64bit on each machine. Large packages such as NumPy and SciPy can now be loaded from the local disk instead of being pulled over the network.

How do users add their own packages? When each user starts Canopy from the same or similar core install, Canopy will create the user-specific virtual environments layered on top of the ones in /usr/local/Canopy/Canopy_64bit. This gives us the structure shown in the diagram below where Canopy Core and the common virtual environments are local to each node (ie, fast I/O access) and the user environments are on a networked file system.

Canopy cluster install

It should be noted that while the Canopy GUI may be available on the cluster one would typically not use the GUI on the compute nodes. Instead, the “User’s User” virtual environment can be used like a standard Python distribution, such as EPD, to execute the Python application. But the big advantage to this structure over a plain Python installation is that we have the performance advantage of having most of the Python packages local to each node while also providing an easy means for users to customize their environments. Users can run the Canopy GUI on their desktop to prototype an application and then run the same application on the compute cluster using the same package set — no additional configuration needed.

For more, get Canopy v1.1 and try it out.

For all you EPD Users: Canopy v1.1

EPD (Enthought Python Distribution) provided a simple install of Python for scientific computing on the major platforms: Windows, Linux and Mac-OS. Those looking for a clean, straightforward Python stack to unpack into a particular directory found EPD to be pretty ideal.

With the introduction of Enthought Canopy, we began addressing users who are more engineer or scientist than programmer and were much less familiar with command-line interfaces. The Canopy desktop (in the vein of MATLAB or Spyder) aims at these technical users who want to use Python, but more as an application or IDE. To implement the desktop in Python and to allow both it and a user-defined Python environment to co-exist and be separately updated, we used virtual environments. As a consequence Canopy can feel a bit foreign to EPD users. With 1.1 we have added a new command line interface (CLI) that will hopefully make EPD users feel more at home in Canopy while retaining many of the Canopy advantages such as in-place update and virtual environment support.

Now, EPD users who just want to use Canopy as a plain Python environment with their own tools or IDE can easily create one or more Python environments. For example, from the command line on Windows:

        Canopy_cli.exe setup C:\Python27

or on Linux:

        canopy_cli setup ~/canopy

The target directory can be any you choose. If you want to make this Python environment the default on your system, you can specify the –default switch, and Canopy will add the appropriate bin directory (Scripts directory on Windows) to your PATH environment variable. On Mac OS and Linux systems, Canopy does this by appending a line to your ~/.bash_profile file which activates the correct virtual environment. On Windows, this Python environment is also added to the system registry so third-party tools can correctly find it.

Since we use virtual environments, the installation layout for Canopy is different. With Canopy we install what is referred to as “Canopy Core”: the core Python environment and a minimum set of packages needed to bootstrap Canopy itself. With it we can lock down the Canopy environment, facilitate the automatic update mechanism, and provide reliable startup and fail-safe recovery. For the user, there is a different environment. This means when a Python update comes out, it is no longer necessary to install a whole new environment plus all of your packages and get everything working again. Instead, simply update Canopy and go back to working — all of your packages are still installed but Python has been upgraded.

To complete an install, Canopy creates two virtual environments named ‘System’ and ‘User’. System is where the Canopy GUI runs; no user code runs in this environment. Updates to this virtual environment are done via the Canopy update mechanisms. The User environment is where the kernel and all user code runs. This virtual environment is managed by Package Manager from the desktop or by enpkg from the command line; any packages can be updated and installed without fear of disrupting the GUI. Similarly, updates to the Canopy GUI will not affect packages installed in the User environment and break your code.

So why stick with virtual environments for an “EPD-like” install? One of the big challenges with the old, “flat” EPD installation method was updating an install, or trying out different package configurations. With virtual environments, you can create a new environment which inherits packages from another virtual environment, and try out a few package changes. When you are satisfied, it’s straightforward to throw away the experimentation area and make the changes to the original, stable virtual environment.

For more details, check out Creating an EPD-like Python environment in our online docs. And you can download Canopy v1.1 now.

“Why We Built Enthought Canopy, An Inside Look” Recorded Webinar

We posted a recording of a 30 minute webinar that we did on the 20th that covers what Canopy is and why we developed it. There’s a few minutes of Brett Murphy(Product Manager at Enthought) discussing the “why” with some slides, and then Jason McCampbell (Development Manager for Canopy) gets into the interesting part with a 15+ minute demo of some of the key capabilities and workflows in Canopy. If you would like to watch the recorded webinar, you can find it here (the different formats will play directly in different browsers so check them and you won’t have to download the whole recording first):

Summed up in one line: Canopy provides the minimal set of tools for non-programmers to access, analyze and visualize data in an open-source Python environment.

The challenge in the past for scientists, engineers and analysts who wanted to use Python had been pulling together a working, integrated Python environment for scientific computing. Finding compatible versions of the dozens of Python packages, compiling them and integrating it all was very time consuming. That’s why we released the Enthought Python Distribution (EPD) many years back. It provided a single install of all the major packages you needed to do scientific and analytic computing with Python.

But the primary interface for a user of EPD was the command line. For a scientist or analyst used to an environment like MATLAB or one of the R IDEs, the command line is a little unapproachable and makes Python challenging to adopt. This is why we developed Canopy.

Enthought Canopy is both a Python distribution (like EPD) and an analysis environment. The analysis environment includes an integrated editor and IPython prompt to faciliate script development & testing and data analysis & plotting. The graphical package manager becomes the main interface to the Python ecosystem with its package search, install and update capabilities. And the documentation browser makes online documentation for Canopy, Python and the popular Python packages available on the desktop.

Check out the Canopy demo in the recorded webinar (link above). We hope it’s helpful.

Welcome EdX students!

Welcome to EPDFree EdX students! We are honored to partner with EdX to offer up a Python environment you can use for your studies. For those of you who don’t know, “EdX is a not-for-profit enterprise of its founding partners Harvard University and the Massachusetts Institute of Technology that features learning designed specifically for interactive study via the web.”

We’ve always had a strong relationship with the academic community and are excited about accelerating the speed of science with our tools and consulting work. It’s exciting to know we will be a part of this ambitious initiative to train the next generation of scientists around the world. Good luck on those problem sets!

DataGotham…Complete!

Well, DataGotham is over. The conference featured a wide cross section of the data community in NYC. Talks spanned topics from “urban science” to “finding racism on FourSquare” to “creating an API for spaces.” Don’t worry, the videos will be online soon so you can investigate yourself. The organizers did a great job putting a conference of this size together on relatively short notice. Bravo NYC data crunchers!

One thing I somehow missed was a network graph created by the organizers to illustrate the tools used by attendees. I am happy to see python leading the way! The thickness of the edge indicates the number of people using both tools. It seems there are a lot of people trying to make Python and R “two great tastes that go great together.” I’m curious as to why more Python users aren’t using numpy and scipy. Food for thought…

Got tools?