Category Archives: Enthought Canopy

Enthought Canopy 1.4 Released: Includes New Canopy-Configured Command Prompt

Enthought Canopy Product Page | Download Enthought Canopy

Enthought Canopy Update AvailableEnthought Canopy 1.4 is now available! Users can easily update to this latest version by clicking on the green “Update available” link at the bottom right of the Canopy intro screen window or by going to Help > Canopy Application Updates within the application.

Key additions in this release are a Canopy-configured command prompt, inclusion of new packages in the full installer utilized by IT groups and users running from disconnected networks, and continued stability upgrades. We’ve also updated or added over 50 supported packages in Canopy’s Package Manager on a continual basis since the v.1.3 release. See the full release notes and the full list of currently available Canopy packages.

New Canopy-Configured Command Prompt

Enthought Canopy Command PromptAn important usability feature added in Enthought Canopy 1.4 is a Canopy-configured command prompt available from the Canopy Editor window on all platforms via Tools > Command Prompt. When selected, this opens a Command Prompt (Windows) or Terminal (Linux, Mac OS) window pre-configured with the correct environment settings to use Canopy’s Python installation from the command line. This avoids having to modify your login environment variables. In particular, on Windows when using standard (ie, non-administrative) user accounts it can be difficult to override some system settings. Continue reading

PyXLL: Deploy Python to Excel Easily

PyXLL Solution Home | Buy PyXLL | Press Release

Today Enthought announced that it is now the worldwide distributor for PyXLL, and we’re excited to offer this key product for deploying Python models, algorithms and code to Excel. Technical teams can use the full power of Enthought Canopy, or another Python distro, and end-users can access the results in their familiar Excel environment. And it’s straightforward to set up and use.

Installing PyXLL from Enthought Canopy

PyXLL is available as a package subscription (with significant discounts for multiple users). Once you’ve purchased a subscription you can easily install it via Canopy’s Package Manager as shown in the screenshots below (note that at this time PyXLL is only available for Windows users). The rest of the configuration instructions are in the Quick Start portion of the documentation. PyXLL itself is a plug-in to Excel. When you start Excel, PyXLL loads into Excel and reads in Python modules that you have created for PyXLL. This makes PyXLL especially useful for organizations that want to manage their code centrally and deploy to multiple Excel users.

Enthought Canopy Package Manager   Install PyXLL from Enthought Canopy's Package Manager

Creating Excel Functions with PyXLL

To create a PyXLL Python Excel function, you use the @xl_func decorator to tell PyXLL the following function should be registered with Excel, what its argument types are, and optionally what its return type is. PyXLL also reads the function’s docstring and provides that in the Excel function description. As an example, I created a module my_pyxll_module.py and registered it with PyXLL via the Continue reading

Enthought Canopy 1.3 Released: Includes Move to Python 2.7.6

Enthought Canopy Product Page | Download Enthought Canopy

Enthought Canopy 1.3 is now available and users should see the update notification in the bottom right corner of the Canopy welcome screen (as shown in the image below). This is a fairly small update primarily focused on bug fixing and stability improvement. The biggest change is the move to Python 2.7.6 from 2.7.3.

Enthought Canopy Update Available Notification
The bottom right of the Enthought Canopy window notifies users to available updates

Python 2.7.6 rolls up a couple of minor updates to the core Python environment. The most important changes from our perspective are a number of security fixes required by some users as well as fixes for Mac OS “Mavericks.” Details can be found in the Python release notes, but in general the change should be transparent to most users. The only caveat is for users building Python eggs with native C or FORTRAN extensions and publishing those eggs to users who may still be running earlier versions of Canopy or Python 2.7.3 in general. In this case, it is safest to continue building against earlier versions of Canopy.

But isn’t updating Python versions painful you may ask? In the past, yes, updating to a new Python version often required a new Python install and then re-installing all of your custom packages. However, with Canopy’s auto-update mechanism, it’s simply a matter of clicking the “Update available” link and choosing “Install and relaunch” or “Install after quit.” Canopy will automatically update the core Python installation and restart without impacting your environment. Additionally, whether you are running Canopy 1.1, 1.1.1, or 1.2, Canopy will jump straight to 1.3 and get you all of the latest updates.

We encourage all users to update to Canopy 1.3 as the 1.2 and 1.3 versions include a large number of stability fixes as well as cleaning up a lot of other less serious, but still important aspects of the user experience. For those new to Canopy, you can get Canopy here.

Enthought Canopy makes Python updates convenient
Enthought Canopy makes updates convenient with automatic downloads that install without impacting user environments

Keep up with all of the latest news from Enthought on our social media channels:  Linked In | Twitter | Google+ | Facebook | YouTube

Enthought Canopy v1.2 is Out: PTVS, Mavericks, and Qt

Author: Jason McCampbell

Canopy 1.2 is out! The release of Mac OS “Mavericks” as a free update broke a few features, primarily IPython, so we held the release to try to make sure everything worked. That ended up taking longer than we wanted, but 1.2 is finally out and adds support for Mavericks. There is one Mavericks-specific, Qt font issue that we are working on correcting which causes the wrong system font to be selected so UI’s look less-nice than they should.

Enthought Canopy integrated into PTVS

Enthought Canopy integrated into PTVS

The biggest new feature is integration with Microsoft’s Python Tools for Visual Studio (PTVS) package. PTVS is a full, professional-grade development IDE for Python based on Visual Studio and provides mixed Python/C debugging. The ability to do mixed-mode debugging is a huge boon to software developers creating C (or FORTRAN) extensions to Python. Canopy v1.2 includes a custom DLL that allows us to integrate more completely with PTVS and solves some issues with auto-completion of Python standard library calls.

Beyond PTVS, we have added the Qt development tools, such as qmake and the UIC compiler, to the Canopy installation tree. These tools are available on all platforms now and enable Qt developers to access them from Canopy directly rather than having to build the tools themselves.

Canopy 1.2 includes a large number of smaller additions and stability improvements. Highlights can be found in the release notes and we encourage all users to update existing installs. As always, thanks for using Canopy and please don’t hesitate to drop us a note letting us know what you like or what you would like to see improved. You can contact us via the Help -> Suggestions/Feedback menu item or by sending email to canopy.support@enthought.com.

And you can download Canopy from the Enthought Store page.

Python at Inflection Point in HPC

Authors: Kurt Smith, Robert Grant, and Lauren Johnson

We attended SuperComputing 2013, held November 17-22 in Denver, and saw huge interest around Python. There were several Python related events, including the “Python in HPC” tutorial (Monday), the Python BoF (Tuesday), and a “Python for HPC” workshop held in parallel with the tutorial on Monday. But we had some of our best conversations on the trade show floor.

Python Buzz on the Floor

The Enthought booth had a prominent “Python for HPC: High Productivity Computing” headline, and we looped videos of our parallelized 2D Julia set rendering GUI (video below).  The parallelization used Cython’s OpenMP functionality, came in at around 200 lines of code, and generated lots of discussions.  We also used a laptop to display an animated 3D Julia set rendered in Mayavi and to demo Canopy.

Many people came up to us after seeing our banner and video and asked “I use Python a little bit, but never in HPC – what can you tell me?”  We spoke with hundreds of people and had lots of good conversations.

It really seems like Python has reached an inflection point in HPC.

Python in HPC Tutorial, Monday

Kurt Smith presented a 1/4 day section on Cython, which was a shortened version of what he presented at SciPy 2013.  In addition, Andy Terrel presented “Introduction to Python”; Aron Ahmadia presented “Scaling Python with MPI”; and Travis Oliphant presented “Python and Big Data”. You can find all the material on the PyHPC.org website.

The tutorial was generally well attended: about 100–130 people.  A strong majority of attendees were already programming in Python, with about half using Python in a performance-critical area and perhaps 10% running Python on supercomputers or clusters directly.

In the Cython section of the tutorial, Kurt went into more detail on how to use OpenMP with Cython, which was of interest to many based on questions during the presentation. For the exercises, students were given temporary accounts on  Stampede (TACC’s latest state-of-the-art supercomputer) to help ensure everyone was able to get their exercise environment working.

Andy’s section of the day went well, covering the basics of using Python.  Aron’s section was good for establishing that Python+MPI4Py can scale to ~65,000 nodes on massive supercomputers, and also for adressing people’s concerns regarding the import challenge.

Python in HPC workshop, Monday

There was a day-long workshop of presentations on “Python in HPC” which ran in parallel with the “Python for HPC” tutorial. Of particular interest were the talks on “Doubling the performance of NumPy” and “Bohrium: Unmodified NumPy code on CPU, GPU, and Cluster“.

Python for High Performance and Scientific Computing BoF, Tuesday

Andy Terrel, William Scullin, and Andreas Schreiber organized a Birds-of-a-Feather session on Python, which had about 150 attendees (many thanks to all three for organizing a great session!).  Kurt gave a lightning talk on Enthought’s SBIR work.  The other talks focused on applications of Python in HPC settings, as well as IPython notebooks on the basics of the Navier-Stokes equations.

It was great to see so much interest in Python for HPC!

Enthought Tool Suite Release 4.4 (Traits, Chaco, and more)

Authors: The ETS Developers

We’re happy to announce the release of multiple major projects, including:

  • Traits 4.4.0
  • Chaco 4.4.1
  • TraitsUI 4.4.0
  • Envisage 4.4.0
  • Pyface 4.4.0
  • Codetools 4.2.0
  • ETS 4.4.1

These packages form the core of the Enthought Tool Suite (ETS, http://code.enthought.com/projects), a collection of free, open-source components developed by Enthought and our partners to construct custom scientific applications. ETS includes a wide variety of components, including:

  • an extensible application framework (Envisage)

  • application building blocks (Traits, TraitsUI, Enaml, Pyface, Codetools)

  • 2-D and 3-D graphics libraries (Chaco, Mayavi, Enable)

  • scientific and math libraries (Scimath)

  • developer tools (Apptools)

You can install any of the packages using Canopy‘s package manager, using the Canopy or EPD ‘enpkg \’ command, from PyPI (using pip or easy_install),  or by building them from source code on github. For more details, see the ETS intallation page.

Contributors

==================

This set of releases was an 8-month effort of Enthought developers along with:

  • Yves Delley
  • Pieter Aarnoutse
  • Jordan Ilott
  • Matthieu Dartiailh
  • Ian Delaney
  • Gregor Thalhammer

Many thanks to them!

General release notes

==================

  1. The major new feature in this Traits release is a new adaptation mechanism in the “traits.adaptation“ package.  The new mechanism is intended to replace the older traits.protocols package.  Code written against “traits.protocols“ will continue to work, although the “traits.protocols“ API has been deprecated, and a warning will be logged on first use of “traits.protocols“.  See the ‘Advanced Topics’ section of the user manual for more details.

  2. These new releases of TraitsUI, Envisage, Pyface and Codetools include an update to this new adaptation mechanism.

  3. All ETS projects are now on TravisCI, making it easier to contribute to them.

  4. As of this release, the only Python versions that are actively supported are 2.6 and 2.7. As we are moving to future-proof ETS over the coming months, more code that supported Python 2.5 will be removed.

  5. We will retire chaco-users@enthought.com since it is lightly used and are now recommending all users of Chaco to send questions, requests and comments to enthought-dev@enthought.com or to StackOverflow (tag “enthought” and possibly “chaco”).

More details about the release of each project are given below. Please see the CHANGES.txt file inside each project for full details of the changes.

Happy coding!

The ETS developers

Traits 4.4.0 release notes

=====================

The Traits library enhances Python by adding optional type-checking and an event notification system, making it an ideal platform for writing data-driven applications.  It forms the foundation of the Enthought Tool Suite.

In addition to the above-mentioned rework of the adaptation mechanism, the release also includes improved support for using Cython with `HasTraits` classes, some new helper utilities for writing unit tests for Traits events, and a variety of bug fixes, stability enhancements, and internal code improvements.

Chaco 4.4.0 release notes

=====================

Chaco is a Python package for building efficient, interactive and custom 2-D plots and visualizations. While Chaco generates attractive static plots, it works particularly well for interactive data visualization and exploration.

This release introduces many improvements and bug fixes, including fixes to the generation of image files from plots, improvements to the ArrayPlotData to change multiple arrays at a time, and improvements to multiple elements of the plots such as tick labels and text overlays.

TraitsUI 4.4.0 release notes

======================

The TraitsUI project contains a toolkit-independent GUI abstraction layer, which is used to support the “visualization” features of the Traits package. TraitsUI allows developers to write against the TraitsUI API (views, items, editors, etc.), and let TraitsUI and the selected toolkit and back-end take care of the details of displaying them.

In addition to the above-mentioned update to the new Traits 4.4.0 adaptation mechanism, there have also been a number of improvements to drag and drop support for the Qt backend and some modernization of the use of WxPython to support Wx 2.9.  This release also includes a number of bug-fixes and minor functionality enhancements.

Envisage 4.4.0 release notes

=======================

Envisage is a Python-based framework for building extensible applications, providing a standard mechanism for features to be added to an

application, whether by the original developer or by someone else.

In addition to the above-mentioned update to the new Traits 4.4.0 adaptation mechanism, this release also adds a new method to retrieve a service that is required by the application and provides documentation and test updates.

Pyface 4.4.0 release notes

======================

The pyface project provides a toolkit-independent library of Traits-aware widgets and GUI components, which are used to support the “visualization” features of Traits.

The biggest change in this release is support for the new adaptation mechanism in Traits 4.4.0. This release also includes Tasks support for Enaml 0.8 and a number of other minor changes, improvements and bug-fixes.

Codetools release notes

====================

The codetools project includes packages that simplify meta-programming and help the programmer separate data from code in Python. This library provides classes for performing dependency-analysis on blocks of Python code, and Traits-enhanced execution contexts that can be used as execution namespaces.

In addition to the above-mentioned update to the new Traits 4.4.0 adaptation mechanism, this release also includes a number of modernizations of the code base, including the consistent use of absolute imports, and a new execution manager for deferring events from Contexts.

Installing and Managing a Central Python Install with Enthought Canopy v1.1

Author: Jason McCampbell

In the last post we talked about virtual environments and how we have back-ported venv from Python 3 and extended it in Canopy 1.1. This post will now walk through how we use virtual environments to provide new options to organizations and workgroups who want to install Canopy on a multi-user network and how Canopy provides a flexible Python environment on large compute clusters without sacrificing performance.

Multi-user Network Installs

In a standard, single-user installation, Canopy creates two virtual environments, System and User. System is used for running the GUI itself and User is the main Python environment for running user code. The package set in User is completely under the user’s control (ie, won’t break the GUI).

With the 1.1 release, Canopy supports the creation of shared versions of the System and User virtual environments. These virtual environments, referred to as Common System and Common User, can be centrally managed, providing an easy means of managing a consistent set of package versions and dramatically reducing disk usage by having shared copies of the packages. Each individual user’s System and User virtual environment are layered on top of the common installs as shown below.

Canopy venv layout

In this case, Canopy Core and the two virtual environments “Common System” and “Common User” are installed in a central networked disk. Typically, all of the standard packages would be installed in “Common User”, making them available to all users. When each user first starts Canopy, the per-user virtual environments “User’s System” and “User’s User” are automatically created. Users have the freedom to install new packages and alternate package versions in their own virtual environments while still benefitting from the centrally managed package set.

To set up this structure, after installing Canopy, an administrator first runs Canopy and creates the System (“Common System”) and User (“Common User”) virtual environment in the desired location as one would in a single-user environment. Changes to the package set in User can be made by this administrative user. To make these environments available to all users, the following command is run, again as the administrative user:

canopy_cli –common-install

This writes a file named ‘location.cfg’ to Canopy Core. Now whenever a user starts Canopy, the per-user environments will be layered on top of the common environments.

The initial setup of the virtual environments, by default, uses the Canopy GUI, which is not always available or desired. To address these cases, Canopy now supports a new switch “–no-gui-setup’. See the Canopy Users Guide for more details.

Cluster Installs

Large compute clusters are an interesting special case of the multi-user network because a large number of nodes may be requiring the same resources at the same time. Starting a 1000-node job where a large number of files are required from a networked disk can increase startup time substantially, wasting precious time on an expensive cluster. Ideally, most or all of the files will be local to each node.

We can use a modified version of the multi-user setup above to address this. After installing Canopy on each node, we want to create the System and User virtual environments with all of the standard packages installed. Running the GUI to install to 1000+ machines is … inefficient… so we will use the non-GUI setup option (assuming Canopy is installed in /usr/local/Canopy on each machine):

ssh node1 /usr/local/Canopy/bin/canopy_cli –no-gui-setup –install-dir /usr/local/Canopy –common-install

Running this command once for each node in the cluster results in the virtual environments being installed to /usr/local/Canopy/Canopy_64bit on each machine. Large packages such as NumPy and SciPy can now be loaded from the local disk instead of being pulled over the network.

How do users add their own packages? When each user starts Canopy from the same or similar core install, Canopy will create the user-specific virtual environments layered on top of the ones in /usr/local/Canopy/Canopy_64bit. This gives us the structure shown in the diagram below where Canopy Core and the common virtual environments are local to each node (ie, fast I/O access) and the user environments are on a networked file system.

Canopy cluster install

It should be noted that while the Canopy GUI may be available on the cluster one would typically not use the GUI on the compute nodes. Instead, the “User’s User” virtual environment can be used like a standard Python distribution, such as EPD, to execute the Python application. But the big advantage to this structure over a plain Python installation is that we have the performance advantage of having most of the Python packages local to each node while also providing an easy means for users to customize their environments. Users can run the Canopy GUI on their desktop to prototype an application and then run the same application on the compute cluster using the same package set — no additional configuration needed.

For more, get Canopy v1.1 and try it out.

“venv” in Python 2.7 and how it Simplifies Life

Virtual environments, specifically ‘venv’ which we backported from Python 3.x, are a technology that enables the creation of multiple, lightweight, independent Python environments. Each virtual environment appears to be a self-contained Python installation, but loads the Python standard library and other common resources from a common base Python installation. Optionally, a virtual environment can also load packages from its base Python environment, whether that’s Canopy Core itself or another virtual environment.

What makes virtual environments so interesting? Well, they reduce disk space by not having to duplicate the full Python environment each time. But more than that, making Python environments far “lighter” enables several interesting capabilities.

First, the most common use of virtual environments is to allow separate projects to run in separate environments with different packages requirements. Each Python application runs in a separate virtual environment so package updates needed for one application don’t break the others. This model has long been used by web developers as well as a few scientific software developers.

The second case is specifically enabled by Canopy. Sharp-eyed readers will have noted in the first paragraph that we said that a virtual environment can have Canopy Core or another virtual environment as the base. But virtual environments can’t be layered, right? Now they can.

We have extended venv to support arbitrary numbers of layers, so we can do something like this:

'venv' in Canopy

‘venv’ in Canopy

‘Project1’ can be created with the following Canopy command:

canopy_cli  setup  ./Project1

Canopy constructs Project1 with all of the standard Canopy packages installed, and Project1 can now be customized to run the application. Once we’ve got Project1 working with a particular Python configuration, what if we want to see if the application works with the latest version of NumPy? We could update and potentially break the stable environment. Or, we can do this:

./Project1/bin/ venv  -s  ./Project1_play

Now ‘Project1_Play’ is a virtual environment which has by default all of Project1’s packages and package versions available. We can now update NumPy or other packages in Project1_play and test the application. If it doesn’t work, no big deal, we just delete it. We now have the ability to rapidly experiment with different (safe) Python environments without breaking our stable working area.

Canopy makes use of virtual environments to provide a protected Python environment for the Canopy GUI application to run in, and to provide one or more User Python environments which you can customize and run your own code in. Canopy Core is the base for each of these virtual environments, providing the core Python components and several common, large packages such as Qt and PySide. This structure means that the Canopy GUI can be updated without impacting your code, and any package updates you install won’t destabilize the Canopy GUI.

Canopy Core can be updated if you want, such as to move to a new version of Python, and each of the virtual environments will be updated automatically as well. This eliminates the need to install a new Python environment and then re-install any third-party packages into that new environment just to update Python.

For more information on how to set up virtual environments with Canopy, check the online docs, or get Canopy v1.1 and try it out.

Our next post will detail how to use Canopy and virtual environments to set up multi-user networks and cluster environments.