Tag Archives: Enthought Canopy

Enthought Announces Canopy 2.1: A Major Milestone Release for the Python Analysis Environment and Package Distribution

Python 3 and multi-environment support, new state of the art package dependency solver, and over 450 packages now available free for all users

Enthought Canopy logoEnthought is pleased to announce the release of Canopy 2.1, a significant feature release that includes Python 3 and multi-environment support, a new state of the art package dependency solver, and access to over 450 pre-built and tested scientific and analytic Python packages completely free for all users. We highly recommend that all current Canopy users upgrade to this new release.

Ready to dive in? Download Canopy 2.1 here.


For those currently familiar with Canopy, in this blog we’ll review the major new features in this exciting milestone release, and for those of you looking for a tool to improve your workflow with Python, or perhaps new to Python from a language like MATLAB or R, we’ll take you through the key reasons that scientists, engineers, data scientists, and analysts use Canopy to enable their work in Python.

First, let’s talk about the latest and greatest in Canopy 2.1!

  1. Support for Python 3 user environments: Canopy can now be installed with a Python 3.5 user environment. Users can benefit from all the Canopy features already available for Python 2.7 (syntax checking, debugging, etc.) in the new Python 3 environments. Python 3.6 is also available (and will be the standard Python 3 in Canopy 2.2).
  2. All 450+ Python 2 and Python 3 packages are now completely free for all users: Technical support, full installers with all packages for offline or shared installation, and the premium analysis environment features (graphical debugger and variable browser and Data Import Tool) remain subscriber-exclusive benefits. See subscription options here to take advantage of those benefits.
  3. Built in, state of the art dependency solver (EDM or Enthought Deployment Manager): the new EDM back end (which replaces the previous enpkg) provides additional features for robust package compatibility. EDM integrates a specialized dependency solver which automatically ensures you have a consistent package set after installation, removal, or upgrade of any packages.
  4. Environment bundles, which allow users to easily share environments directly with co-workers, or across various deployment solutions (such as the Enthought Deployment Server, continuous integration processes like Travis-CI and Appveyor, cloud solutions like AWS or Google Compute Engine, or deployment tools like Ansible or Docker). EDM environment bundles not only allow the user to replicate the set of installed dependencies but also support persistence for constraint modifiers, the list of manually installed packages, and the runtime version and implementation.
  5. Multi-environment support: with the addition of Python 3 environments and the new EDM back end, Canopy now also supports managing multiple Python environments from the user interface. You can easily switch between Python 2.7 and 3.5, or between multiple 2.7 or 3.5 environments. This is ideal especially for those migrating legacy code to Python 3, as it allows you to test as you transfer and also provides access to historical snapshots or libraries that aren’t yet available in Python 3.


Why Canopy is the Python platform of choice for scientists and engineers

Since 2001, Enthought has focused on making the scientific Python stack accessible and easy to use for both enterprises and individuals. For example, Enthought released the first scientific Python distribution in 2004, added robust and corporate support for NumPy on 64-bit Windows in 2011, and released Canopy 1.0 in 2013.

Since then, with its MATLAB-like experience, Canopy has enabled countless engineers, scientists and analysts to perform sophisticated analysis, build models, and create cutting-edge data science algorithms. Canopy’s all-in-one package distribution and analysis environment for Python has also been widely adopted in organizations who want to provide a single, unified platform that can be used by everyone from data analysts to software engineers.

Here are five of the top reasons that people choose Canopy as their tool for enabling data analysis, data modelling, and data visualization with Python:

Continue reading

Enthought Canopy 1.4 Released: Includes New Canopy-Configured Command Prompt

Enthought Canopy Product Page | Download Enthought Canopy

Enthought Canopy Update AvailableEnthought Canopy 1.4 is now available! Users can easily update to this latest version by clicking on the green “Update available” link at the bottom right of the Canopy intro screen window or by going to Help > Canopy Application Updates within the application.

Key additions in this release are a Canopy-configured command prompt, inclusion of new packages in the full installer utilized by IT groups and users running from disconnected networks, and continued stability upgrades. We’ve also updated or added over 50 supported packages in Canopy’s Package Manager on a continual basis since the v.1.3 release. See the full release notes and the full list of currently available Canopy packages.

New Canopy-Configured Command Prompt

Enthought Canopy Command PromptAn important usability feature added in Enthought Canopy 1.4 is a Canopy-configured command prompt available from the Canopy Editor window on all platforms via Tools > Command Prompt. When selected, this opens a Command Prompt (Windows) or Terminal (Linux, Mac OS) window pre-configured with the correct environment settings to use Canopy’s Python installation from the command line. This avoids having to modify your login environment variables. In particular, on Windows when using standard (ie, non-administrative) user accounts it can be difficult to override some system settings. Continue reading

Enthought Canopy 1.3 Released: Includes Move to Python 2.7.6

Enthought Canopy Product Page | Download Enthought Canopy

Enthought Canopy 1.3 is now available and users should see the update notification in the bottom right corner of the Canopy welcome screen (as shown in the image below). This is a fairly small update primarily focused on bug fixing and stability improvement. The biggest change is the move to Python 2.7.6 from 2.7.3.

Enthought Canopy Update Available Notification
The bottom right of the Enthought Canopy window notifies users to available updates

Python 2.7.6 rolls up a couple of minor updates to the core Python environment. The most important changes from our perspective are a number of security fixes required by some users as well as fixes for Mac OS “Mavericks.” Details can be found in the Python release notes, but in general the change should be transparent to most users. The only caveat is for users building Python eggs with native C or FORTRAN extensions and publishing those eggs to users who may still be running earlier versions of Canopy or Python 2.7.3 in general. In this case, it is safest to continue building against earlier versions of Canopy.

But isn’t updating Python versions painful you may ask? In the past, yes, updating to a new Python version often required a new Python install and then re-installing all of your custom packages. However, with Canopy’s auto-update mechanism, it’s simply a matter of clicking the “Update available” link and choosing “Install and relaunch” or “Install after quit.” Canopy will automatically update the core Python installation and restart without impacting your environment. Additionally, whether you are running Canopy 1.1, 1.1.1, or 1.2, Canopy will jump straight to 1.3 and get you all of the latest updates.

We encourage all users to update to Canopy 1.3 as the 1.2 and 1.3 versions include a large number of stability fixes as well as cleaning up a lot of other less serious, but still important aspects of the user experience. For those new to Canopy, you can get Canopy here.

Enthought Canopy makes Python updates convenient
Enthought Canopy makes updates convenient with automatic downloads that install without impacting user environments

Keep up with all of the latest news from Enthought on our social media channels:  Linked In | Twitter | Google+ | Facebook | YouTube

Enthought Canopy v1.2 is Out: PTVS, Mavericks, and Qt

Author: Jason McCampbell

Canopy 1.2 is out! The release of Mac OS “Mavericks” as a free update broke a few features, primarily IPython, so we held the release to try to make sure everything worked. That ended up taking longer than we wanted, but 1.2 is finally out and adds support for Mavericks. There is one Mavericks-specific, Qt font issue that we are working on correcting which causes the wrong system font to be selected so UI’s look less-nice than they should.

Enthought Canopy integrated into PTVS

Enthought Canopy integrated into PTVS

The biggest new feature is integration with Microsoft’s Python Tools for Visual Studio (PTVS) package. PTVS is a full, professional-grade development IDE for Python based on Visual Studio and provides mixed Python/C debugging. The ability to do mixed-mode debugging is a huge boon to software developers creating C (or FORTRAN) extensions to Python. Canopy v1.2 includes a custom DLL that allows us to integrate more completely with PTVS and solves some issues with auto-completion of Python standard library calls.

Beyond PTVS, we have added the Qt development tools, such as qmake and the UIC compiler, to the Canopy installation tree. These tools are available on all platforms now and enable Qt developers to access them from Canopy directly rather than having to build the tools themselves.

Canopy 1.2 includes a large number of smaller additions and stability improvements. Highlights can be found in the release notes and we encourage all users to update existing installs. As always, thanks for using Canopy and please don’t hesitate to drop us a note letting us know what you like or what you would like to see improved. You can contact us via the Help -> Suggestions/Feedback menu item or by sending email to canopy.support@enthought.com.

And you can download Canopy from the Enthought Store page.

Installing and Managing a Central Python Install with Enthought Canopy v1.1

Author: Jason McCampbell

In the last post we talked about virtual environments and how we have back-ported venv from Python 3 and extended it in Canopy 1.1. This post will now walk through how we use virtual environments to provide new options to organizations and workgroups who want to install Canopy on a multi-user network and how Canopy provides a flexible Python environment on large compute clusters without sacrificing performance.

Multi-user Network Installs

In a standard, single-user installation, Canopy creates two virtual environments, System and User. System is used for running the GUI itself and User is the main Python environment for running user code. The package set in User is completely under the user’s control (ie, won’t break the GUI).

With the 1.1 release, Canopy supports the creation of shared versions of the System and User virtual environments. These virtual environments, referred to as Common System and Common User, can be centrally managed, providing an easy means of managing a consistent set of package versions and dramatically reducing disk usage by having shared copies of the packages. Each individual user’s System and User virtual environment are layered on top of the common installs as shown below.

Canopy venv layout

In this case, Canopy Core and the two virtual environments “Common System” and “Common User” are installed in a central networked disk. Typically, all of the standard packages would be installed in “Common User”, making them available to all users. When each user first starts Canopy, the per-user virtual environments “User’s System” and “User’s User” are automatically created. Users have the freedom to install new packages and alternate package versions in their own virtual environments while still benefitting from the centrally managed package set.

To set up this structure, after installing Canopy, an administrator first runs Canopy and creates the System (“Common System”) and User (“Common User”) virtual environment in the desired location as one would in a single-user environment. Changes to the package set in User can be made by this administrative user. To make these environments available to all users, the following command is run, again as the administrative user:

canopy_cli –common-install

This writes a file named ‘location.cfg’ to Canopy Core. Now whenever a user starts Canopy, the per-user environments will be layered on top of the common environments.

The initial setup of the virtual environments, by default, uses the Canopy GUI, which is not always available or desired. To address these cases, Canopy now supports a new switch “–no-gui-setup’. See the Canopy Users Guide for more details.

Cluster Installs

Large compute clusters are an interesting special case of the multi-user network because a large number of nodes may be requiring the same resources at the same time. Starting a 1000-node job where a large number of files are required from a networked disk can increase startup time substantially, wasting precious time on an expensive cluster. Ideally, most or all of the files will be local to each node.

We can use a modified version of the multi-user setup above to address this. After installing Canopy on each node, we want to create the System and User virtual environments with all of the standard packages installed. Running the GUI to install to 1000+ machines is … inefficient… so we will use the non-GUI setup option (assuming Canopy is installed in /usr/local/Canopy on each machine):

ssh node1 /usr/local/Canopy/bin/canopy_cli –no-gui-setup –install-dir /usr/local/Canopy –common-install

Running this command once for each node in the cluster results in the virtual environments being installed to /usr/local/Canopy/Canopy_64bit on each machine. Large packages such as NumPy and SciPy can now be loaded from the local disk instead of being pulled over the network.

How do users add their own packages? When each user starts Canopy from the same or similar core install, Canopy will create the user-specific virtual environments layered on top of the ones in /usr/local/Canopy/Canopy_64bit. This gives us the structure shown in the diagram below where Canopy Core and the common virtual environments are local to each node (ie, fast I/O access) and the user environments are on a networked file system.

Canopy cluster install

It should be noted that while the Canopy GUI may be available on the cluster one would typically not use the GUI on the compute nodes. Instead, the “User’s User” virtual environment can be used like a standard Python distribution, such as EPD, to execute the Python application. But the big advantage to this structure over a plain Python installation is that we have the performance advantage of having most of the Python packages local to each node while also providing an easy means for users to customize their environments. Users can run the Canopy GUI on their desktop to prototype an application and then run the same application on the compute cluster using the same package set — no additional configuration needed.

For more, get Canopy v1.1 and try it out.

SciPy 2013 Cython Tutorial

Author: Kurt Smith

Thanks to everyone who attended the Cython tutorial at this year’s SciPy conference, and thanks to the conference organizers and tutorial chairs for ensuring everything ran smoothly. The enthusiasm of the students came through in their questions, and there were several good conversations after the tutorial throughout the week.

If you want the tutorial experience from the comfort of your couch, you can download the tutorial slides, exercises, and demos, and follow along with the videos. Please read the setup instructions on the tutorial webpage. The easiest way to satisfy the requirements for the tutorial is to download and install an existing scientific Python distribution, such as Enthought Canopy. You will need Cython version 0.19 or greater.

SciPy Talk

Tutorial Highlights

Cython is a language for adding static type information to Python with the objective of improving Python’s performance; it is a compiler (the cython command) for generating Python extension modules; and it helps wrap C and C++ libraries in a nice and Pythonic way with a minimum of overhead. These three aspects of Cython are tightly integrated with each other and shouldn’t be thought about in isolation: it is common to have Cython code that is intended to both speed up a pure-Python algorithm and that also calls out to C or C++ libraries.

Compared with some of the newer Python JIT compilers that are on the rise, Cython is relatively mature — not SWIG mature, but it certainly has been around long enough that it has grown features beyond its core functionality: annotated source files for compile-time performance profiling, runtime profiling that integrates with Python’s profilers, cross-language debugging capabilities, integration with IPython and the IPython notebook via the %%cython magic and other magic commands, the pyximport import hook support, Python 2 and Python 3 support, parallelization support via OpenMP, and others. I wanted this tutorial to touch on several of these extra capabilities that make Cython easier to use.

The tutorial was in the advanced track because I wanted to dive into newer and more advanced Cython features, especially typed memoryviews. Typed memoryviews are Cython’s interface to PEP-3118 buffers, the new buffer protocol for accessing and passing around (possibly strided) blobs of memory without copying. This is of considerable interest to scientific computing audiences for whom non-copying array operations are essential. NumPy arrays support this protocol, and are the primary object used with this protocol. Cython’s syntax for typed memoryviews is nice, and taking a slice of a typed memoryview yields another typed memoryview and, as you would expect, does not copy memory.

I was able to cover typed memoryviews towards the end of the tutorial, but didn’t have quite enough time to demonstrate their full power. Typed memoryviews are in every way superior to the existing numpy buffer support in Cython (the cdef np.ndarray[double, ndim=2] declarations) — they are faster, they are supported in function signatures for every kind of Cython function definition (def, cdef, and cpdef), they are easier to declare and use, and they do not have any external dependencies (i.e., you do not have to cimport anything to use them, and you do not have to add extra include flags when compiling).

Another advanced topic I touched on in the tutorial was wrapping templated C++ classes. Cython’s syntax for wrapping templated C++ is fairly easy to work with if you are wrapping just one template instantiation. Once you need to wrap several template instantiations, I recommend you use a code generation tool like cheetah or jinja2 to avoid manual code duplication. It is often helpful to provide a top-level wrapper class for a more Pythonic experience. Examples of this approach are in the tutorial material zip file. Cython’s fused types can alleviate the need for these workarounds, but can require some gymnastics of their own to use. The “real” solution to wrapping many instantiations of C++ templates is a templating system or macro system in Cython itself, which is hard to get right and is likely beyond the scope of the project.

I also wanted to demonstrate how to wrap modern Fortran: user derived types, assumed shape and assumed size arrays, module procedures, etc. You can accomplish this using the ISO_C_BINDING module that is part of the Fortran 2003 standard and supported by nearly every modern Fortran compiler. Alas, this had to be cut due to time constraints. I want to emphasize that it is possible to use Cython to provide very nice wrappers for modern Fortran, keeping Fortran relevant as a viable performance-oriented language that gains expressivity with Cython-generated Python wrappers.

Cython Webinar

I will be giving a Cython webinar to cover some of the topics that were skipped during the SciPy tutorial. I will likely cover typed memoryviews in more depth, and perhaps give more detail on getting Cython to work with modern Fortran and templated C++. If you have a subscription to Enthought Canopy, you will receive a notification for the webinar, so stay tuned. Or sign up for an Enthought account to get a notification.

“Why We Built Enthought Canopy, An Inside Look” Recorded Webinar

We posted a recording of a 30 minute webinar that we did on the 20th that covers what Canopy is and why we developed it. There’s a few minutes of Brett Murphy(Product Manager at Enthought) discussing the “why” with some slides, and then Jason McCampbell (Development Manager for Canopy) gets into the interesting part with a 15+ minute demo of some of the key capabilities and workflows in Canopy. If you would like to watch the recorded webinar, you can find it here (the different formats will play directly in different browsers so check them and you won’t have to download the whole recording first):

Summed up in one line: Canopy provides the minimal set of tools for non-programmers to access, analyze and visualize data in an open-source Python environment.

The challenge in the past for scientists, engineers and analysts who wanted to use Python had been pulling together a working, integrated Python environment for scientific computing. Finding compatible versions of the dozens of Python packages, compiling them and integrating it all was very time consuming. That’s why we released the Enthought Python Distribution (EPD) many years back. It provided a single install of all the major packages you needed to do scientific and analytic computing with Python.

But the primary interface for a user of EPD was the command line. For a scientist or analyst used to an environment like MATLAB or one of the R IDEs, the command line is a little unapproachable and makes Python challenging to adopt. This is why we developed Canopy.

Enthought Canopy is both a Python distribution (like EPD) and an analysis environment. The analysis environment includes an integrated editor and IPython prompt to faciliate script development & testing and data analysis & plotting. The graphical package manager becomes the main interface to the Python ecosystem with its package search, install and update capabilities. And the documentation browser makes online documentation for Canopy, Python and the popular Python packages available on the desktop.

Check out the Canopy demo in the recorded webinar (link above). We hope it’s helpful.

Avoiding “Excel Hell!” using a Python-based Toolchain

Update (Feb 6, 2014):  Enthought is now the exclusive distributor of PyXLL, a solution that helps users avoid “Excel Hell” by making it easy to develop add-ins for Excel in Python. Learn more here.

Didrik Pinte gave an informative, provocatively-titled presentation at the second, in-person New York Quantitative Python User’s Group (NY QPUG) meeting earlier this month.

There are a lot of examples in the press of Excel workflow mess-ups and spreadsheet errors contributing to some eye-popping mishaps in the finance world (e.g. JP Morgan’s spreadsheet issues may have led to the 2012 massive loss at “the London Whale”). Most of these can be traced to similar fundamental issues:

  • Data referencing/traceability

  • Numerical errors

  • Error-prone manual operations (cut & paste, …)

  • Tracing IO’s in libraries/API’s

  • Missing version control

  • Toolchain that doesn’t meet the needs of researchers, analysts, IT, etc.

Python, the coding language and its tool ecosystem, can provide a nice solution to these challenges, and many organizations are already turning to Python-based workflows in response. And with integration tools like PyXLL (to execute Python functions within Excel) and others, organizations can adopt Python-based workflows incrementally and start improving their current “Excel Hell” situation quickly.

For the details check out the video of Didrik’s NY QPUG presentation.  He demonstrates a an example solution using PyXLL and Enthought Canopy.

[vimeo 67327735 http://vimeo.com/67327735]

And grab the PDF of his slides here.

QPUG_20130514_ExcelHell_Slides

It would be great to hear your stories about “Excel Hell”. Let us know below.

–Brett Murphy