Category Archives: Mayavi2

Python at Inflection Point in HPC

Authors: Kurt Smith, Robert Grant, and Lauren Johnson

We attended SuperComputing 2013, held November 17-22 in Denver, and saw huge interest around Python. There were several Python related events, including the “Python in HPC” tutorial (Monday), the Python BoF (Tuesday), and a “Python for HPC” workshop held in parallel with the tutorial on Monday. But we had some of our best conversations on the trade show floor.

Python Buzz on the Floor

The Enthought booth had a prominent “Python for HPC: High Productivity Computing” headline, and we looped videos of our parallelized 2D Julia set rendering GUI (video below).  The parallelization used Cython’s OpenMP functionality, came in at around 200 lines of code, and generated lots of discussions.  We also used a laptop to display an animated 3D Julia set rendered in Mayavi and to demo Canopy.

Many people came up to us after seeing our banner and video and asked “I use Python a little bit, but never in HPC – what can you tell me?”  We spoke with hundreds of people and had lots of good conversations.

It really seems like Python has reached an inflection point in HPC.

Python in HPC Tutorial, Monday

Kurt Smith presented a 1/4 day section on Cython, which was a shortened version of what he presented at SciPy 2013.  In addition, Andy Terrel presented “Introduction to Python”; Aron Ahmadia presented “Scaling Python with MPI”; and Travis Oliphant presented “Python and Big Data”. You can find all the material on the PyHPC.org website.

The tutorial was generally well attended: about 100–130 people.  A strong majority of attendees were already programming in Python, with about half using Python in a performance-critical area and perhaps 10% running Python on supercomputers or clusters directly.

In the Cython section of the tutorial, Kurt went into more detail on how to use OpenMP with Cython, which was of interest to many based on questions during the presentation. For the exercises, students were given temporary accounts on  Stampede (TACC’s latest state-of-the-art supercomputer) to help ensure everyone was able to get their exercise environment working.

Andy’s section of the day went well, covering the basics of using Python.  Aron’s section was good for establishing that Python+MPI4Py can scale to ~65,000 nodes on massive supercomputers, and also for adressing people’s concerns regarding the import challenge.

Python in HPC workshop, Monday

There was a day-long workshop of presentations on “Python in HPC” which ran in parallel with the “Python for HPC” tutorial. Of particular interest were the talks on “Doubling the performance of NumPy” and “Bohrium: Unmodified NumPy code on CPU, GPU, and Cluster“.

Python for High Performance and Scientific Computing BoF, Tuesday

Andy Terrel, William Scullin, and Andreas Schreiber organized a Birds-of-a-Feather session on Python, which had about 150 attendees (many thanks to all three for organizing a great session!).  Kurt gave a lightning talk on Enthought’s SBIR work.  The other talks focused on applications of Python in HPC settings, as well as IPython notebooks on the basics of the Navier-Stokes equations.

It was great to see so much interest in Python for HPC!

Mayavi – Talk at Fifth Elephant

The lead developer and creator of the 3D visualization package Mayavi, Dr. Prabhu Ramachandran, will provide a brief overview of Mayavi followed by his experience throughout the development of the package at the Fifth Elephant conference to be held in Bangalore, India.

Here is a brief video preview of the Mayavi user interface and Prabhu’s talk at Fifth Elephant.

Enthought at EAGE Copenhagen!

Enthought is at the European Association of Geophysicists and Engineers Conference & Exhibition in Copenhagen! Phew, that’s a mouthful. Enthought cut its teeth on seismic applications and, as you can see from the screenshot above, continues to cultivate its geoscience roots.

If you are in town, please come visit us at Stand 2132 in the Bella Center. We’d love to chat over a smørrebrød!

SciPy 2008 Swarm

During the “State of SciPy” presentation that Jarrod Millman and I gave this year at the SciPy 2008 Conference, we showed a “code swarm” video that was put together by David Martin and Chris Galvan. It’s a first-of-it’s-kind, because it combines repositories from several projects; namely NumPy, SciPy, Mayavi, Chaco, Traits, and Matplotlib.

Among other things, you can see how the number of people involved and the level of code production has really accelerated over time.

Enjoy!

Click Here for the Video

My Scientific Tool Stack

Numenta is distributing some of my favorite tools with their NuPIC toolkit including Python itself (on windows), wxPython, IPython, Traits and matplotlib. They’re also building some of their prototype and example applications using Traits. Enthought’s typical tool stack for building scientific apps looks like this (with the stuff Numenta currently distributes in white letters):

Yesterday, I gave a brief talk [.pdf] about the stack of tools that Enthought uses for building scientific applications at Numenta’s HTM Workshop (a gathering of over 200 folks interested in their library which implements Jeff Hawkins’ Hierarchical Temporal Memory ideas). My talk included a very gentle introduction to Traits/TraitsGUI.

I also briefly demonstrated some of the capabilities of the Chaco library (2D visualization) by showing some of the examples, and Mayavi (3D visualization) as an example of a Traits and Envisage (plugin framework) application.

Of particular interest to me today: there was a presentation by Subutai Ahmad showing a nice TraitsGUI-based node inspector for Numenta’s HTM library. It reminded me a lot of the “GUI for free” that happened with the TConfig project.

PointIR – Multitouch with Python (PyCon 2008)

Ive just uploaded a video from Peters presentation about the multitouch prototype that he and Dave Kammeyer and Robert Kern have developed over the past few months. It not only showcases what a few brilliant guys can pull off in their spare time, it shows the power of the open source tool stack that they used to get it done (Numpy, Scipy, Kiva, Enable, Chaco, Mayavi, Traits, Pyglet, to name a few).

PointIR in action

The full version is here:There’s a shorter version of a portion of the demo here: