Category Archives: Finance

Just Released: PyXLL v 3.0 (Python in Excel). New Real Time Data Stream Capabilities, Excel Ribbon Integration, and More.

Download a free 30 day trial of PyXLL and try it with your own data.

Since PyXLL was first released back in 2010 it has grown hugely in popularity and is used by businesses in many different sectors.

The original motivation for PyXLL was to be able to use all the best bits of Excel combined with a modern programming language for scientific computing, in a way that fits naturally and works seamlessly.

Since the beginning, PyXLL development focused on the things that really matter for creating useful real-world spreadsheets; worksheet functions and macro functions. Without these all you can do is just drive Excel by poking numbers in and reading numbers out. At the time the first version of PyXLL was released, that was already possibly using COM, and so providing yet another API to do the same was seen as little value add. On the other hand, being able to write functions and macros in Python opens up possibilities that previously were only available in VBA or writing complicated Excel Addins in C++ or C#.

With the release of PyXLL 3, integrating your Python code into Excel has become more enjoyable than ever. Many things have been simplified to get you up and running faster, and there are some major new features to explore.

  • If you are new to PyXLL have a look at the Getting Started section of the documentation.
  • All the features of PyXLL, including these new ones, can be found in the Documentation

NEW FEATURES IN PYXLL V. 3.0

1. Ribbon Customization

Screen Shot 2016-02-29 at 15.57.12

Ever wanted to write an add-in that uses the Excel ribbon interface? Previously the only way to do this was to write a COM add-in, which requires a lot of knowledge, skill and perseverance! Now you can do it with PyXLL by defining your ribbon as an XML document and adding it to your PyXLL config. All the callbacks between Excel and your Python code are handled for you.

See the Customizing the Ribbon for more detailed information or try the example included in the download.

2. RTD (Real Time Data) Functions

rtd

PyXLL can stream live data into your spreadsheet without you having to write any extra services or register any COM controls. Any Python function exposed to Excel through PyXLL can return a new RTD type that acts as a ticking data source; Excel updates whenever the returned RTD publishes new data.

See Real Time Data for more detailed information or try the example included in the download.

3. Function Signatures and Type Annotation

xl_func and xl_macro need to know the argument and return types to be
able to tell Excel how they should be called. In previous versions that was always done by passing a ‘signature’ string to these decorators.

Now in PyXLL 3 the signature is entirely optional. If a signature is not supplied PyXLL will inspect the function and determine the signature for you.

If you use Python type annotations when declaring the function, PyXLL will use those when determining the function signature. Otherwise all arguments and the return type will be assumed to be `var`.

4. Default Keyword Arguments

Python functions with default keyword arguments now preserve their default value when called from Excel with missing arguments. This means that a function like the one below
when called from Excel with b or c missing will be invoked with the correct default values for b and c.

@xl_func
 def func_with_kwargs(a, b=1, c=2):
 return a + b + c

 5. Deep Reloading

If you’ve used PyXLL for a while you will have noticed that when you reload PyXLL only the modules listed in your pyxll.cfg file get reloaded. If you are working on a project that has multiple modules and not all of them are added to the config those won’t get reloaded, even if modules that are listed in the config file import them.

PyXLL can now track all the imports made by each module listed in the config file, and when you reload PyXLL all of those modules will be reloaded in the right order.

This feature is enabled in the config file by setting

[PYXLL]
deep_reload = 1

6. Error Caching

Sometimes it’s not convenient to have to pick through the log file to determine why a particular cell is failing to calculate.

The new function get_last_error takes an XLCell or a COM Range and returns the last exception (and traceback) to have occurred in that cell.

This can be used in menu functions or other worksheet functions to give end users better feedback about any errors in the worksheet.

7. Python Functions for Reload and Rebind

PyXLL can now be reloaded or it can rebind its Excel functions using the new Python functions reload and rebind.

8. Better win32com and comtypes Support

PyXLL has always had some integration with the pythoncom module, but it required some user code to make it really useful. It didn’t have any direct integration with the higher level win32com package or the
comtypes package.

The new function xl_app returns the current Excel Application instance either as a pythoncom PyIDispatch instance, a win32com.client.Dispatch instance or a wrapped comtypes POINTER(IUnknown) instance.

You may specify which COM library you want to use with PyXLL in the pyxll.cfg file

[PYXLL]
com_package = <win32com, comtypes or pythoncom>

Download a free 30 day trial of PyXLL and see how PyXLL can help you use the power of Python to make Excel an even more powerful data analysis tool.

Plotting in Excel with PyXLL and Matplotlib

Author: Tony Roberts, creator of PyXLL, a Python library that makes it possible to write add-ins for Microsoft Excel in Python. Download a FREE 30 day trial of PyXLL here.


Plotting in Excel with PyXLL and MatplotlibPython has a broad range of tools for data analysis and visualization. While Excel is able to produce various types of plots, sometimes it’s either not quite good enough or it’s just preferable to use matplotlib.

Users already familiar with matplotlib will be aware that when showing a plot as part of a Python script the script stops while a plot is shown and continues once the user has closed it. When doing the same in an IPython console when a plot is shown control returns to the IPython prompt immediately, which is useful for interactive development.

Something that has been asked a couple of times is how to use matplotlib within Excel using PyXLL. As matplotlib is just a Python package like any other it can be imported and used in the same way as from any Python script. The difficulty is that when showing a plot the call to matplotlib blocks and so control isn’t returned to Excel until the user closes the window.

This blog shows how to plot data from Excel using matplotlib and PyXLL so that Excel can continue to be used while a plot window is active, and so that same window can be updated whenever the data in Excel is updated. Continue reading

The Latest and Greatest Pandas Features (since v 0.11)

On May 28, 2014 Phillip Cloud, core contributor for the Pandas data analytics Python library, spoke at a joint meetup of the New York Quantitative Python User’s Group (NY QPUG) and the NY Finance PUG. Enthought hosted and about 60 people joined us to listen to Phillip present some of the less-well-known, but really useful features that have come out since Pandas version 0.11 and some that are coming soon. We all learned more about how to take full advantage of the Pandas Python library, and got a better sense of how excited Phillip was to discover Pandas during his graduate work.

Pandas to MATLAB

After a fairly comprehensive overview of Pandas, Phillip got into the new features. In version 0.11 he covered: Continue reading

PyXLL: Deploy Python to Excel Easily

PyXLL Solution Home | Buy PyXLL | Press Release

Today Enthought announced that it is now the worldwide distributor for PyXLL, and we’re excited to offer this key product for deploying Python models, algorithms and code to Excel. Technical teams can use the full power of Enthought Canopy, or another Python distro, and end-users can access the results in their familiar Excel environment. And it’s straightforward to set up and use.

Installing PyXLL from Enthought Canopy

PyXLL is available as a package subscription (with significant discounts for multiple users). Once you’ve purchased a subscription you can easily install it via Canopy’s Package Manager as shown in the screenshots below (note that at this time PyXLL is only available for Windows users). The rest of the configuration instructions are in the Quick Start portion of the documentation. PyXLL itself is a plug-in to Excel. When you start Excel, PyXLL loads into Excel and reads in Python modules that you have created for PyXLL. This makes PyXLL especially useful for organizations that want to manage their code centrally and deploy to multiple Excel users.

Enthought Canopy Package Manager   Install PyXLL from Enthought Canopy's Package Manager

Creating Excel Functions with PyXLL

To create a PyXLL Python Excel function, you use the @xl_func decorator to tell PyXLL the following function should be registered with Excel, what its argument types are, and optionally what its return type is. PyXLL also reads the function’s docstring and provides that in the Excel function description. As an example, I created a module my_pyxll_module.py and registered it with PyXLL via the Continue reading

PyQL and QuantLib: A Comprehensive Finance Framework

Authors: Kelsey Jordahl, Brett Murphy

Earlier this month at the first New York Finance Python User’s Group (NY FPUG) meetup, Kelsey Jordahl talked about how PyQL streamlines the development of Python-based finance applications using QuantLib. There were about 30 people attending the talk at the Cornell Club in New York City. We have a recording of the presentation below.

FPUG Meetup Presentation Screenshot

QuantLib is a free, open-source (BSD-licensed) quantitative finance package. It provides tools for financial instruments, yield curves, pricing engines, creating simulations, and date / time management. There is a lot more detail on the QuantLib website along with the latest downloads. Kelsey refers to a really useful blog / open-source book by one of the core QuantLib developers on implementing QuantLib. Quantlib also comes with different language bindings, including Python.

So why use PyQL if there are already Python bindings in QuantLib? Well, PyQL provides a much more Pythonic set of APIs, in short. Kelsey discusses some of the differences between the original QuantLib Python API and the PyQL API and how PyQL streamlines the resulting Python code. You get better integration with other packages like NumPy, better namespace usage and better documentation. PyQL is available up on GitHub in the PyQL repo. Kelsey uses the IPython Notebooks in the examples directory to explore PyQL and QuantLib and compares the use of PyQL versus the standard (SWIG) QuantLib Python APIs.

PyQL remains a work in progress, with goals to make its QuantLib coverage more complete, the API even more Pythonic, and getting a successful build on Windows (works on Mac OS and Linux now). It’s open source, so feel free to step up and contribute!

For the details, check out the video of Kelsey’s presentation (44 minutes).

And here are the slides online if you want to check the links in the presentation.

If you are interested in working on either QuantLib or PyQL, let the maintainers know!

“Why We Built Enthought Canopy, An Inside Look” Recorded Webinar

We posted a recording of a 30 minute webinar that we did on the 20th that covers what Canopy is and why we developed it. There’s a few minutes of Brett Murphy(Product Manager at Enthought) discussing the “why” with some slides, and then Jason McCampbell (Development Manager for Canopy) gets into the interesting part with a 15+ minute demo of some of the key capabilities and workflows in Canopy. If you would like to watch the recorded webinar, you can find it here (the different formats will play directly in different browsers so check them and you won’t have to download the whole recording first):

Summed up in one line: Canopy provides the minimal set of tools for non-programmers to access, analyze and visualize data in an open-source Python environment.

The challenge in the past for scientists, engineers and analysts who wanted to use Python had been pulling together a working, integrated Python environment for scientific computing. Finding compatible versions of the dozens of Python packages, compiling them and integrating it all was very time consuming. That’s why we released the Enthought Python Distribution (EPD) many years back. It provided a single install of all the major packages you needed to do scientific and analytic computing with Python.

But the primary interface for a user of EPD was the command line. For a scientist or analyst used to an environment like MATLAB or one of the R IDEs, the command line is a little unapproachable and makes Python challenging to adopt. This is why we developed Canopy.

Enthought Canopy is both a Python distribution (like EPD) and an analysis environment. The analysis environment includes an integrated editor and IPython prompt to faciliate script development & testing and data analysis & plotting. The graphical package manager becomes the main interface to the Python ecosystem with its package search, install and update capabilities. And the documentation browser makes online documentation for Canopy, Python and the popular Python packages available on the desktop.

Check out the Canopy demo in the recorded webinar (link above). We hope it’s helpful.

Avoiding “Excel Hell!” using a Python-based Toolchain

Update (Feb 6, 2014):  Enthought is now the exclusive distributor of PyXLL, a solution that helps users avoid “Excel Hell” by making it easy to develop add-ins for Excel in Python. Learn more here.

Didrik Pinte gave an informative, provocatively-titled presentation at the second, in-person New York Quantitative Python User’s Group (NY QPUG) meeting earlier this month.

There are a lot of examples in the press of Excel workflow mess-ups and spreadsheet errors contributing to some eye-popping mishaps in the finance world (e.g. JP Morgan’s spreadsheet issues may have led to the 2012 massive loss at “the London Whale”). Most of these can be traced to similar fundamental issues:

  • Data referencing/traceability

  • Numerical errors

  • Error-prone manual operations (cut & paste, …)

  • Tracing IO’s in libraries/API’s

  • Missing version control

  • Toolchain that doesn’t meet the needs of researchers, analysts, IT, etc.

Python, the coding language and its tool ecosystem, can provide a nice solution to these challenges, and many organizations are already turning to Python-based workflows in response. And with integration tools like PyXLL (to execute Python functions within Excel) and others, organizations can adopt Python-based workflows incrementally and start improving their current “Excel Hell” situation quickly.

For the details check out the video of Didrik’s NY QPUG presentation.  He demonstrates a an example solution using PyXLL and Enthought Canopy.

[vimeo 67327735 http://vimeo.com/67327735]

And grab the PDF of his slides here.

QPUG_20130514_ExcelHell_Slides

It would be great to hear your stories about “Excel Hell”. Let us know below.

–Brett Murphy

Enthought Sponsors First NY QPUG Meetup

Though all eyes are probably on the aftermath of Pycon (which, from all reports, was another great conference), Enthought was happy to sponsor the first New York Quantitative Python User Group Meetup (wow that’s a mouthful) on March 6th. If you are in the New York area, you can sign up for the group here.

The program for the evening featured Marcos Lopez de Prado and our own Kelsey Jordahl (with an assist from yours truly). The meetup focused on the topic of portfolio optimization and some of its foibles. Marcos conducted an in-depth discussion of portfolio optimization in general and outlined his open source implementation of the CLA algorithm. He also discussed why he is such a fan of Python.

Our contribution to the evening focused on the the theme “From Research to Application.” And by “research” we meant both research code (Marcos’ CLA code is one example) and actual investment research. Firms are wrestling with data and trying to marshal all the expertise within the organization to make decisions. Increasingly, software is being used to help synthesize this information. In our thought experiment, we imagined a hypothetical portfolio manager or strategist that is trying to integrate the quantitative and fundamental expertise within the firm. What kind of information would this PM want to see? How could we make the application visually appealing and intuitively interactive?

We chose to use the Black-Litterman model to tie some of these threads together. In a nutshell, Black-Litterman takes a Bayesian approach to portfolio optimization. It assumes that the capital allocations in the market are decent and reverses the classical optimization process to infer expected returns (rather than weights). It also allows modification of these expected returns to reflect analyst views on a particular asset. For those of you not familiar with this subject, you can find an accessible discussion of the approach in He and Litterman (1999). Using the Black-Litterman model as our organizing principle, we put together an application that provides context for historical returns, relative value, and pairwise asset correlations, all wired together to provide full interactivity.

Given the limited time we had to put this together, there are obviously things we would have changed and things we would have liked to include. Nevertheless, we think the demo is a good example of how one can use open source technology to not only take advantage of research code but also integrate quantitative models and fundamental research.

FYI, the libraries used in the app are: Numpy/Pandas, Scipy, Traits, Chaco, and Enaml.

Videos of the talks are below. Tell us what you think!

QPUG_20130306_PortfolioDemo from NYQPUG on Vimeo.

QPUG_20130306_Marcos from NYQPUG on Vimeo.