Monthly Archives: May 2008

Traits for doing Reactive Programming

I have done event-driven programming. I have done structured programming. I’ve done OOD that was exceeded only by the Platonic Ideal. I’ve done spaghetti code that would make an Italian chef proud. But I’ve never, until coming to Enthought, done Reactive Programming.

Enthought’s Traits module leads to doing a different kind of software development. I think Reactive Programming is the best way to describe Traits programming. It’s possible to do some really elegant things with it: clean, tight, transparent. And it’s possible to do some evil things too: hidden-dependencies, and obfuscated code.

I’ll see about getting some examples up soon. But not today. Not on my Friday afternoon.

Project Estimation Guidelines

When preparing to embark on a new project it is useful to establish a proper expectation on cost, time, and quality of the project. So here are a few things that you should think about and document as part of your planning process,

  1. Project scope statement: What exactly will the project accomplish?
  2. Project management plan: How will the project be monitored? Will a single person keep track of the hours and scheduling? How will the customer, or stake-holders, be kept informed of how the project is proceeding?
  3. Work breakdown structure: The chunks necessary to complete the tasks assigned in the scope. This may be Trac tickets, or code modules, etc..
  4. Resource allocation: Who will be on the project and when?
  5. Risk analysis: What are the most likely things to go wrong? Often it’s that the code takes longer to write than expected, but what about unexpected hardware problems, time constraints, or specific customer issues?

When doing your analysis, it’s easy to see the big picture. However, there’s a lot of small and not-so-small tasks that you need to remember. There’s a good chance you’ll need to take into account the following tasks, even if individual task sizes will vary based on the size of the development team and the scope of the application.

Project overhead

Status meetings, phone calls, talking to customers, managing people, generating reports, mid-stream application redesigns all add up to a big chunk of change. Some projects can get by with 10% overhead, others will push 50%. It depends on the size of the project and the degree of reporting desired. On some projects I was able to get the overhead down to about 8-10 hours for every 120 hours logged. But the numbers went up markedly at the start and the end of the project life-cycle.

Time to estimate the project task estimates

Since Enthought is a software company, we estimate how long it will take to write or provide a software application. Estimating how long it will take to write code is one of the riskiest estimates in a project since ongoing development often reveals additional features or hurdles that were not originally planned for. Creating the task estimates requires having some of the design work completed to make sure all subtasks have been identified. Therefore, the risk of estimate overruns can be reduced by doing a design phase to identify, design, and estimate the subtasks. But extensive design phases can be expensive and possibly wasteful if the project scope is changed to not include the estimated subtasks.

Software build deliverable

Generating a build of the application for the customer should be a trivial task but historically all kinds of problems crop up with libraries, paths, versioning, you name it. Make sure to plan enough time to deliver to the customer the product or service they requested.

Team member training

There will need to be an initial training phase for any new team members joining the project. If the team is already trained then the estimates can be low, but if there is a lot of new domain knowledge to be learned, or the initial development environment takes a long time to set up, or there will be a lot of members joining and leaving the team then this number could be much higher.

Demos and project realignment

In a dynamic and iterative development environment like we have at Enthought, the customers play an active role in the guidance of the tools we write. That means we take time to demo the application, and integrate the feedback they provide into what has already been written and where the project is going. The demos and realignments take time, so include them into your schedule and budget. One possible time-sink is rewriting GUIs that already work, but need to be redesigned based upon customer feedback.

Integration Testing

Integration testing is different than unit tests, in that problems can arise when pieces start fitting together. Integration can mean different things for specific projects. For example, integration testing might verify that Envisage Plugins that were written separately do actually work together, or it could be testing that the application integrates with the customer’s target server, or it could be integrating an old code base into an new environment.

Quality Assurance

Before delivering the final product it should be rigorously tested to verify that everything works as expected. New features may have accidentally broken something that was tested in the past, so the application should be locked-down and tested one last time before being sent out.

Documentation

Documentation is an important component of a project, yet often forgotten by developers when doing a work breakdown analysis. When estimating documentation costs, be sure to include the time that the technical writer will spend on writing the documentation, AND the time that developers will spend being interviewed by the documentation team.

Known Unknowns

A time reserve, or ”contingency allowance” should also be created. This pool of resources is then managed by the project manager to be used as needed. Unexpected events occur in projects with such regularity that they often are included in the initial estimates and called ”known unknowns” so that other task estimates can be exceeded without a need to modify the project schedule. Some developers calculate a value by multiplying the total estimates by a scaling factor, but the multiplier is only useful if the original estimates are correct. Whether the reserve resources are actually included in the budget plan depends upon who the estimator is. A developer shouldn’t be able to dip into the contingency allowance without the project manager’s approval. But then the Management Team may have a second contingency allowance for ”unknown unknowns” that even the project manager does not have access to without approval.
When doing your analysis, the big pictuce

Greg Wilson speaking at the Austin Python User Group meeting

Wednesday, May 14th, Greg Wilson will be joining us in Austin for the monthly APUG meeting. He’ll be talking about Beautiful Code. If you’re in the area, swing by Enthought’s Offices right downtown at the corner of 6th and Congress (the Epicenter for Weirdness, as we like to call it).

There’s more information at the python.org wiki page for the User Group and at the meetup site for APUG.

Facelift for code.enthought.com: of Commodities, Communities and Mullets

We’ve recently refreshed the look and content of our code.enthought.com site, so I thought I’d provide my thoughts about what the site is about and what it means to Enthought (and the world). I’ll apologize in advance for rambling.

code.enthought.com

For several years Enthought has hosted code.enthought.com as a site for the Open Source tools that we’ve developed and used in our business of custom application development. The collective set of tools is called the Enthought Tool Suite (ETS). The impetus for the site came from some technologies that we believe will change the landscape of scientific computing and which we believe should be software commodities.

What do we mean by Software Commodity?

A commodity is a product that you don’t ship and your competition does Jonathan Schwartz, Sun Microsystems

There are many aspects of traditional commodities that provide nuance to the idea of commodity software, like equilibrium price, interoperability, fungibility, substitution, etc. My working definition for the purpose of Enthought’s mission hearkens to the French commodit, or even better, the Latin commoditas, which both connote convenience, appropriateness, suitability, general usefulness, or common utility. So any piece of software about which you may say “No one should ever have to write another ____” fits into my meaning. Some examples we’ve come up with:

  • No one should ever have to write another 2-D visualization library (Chaco)
  • No one should ever have to write another 3-D visualization library/general application (Mayavi, thanks to Prabhu’s vision and hard work)
  • No one should ever have to write another event model (Traits)
  • No one should ever have to write another adapter library (Traits)
  • No one should ever have to write another interface library for Python (Traits)
  • No one should ever have to write another GUI abstraction layer (TraitsGUI)
  • and many more…

So what’s our mechanism for the commoditization of software? In three words, the BSD license. There’s nothing new here. Commodity is an important by-product of Open Source. It’s a beautiful way to compete very aggressively on price (free!) and marshal a community of academic and commercial interests. It doesn’t alienate (most of) the Open Source world and it doesn’t frighten (most of) the commercial world. It provides the most liberal license for an extremely free use of the software while allowing us to protect the aspects of our clients’ business that they view as a competitive advantage (no copyleft). To my thinking, it not only creates commodity, it creates abundance. While there are many benefits to open licensing, the most important to us is the community it creates.

Speaking of Community…

Forming and working on the experiment that we call “Enthought” has been a wild ride. I can’t imagine a more lasting thing I’ll help create (except my kids; and my contribution there was stunningly trivial). We’ve worked with a community of others to create tools that should outlive us all. The truth about open source community interaction, however, is that it’s hard. It’s messy. It’s not immune to politics. It takes work. We’ve been fortunate in that the SciPy, ETS and the other Python communities are made of extraordinarily bright, thoughtful, slightly crazy peopleso tedious squabbles are very rare. Good things happen because a community can rally around a vision of common utility, esteem for contributions is based on the calculus of that utility and, for the most part, everyone’s goals are aligned. It’s a virtuous circle.

Our hope as an organization is that we can help foster this environmentto equip the community without directing it, to contribute without owning, to create common utility. How does this jibe with the following facts?

  1. Enthought is in business to make money.
  2. We don’t work for free.
  3. We are self interested.
  4. We need to feed those kids somehow.

We founded our company with the quaint, yet earnest, belief that we could do well by doing good. And you know what? It worked. We’ll let others judge the amount of good we’re doing, but we have created a thriving business with a model that leans heavily on using and creating Open Source software.

We fully recognize that there will be folks who are skeptical about what Enthought is aboutthey should be. Any representations we make about our company should be borne out by our actions. We invest heavily in open source software. We do see the benefitsboth in leveraged utility and in the area of marketing our company.

So what does this have to do with the changes at code.enthought.com?

We’re excited about finally committing some resources to spruce up the site and provide some content that’s actually useful. We’ll continue to invest in the tools and in helping the growing community of developers and users. That said, the site is the product of a community effort and we hope that our efforts are just priming the pump.

There are actually two venues of content available for folks interested in these tools. The more-static, more-business-like code.enthought.com site that was recently refurbished, and the constantly-changing, wild party that happens on the trac instance (issue-tracking, wikis, roadmaps, etc.) that supports the community development efforts. These two areas cooperate in what has been eloquently called a mullet strategy by some (business in the front, party in the backthanks to Janet for the reference). The mullet appoach makes a lot of sense for a community site, (and it’s fully neologism compliant: see the pro-am movement, the long tail, crowdsourcing, etc. 😉 ). Cynicism aside, we think it’s a nice way to balance content consistency and relevance.

Both the code.enthought.com site and the trac instance are community sites. If you can get some utility from participating in the community or if you want to pay commodity prices (or less), join us.

There is serious work happening in both professional and academic settings using these tools. The productivity gains afforded are nothing short of astonishing. I encourage anyone who is interested to read the front matter of the site and provide feedback We hope it makes a strong business case for utilizing the tools.

All that said, there are many great ideas that have yet to be implemented. If you’re interested in joining the conversation and getting elbow-deep in code I encourage you to get involved. Come join the party out back.