A New Way

One of the most important developments at Tucows in the past several months is a change in the way we develop the software behind out services. Over time, the software has become more complex, the number of users has climbed, the time-to-market has shrunk and changes to the internet have accelerated. We've had to change the way we write our software, and in the next couple of articles, I'll cover these changes.

The Waterfall Model

Let's consider a classic software methodology — one that's still in widespread use today, and one that was, until recently, the way we used to develop software here at Tucows: the waterfall model of software development.

The waterfall model takes its name from the fact that it views software developments as a set of phases that a development team goes down in a cascading fashion, like water going down a waterfall. In the waterfall model, you start at the first phase, and move on to the next one as soon as the current phase is complete.

The number of phases in a waterfall will depend on which software engineering text or site you consult, but they can be generalized to these five:

1. Requirements: In this phase, the developers communicate with the customers and users to find what they the application to do, analyze those requirements for contradictions or conflicts and resolve them, and turn the resulting requirements into requirements documents — some form of documentation on which the developers will base their design.
2. Design: Based on the requirements documents produced in the preceding phase, the developers create design documents, which act as the blueprints for the application to be built. If the requirements documents specify what the application will do, the design documents specify how the application will do it.
3. Implementation: In this phase, the developers build the application, using the design document as their guide.
4. Verification: Once the application is complete, it goes to the verification phase, where the application is tested to make sure that it meets the requirments and is free of errors. Once the application has passed the verification process, it is released.
5. Maintenance: Software that has been released is in its maintenance phase, during which time it may be modified or updated to meet new user needs, made compatible with other systems or to correct errorsnot detected during the verification phase.

Upsides and Downsides of the Waterfall Method

The waterfall model is a manager's dream. By compartmentalizing the devlopment cycle into distinct phases, it makes it easy to break a software development team into departments, which in turn makes it easier to maintain managerial control. The waterfall cascade also aligns quite nicely with Gantt chart-style planning.

It's also a developer's nightmare. It fails to account for certain realities of software development, one of which is that things change. As the application is being built, there can be all sorts of reasons for making changes to it, including:

• Unforeseen requirements that could not have been anticipated early in the development lifecycle
• Changes in the customer's business or the regulatory environment in which the customer works
• Changes or limitations in technology that take place or are discovered during the project
• It's often hard to specify every last requirement before seeing any working code (hence the customer dilemma most developers have face: “It's what I asked for, but not what I wanted!”)
• Often, an initial design will is not adequate to the task that the application is supposed to perform

The waterfall model is a poor fit for software development because it's no good at solving wicked problems.

Software is a “Wicked Problem”

The term “wicked problem” was coined in 1973 by Horst Rittel and Melvin Webber; they used the term in a paper on social policy planning.

A wicked problem is one that cannot be solved by planning; rather, it must be solved by actually going through the process of trying out solutions. This is because wicked problems have the following characteristics:

• They're not truly understood until you try and come up with a solution.
• They have requirements that are often incomplete, contradict each other, and are subject to change.
• Wicked problems contain all sorts of complex interactions that may not be well-understood or seen at the beginning.
• The solution of one of the aspects of a wicked problem may reveal or create another, even more complex problem.
• Stakeholders have radically different world views and different frames for understanding the problem.
• Constraints and resources to solve the problem change over time.
• The problem is never solved.

Although Rittel and Weber were working in the field of sociology, wicked problems are encountered in many other areas of study, including software development. The process of developing software has the same characteristics as a wicked problem, and because of this, many developers feel that the waterfall model addresses only the management issues of software development and not its “wicked problem” characteristics.

Luckily, there are other ways to develop software, and we've adopted them. I'll cover them in the next article.

Posted on Monday, May 14th, 2007 at 6:51 pm by Tucows.
Filed under: Programming.
Both comments and pings are currently closed.