You will be redirected in 30 seconds or close now.

ColdFusion Authors: Yakov Fain, Jeremy Geelan, Maureen O'Gara, Nancy Y. Nee, Tad Anderson

Related Topics: ColdFusion

ColdFusion: Article

A Case for Methodologies

A Case for Methodologies

Several years ago I came across a statistic that I've since shared with many others and it never ceases to shock me: according to four separate studies, the failure rate for custom corporate software development hovers at 70%. In fact, some studies indicate the figure may be even higher.

What is software failure and what causes it? There's no simple answer, but one thing stands out: software failures are seldom technical. Instead, the causes are much more fundamental - the delivered software does not do what the client needs it to do. Without dealing with this root cause, much of the excellent work being done to improve the technical aspects of our jobs is analogous to strengthening the strongest link of a chain, but providing no overall improvement to the chain itself.

It's my experience that in the vast majority of cases, the success or failure of a software project is determined before the first line of code is written. Software projects certainly fail, quite often, but they seldom do so because the developers were unable to handle multidimensional arrays or matrix math, or deal with recursion.

In fact, the software may run just as the developers intended, and yet the client still deems it a failure. Pointy-haired bosses are often given to oversimplification ("you people did not give it 100%"), but an overwhelming majority of the developers I know are both competent and committed. Something else is at work here. What did Shakespeare write? "The fault, dear Brutus, is not in our stars, but in ourselves..." Or, perhaps in our methodologies.

What Is a Methodology?
If you tell me, "Take these blocks and stack them on the table over there," you've given me an order. Being a conscientious person, I'll try to stack them in a pleasing arrangement - pleasing to me, that is. If you happen to like what I like, you'll love the results. If not...What if you give me the following instructions instead?

1.   Place the blocks from the canvas bag on the table.
2.   Align the red blocks so they form a single, straight line.
3.   Place the blue blocks atop the red blocks, but offset from the blocks underneath, so that the end of one blue block falls exactly in the middle of the red block beneath it.
4.   Place the green blocks atop the blue blocks, but offset from the blocks underneath, so the end of one green block falls exactly in the middle of the blue block beneath it.
5.   Place the remaining black block atop the green blocks, exactly centered.

Now you've given me a procedure for stacking blocks. If I follow it, I'll produce a pyramid. More important, anyone following the procedure will produce a pyramid. With an order, you'll get an infinite number of orderings. With a methodology, you'll consistently produce a pyramid.

I speak, write, and teach a good deal on methodologies in general and Fuse-box in particular. There are often a few (brave) souls who will honestly admit that they don't want to know exactly what they're building beforehand - part of the excitement of their job is discovering new shapes and configurations for the blocks.

I recognize and greatly empathize with this attitude. For years, I was a woodworker, designing and building custom furniture pieces. I found the idea of buying a plan and then carefully following the instructions abhorrent. I saw myself as a craftsman/artist, not a machine turning out replicas.

There is certainly a call for independent craftspeople/artists in every discipline and industry, but unless programming is a hobby, you're a professional - you're getting paid - and unless the people paying you understand that they're paying for an independent craftsperson/artist, there's bound to be conflict. What you consider to be an ingenious, elegant creation too often they'll view as something that "doesn't work." That 70% figure quoted earlier represents a clash in perceptions.

Perhaps, then, the first question to ask is: Do we want a methodology? Are we hobbyists or professionals? Neither is right or wrong; a problem occurs only when we confuse the two. Methodologies are crafted so we can predictably and reliably produce something and do it again when needed. They're suited to the needs of professionals. Software methodologies are crafted so that we can predictably and reliably produce successful software deployments. As Steve Jobs once pointed out to a group of programmers, "Real artists ship."

If we've decided that we're ready for a methodological approach to software development, there are many methodologies available, or you can produce your own. What makes a methodology a methodology is "an organized, documented set of procedures and guidelines," as defined in the dictionary. Too often, developers are asked to build a pyramid, without being given a methodology for building one. There are many methodologies available, but they tend to sort themselves into groups. As Mason Cooley put it, "Methodology is applied ideology." Let's look at a few.

The No-Methodology Methodology
Because there's a lot to master when learning programming, it's understandable that developers concentrate on mastering the alien syntax of a language first. Little reserves are left for reflection on methodologies - ways of approaching not just this problem, but problems in general.

Often, then, the first methodology unwittingly chosen is the most popular: the no-methodology methodology (NMM). Adherents of NMM may find that they each approach problems differently, but underneath this apparent diversity they're united in their approach: do whatever seems good at the time. The NMMers motto is adapted from the movie, Blazing Saddles: "Methodology? We don't need no stinkin' methodology!"

I once had a long discussion with the head of a well-respected consulting/development firm. I asked him if they used a specific methodology for their development. "Yes, indeed," he began. "We certainly do, and if I can ever figure it out, I'm going to write it down!" Here was someone who, for lack of forethought, stumbled into NMM rather than making a conscious choice. I suspect he exemplifies most NMMers.

There are a couple of problems with NMM. First, it doesn't scale well. If as a single coder I write a small application, I may well get by without rigor in my development methodology. But if the size of the project becomes large - as many often do - NMM will fail me. I'll find myself jumping from one file to another to another, not remembering which variables are being passed in, which ones are persistent, what scope variables are, what architecture...well, you get the point.

The no-methodology methodology also doesn't distribute well. A look at virtually any mature industry shows that a division of labor has evolved, allowing different people with different skills to all contribute to a project. "Many hands make light work," they say.

In a software project this causes the application to be broken into pieces. Now I can have other developers help me write code, database experts work on data design and queries, and layout specialists build the user interface. But this requires us to understand the whole well enough to break it into its parts - something that NMM can't offer. NMMers only know the whole (to whatever degree it's fully known) when it's finished. NMM almost guarantees high stress and long nights of lonely coding for its adherents.

As tough as NMM makes life for the writers of an application, the lives of those maintaining that application are exponentially worse. At least the original coders have some sense, however ill-defined, of how the application works. But what of the person who comes along after the application is written and must make changes to that code. He or she has no such advantage. As you might guess, NMMers are not very good about documenting their applications, leaving the maintenance programmer to try to understand the program from raw code, and lots of it. Talk about a Sisyphean task.

The 'Best Developers' Methodology
Let's leave the NMMers to their unhappy lot and look at another popular methodology, which I dub the "Best Developers" methodology. Their motto is vigorous and has an undeniable macho appeal: "Our programmers can beat up your programmers." I first learned of this one while talking with the CTO of a large, well-known dot-com. The subject of methodologies came up. "Our methodology," he told me, "is to hire the best coders and get out of their way."

But if his plan for success involved hiring the best coders, didn't this imply that failure was the result of hiring less than the best coders? Wasn't the whole "hire the best" plan an attempt to transfer all responsibility for software project failure to the programmer? Yet, as I said, my experience was that failed software projects had already failed before the programmer wrote a single line of code. The failure just wasn't apparent yet.

What if other disciplines were to adopt this "methodology"? Would we hear the head of the Federal Aviation Agency discussing their new plan to "hire the best pilots - and get out of their way?" The FAA chief wouldn't be the only one "getting out of their way."

The more I thought of this the more I disliked the approach. While seeming to flatter the chosen developers, it really paints a bull's-eye on them. When failure inevitably follows, then those developers involved must not truly be "the best." Such a shell game may insulate the manager from real responsibility by providing a whipping boy, but I am convinced it would do nothing to deal with the real problem of software failure.

The Proprietary Methodology
Another popular one is the proprietary methodology. It works tremendously well, but it's secret! We can't tell you what it is or, rather, we could tell you what it is, but then we'd have to kill you.

There are several problems with this approach. Because it's secret, it can't be peer-reviewed. Maybe it really is the best thing since sliced bread, but how can we verify this if the method can't be revealed? One of the advantages of an open methodology is that it's open to feedback from others. Proprietary methodologies, by their nature, are closed and secretive.

From the point of view of the client, proprietary methodologies tend to lock them into a single company. What assurance does the client have that the company will be responsive to their future needs or even (as we saw in the meltdown of dot-coms) that it will be around at all?

From the point of view of the individual programmer, working in a proprietary shop is a career dead-end. The investment made in learning the secret methodology is almost certainly useless, and the programmer has lost time that could have been spent learning a standard methodology that would put him or her in a better place for a career move.

Open Standards
It seems odd that we must reexamine the question of whether open standards are better than proprietary techniques, but the Proprietarians argue that the benefits of their secret methods outweigh the disadvantages. I often wonder if the secrecy just masks a no-methodology methodology from public view.

Remember the head of the technology firm who would write down their methodology once they figured it out? Well, apparently, they did figure it out, as the company transformed their "no methodology" into a "secret methodology" that they now tout as being superior. Sadly, of course, it's secret and so you'll just have to trust them. Isn't it possible, though, that the new secret methodology is just the old no-methodology methodology renamed for marketing purposes?

Open methodologies offer several significant benefits over either of the discussed alternatives. They give coders an incentive to become expert in a methodology. Developers know that the time and money they invest in improving their skills in an open methodology will pay off. Even if their current employer doesn't recognize their worth, there are other employers interested in their proven abilities to work within an open standard. Conversely, if my only skill is in your particular methodology, then I can only bargain with you. Such an arrangement disempowers workers.

It's not only programmers who benefit from an open methodology. Development shops that adopt open methodologies are much more likely to find contractors who can help them with overflow or temporary work. Shops are more likely to take on more projects, knowing that there probably won't be a large learning curve to overcome - there will be a number of developers who already know the methodology and can step into the project and begin contributing immediately.

Open methodologies also let companies leverage the investment they make in tools. Without a standard, companies are reluctant to invest in tools that make programming more productive and less tedious. Why invest in one set of tools for one developer using one methodology, when there's no expectation that other developers can also use this tool? But with an open standard agreed upon in advance, companies are much more willing to invest in tools. Their investment can be leveraged (and amortized) over many developers (present and future) and is not merely catering to the personal likes of one developer.

Clients, too, benefit from open standards. The investment they make in code can be leveraged. No one likes "sole-source" solutions and clients are more likely to invest in coding projects if they have confidence that there are many competent sources to turn to.

In short, a public methodology with open standards benefits all involved - developers, development shops, and clients. In this category are several that work with ColdFusion: Blackbox (www.black-box.org), Switchbox (www.switch-box.org), and CF-Objects (www.cfobjects.com). They result from the hard work and intelligence of their creators and are all worth looking at.

To get the full utility from a standard, however, it must be widely adopted. Without this, a standard is simply an idea. Only when people adopt the idea does it become a standard. In this regard, one methodology has an enormous advantage over all others. While advocates of the methodologies listed above number in the hundreds, Fusebox advocates number in the thousands. There's a vital, active community of Fuseboxers, Fusebox training is available, Fusebox publications exist, Fusebox has been ported to multiple languages, and the Fusebox methodology is international.

Why Fusebox?
Fusebox is built on three key principles:

1.   Modularity: Breaks the application into smaller, more tightly defined pieces ("circuits" and "fuses")
2.   Severability: Makes modules as independent of each other as possible by eliminating, where possible, fixed references and replacing them with variable references
3.   Clarity: Documents the interface (API) to individual code files through structured comments

I said earlier that Fusebox was, by far, the most popular development methodology for ColdFusion. While popularity with regard to an effective methodology is very desirable (more developers skilled in the methodology are available), the methodology itself must provide real benefits; popularity alone is no sure guide, as your mother used to remind you. What are the benefits thousands of developers have discovered, and what relationship does the Fusebox principles have to these benefits?

Benefits of Fusebox
One benefit arises from the ability to build independent software modules: separate developers, or separate teams of developers, can work on individual components simultaneously. This is a goal of virtually all methodologies, but few do this as well as Fusebox. This feature of Fusebox means that software can be built faster, as the need for serial correlation ("I must wait to do my task until you have finished yours") is greatly reduced.

It turns out that this severability allows software components, on both a large and small scale, to be reused more easily. Apart from the obvious cost benefit of not having to write software more than once, reusability means that developers can use pretested code, which both lowers the cost and decreases the risk of failure on a project.

Another key feature of Fusebox is an integral documentation/structured comment system known as Fusedoc. Comments are written in XML format. A document type definition (DTD) provides structure and allows other software to work with them more easily. The Fusedoc tells the coder what the individual file is responsible for, what variables are available at runtime, as well as variables the coder is responsible for creating.

The Fusedoc forms a sort of "work order" that the programmer works from. In the same way that machinists work on parts without knowing what they will be used in, Fusedocs let developers work on code without knowing about the application, or even the underlying database. The implications of this are enormous. Some 25 years ago Fred Brooks wrote the classic software text, The Mythical Man-Month. Brooks pointed out that adding people to a project that's late actually increases the amount of time needed to complete the project. The reason? It takes time for those new developers to understand the application enough to contribute. Add to that the increased communication and coordination time needed to integrate those extra workers, and adding developers only makes a bad situation worse.

Part of Fusebox's architecture planning involves separating an application into separate modules (or "circuits"). These modules are then broken into separate code files (or "fuses") and Fusedocs are written for fuses. At the point of coding, a developer is working on a single fuse. He or she knows what the fuse is responsible for and the input/output parameters, but needs to know little else. Now, if the project begins to run late, additional coders can be added to it. The ability to have code completed by competent developers who don't need to know about the application itself means that we can compress the time needed to actually write the code, typically far less than what any experienced developer would think possible.

Such an abbreviated coding time requirement radically shifts the equation of software development. Instead of rushing through the requirements, design, and architecture in order to get to the "real work" of coding, the requirements, design, and architecture become the real work. Coding is simply a matter of "scribbling down the details" as Mozart once remarked of the process of writing down the notes inside his head.

The ability to have multiple teams work productively on the same project must come as pure joy to any project manager. It also greatly lessens stress on the coders, who - freed of the arduous, thankless task of trying to guess what the client wants - can concentrate on writing beautiful code.

Fusebox also makes it possible for people with different skills and talents to contribute to the project. A Fusebox project often includes architects, database experts, an art director and graphic artists, an interface specialist, HTML/layout specialists, and ColdFusion coders. This is quite different from other development environments in which coders must handle all of this by themselves. Fusebox does this by separating an application's logic from its display. With the division of labor allowed by Fusebox, the overall cost of a project, the time to completion, and the risk of failure are all reduced.

Reusability is the holy grail of programmers. The appeal is obvious - write it once and use it over and over; if only it were so simple in practice. Every methodology tries to achieve it with limited success. Yet all that effort has taught us some things about reusability. The most important lesson is: all things being equal, it's easier to reuse a little bit of code than a lot. In short, simpler is better, so Fusebox encourages the use of small, well-defined code files (or "fuses") to complete larger tasks.

If complexity is the chief problem with code reuse, application-specific code must also rank high on a list of "Enemies of Reusability." Fusebox can help with this by providing for XFAs, a method where even page links can be set programmatically.

Once an application is built, the real costs begin. Maintenance of code normally accounts for as much as 80% of the total life-cycle cost. Much of this is due to the chronically poor documentation that forces maintenance programmers into detective work. Unless maintenance is planned for, it can quickly spiral into a black hole of costs for a company. More than one company has found itself saddled with obsolete software that's simply too complex and costly to adapt to new circumstances.

Fusedoc, an integral part of Fusebox, is enormously helpful to coders charged with making changes to others' code. Because each code file is well documented, the task of maintaining code, never a joy, is at least manageable.

No More Holy Wars
Anyone who has been involved in software knows the battles that rage over different languages, different operating systems, and different methodologies. We feel passionate about what we do. We ally ourselves with those who agree with us and avoid those who do not. But as we have seen all too clearly in the last few months, holy wars are capable only of creating suffering; they are powerless to change the hearts and minds of others.

I have personally found Fusebox to be very helpful to many developers I train and consult with. I use it exclusively on the work I do. Yet, I know some excellent developers who use other methodologies to great effect. It's important to remember that the purpose of a methodology - any methodology - is not to perpetuate itself, but to produce some desired result. What we all want, as professionals, is to reliably and predictably produce great software. Methodologies are important to the degree that they help us reach our goal, and that's something we can all agree on.

More Stories By Hal Helms

Hal Helms is a well-known speaker/writer/strategist on software development issues. He holds training sessions on Java, ColdFusion, and software development processes. He authors a popular monthly newsletter series. For more information, contact him at hal (at) halhelms.com or see his website, www.halhelms.com.

Comments (1) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

Most Recent Comments
Matt Lang 03/20/02 01:24:00 AM EST

I've read several articles by Mr. Helms, and by and large, find his articles to be of mixed value. This curent aqrticle however should have been titled "A cheerleaders guide to Fusebox".

Mr. Helms belittles other methods of development for the sole purpose of elevating fusebox. Even the secret world of proprietary methods are deemed unworthy in Mr. Helms view because he concludes that they are inferior simply because HE doesn't know what it is.

Near the end of his article, claiming Fusebox will bring us all together, Helms says "Anyone who has been involved in software knows the battles that rage over different languages, different operating systems, and different methodologies. We feel passionate about what we do. We ally ourselves with those who agree with us and avoid those who do not." This really sounds to me, in the context of his entire articel, that Mr. Helms has in fact declared Holy war on anyone who does not use HIS operating system/language/method... that is, Fusebox!

Please pull in the reins and reduce the workload of Mr. Helms to writing articles that are a bit more helpful, and a lot less evangelical.

IoT & Smart Cities Stories
Digital Transformation and Disruption, Amazon Style - What You Can Learn. Chris Kocher is a co-founder of Grey Heron, a management and strategic marketing consulting firm. He has 25+ years in both strategic and hands-on operating experience helping executives and investors build revenues and shareholder value. He has consulted with over 130 companies on innovating with new business models, product strategies and monetization. Chris has held management positions at HP and Symantec in addition to ...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...
DXWorldEXPO LLC announced today that Big Data Federation to Exhibit at the 22nd International CloudEXPO, colocated with DevOpsSUMMIT and DXWorldEXPO, November 12-13, 2018 in New York City. Big Data Federation, Inc. develops and applies artificial intelligence to predict financial and economic events that matter. The company uncovers patterns and precise drivers of performance and outcomes with the aid of machine-learning algorithms, big data, and fundamental analysis. Their products are deployed...
All in Mobile is a place where we continually maximize their impact by fostering understanding, empathy, insights, creativity and joy. They believe that a truly useful and desirable mobile app doesn't need the brightest idea or the most advanced technology. A great product begins with understanding people. It's easy to think that customers will love your app, but can you justify it? They make sure your final app is something that users truly want and need. The only way to do this is by ...
The challenges of aggregating data from consumer-oriented devices, such as wearable technologies and smart thermostats, are fairly well-understood. However, there are a new set of challenges for IoT devices that generate megabytes or gigabytes of data per second. Certainly, the infrastructure will have to change, as those volumes of data will likely overwhelm the available bandwidth for aggregating the data into a central repository. Ochandarena discusses a whole new way to think about your next...
CloudEXPO | DevOpsSUMMIT | DXWorldEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
Cell networks have the advantage of long-range communications, reaching an estimated 90% of the world. But cell networks such as 2G, 3G and LTE consume lots of power and were designed for connecting people. They are not optimized for low- or battery-powered devices or for IoT applications with infrequently transmitted data. Cell IoT modules that support narrow-band IoT and 4G cell networks will enable cell connectivity, device management, and app enablement for low-power wide-area network IoT. B...
The hierarchical architecture that distributes "compute" within the network specially at the edge can enable new services by harnessing emerging technologies. But Edge-Compute comes at increased cost that needs to be managed and potentially augmented by creative architecture solutions as there will always a catching-up with the capacity demands. Processing power in smartphones has enhanced YoY and there is increasingly spare compute capacity that can be potentially pooled. Uber has successfully ...
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things'). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing. IoT is not about the devices, its about the data consumed and generated. The devices are tools, mechanisms, conduits. This paper discusses the considerations when dealing with the...