Components should automate their own usage

(Achieving high productivity in complex software environments)
Peter Ruttan
Netron Inc., (416)636-8333
November 10, 1997


Everyone has an appreciation for the complexities of dealing with multiple hardware platforms, operating systems, communications protocols, languages, etc. Standards will reduce the complexity in these areas, as will industry consolidation. Rather than focusing on those phenomena, this paper explores how to enable a development staff to be productive in spite of the environmental complexities that burden the system.

Let us accept the following facts:

All of this means that the complexities of today's software development are inescapable, presenting an everlasting challenge; it does keep life interesting.

Being a bold optimist, the author is confident that every complex problem facing developers today can be solved. The challenge is to ensure that each problem is assigned to someone who can solve it. It is unlikely that any one person can know enough to solve all problems. Even if that were possible, we can't afford to be dependent on any one individual to solve every problem.

Achieving high productivity requires establishment of an environment that allows technological problems to be solved without bringing development to a grinding halt. To accomplish this we need several things:

Characterizing the Problem

Let's focus on source code, remembering that this is only part of the development process, and that all of the parts are interrelated (that's why we refer to it as a life cycle). In the end, source code is the one thing we can't skip if we're going to get a system into production. All solutions to all of our business problems and the implementation of all of our standards must get rendered in source code. And, no, we won't quibble over whether we wrote, rented, or bought the solution.

There is an aspect of writing software that is not getting enough attention: Too many people are writing too much code over and over again. We spend all of our time solving difficult problems that are very interesting and in doing so ignore the obvious. Here's an example:

For years, many people have written about the different layers of abstraction appropriate for communications protocols. So many people have thought about this problem that we have quite a few standards to choose from. Vendors have figured out that implementing these standards is tricky enough that people will pay to have it done for them. This gives us a number of products that we can purchase to handle the hard part of getting application components to talk to each other. Without belaboring the fact that most of these vendors add a little something extra to improve upon the standards, or address some dimension of the problem space that the standard hasn't gotten to yet, it bears mentioning that we are still left needing to write fussy code to make use of these middleware packages. IDL, of course, comes to mind, but IDL really just displaces the problem to being one of needing to write around the fussy middleware code.

There are two principal concerns about this approach to handling technology problems:

We don't have to know how we are going to solve a problem to know what information we need from the application or what information we are going to give back to the application when we're done. In fact, virtually all development methodologies say that you should define your inputs and outputs before you address the internals. But still, we tend to defer clearly defining to application developers what they need to do until after we've got the hard stuff ready for production.

The result of these two behaviors is:

A quick survey of the source code for most applications will show that in spite of our best intentions, code dealing with technology is tightly coupled to code dealing directly with the application at hand. In the interest of meeting deadlines, it is our nature to cheat and defer structuring the application for maintainability. We all know that once we defer the clarity and structure required to have a maintainable application, it is deferred forever. Once an application is functionally ready for production, the risk of breaking the functionality makes it virtually impossible to adjust the code for the sake of making it better.

Solving the problem: make components usable and stable

So what can we do? There is no question that component-based development will improve our lot. It will force us to be more conscious of problem decomposition. We still have a huge amount to learn about the appropriate level of granularity of components. This will come over time. The rapid pace of improvements in hardware performance makes it much easier for us to optimize for clear boundaries between components and accept the performance penalty that highly layered call structures introduce.

While we are going through the natural learning process, we need to fundamentally change our measure of success in producing components for reuse. In addition to the fairly routine evaluation based on the architecture of the component, we need to consider a component's usability and stability:

These questions need to be explicitly asked from the point of view of an application developer. Far too often, it is the developer's perception that it is easier to just write the code than it is to find and figure out how to use a component.

Why is it so hard to inventory components? The problem domain is much smaller than that of managing the parts used to, say, build a 747. The parts are all in a readable form so relevant information about them can be extracted by a relatively simple utility. The only apparent constraint is the discipline to make the cataloging of the component an inherent part of the creation of the component.

The author's organization took this approach, leading to the conclusion that the incremental effort is tiny and the payoff enormous. Components can actually be found in not much more time than it would take to decide what to name a new component.

Once a developer finds a component, the next challenge is to use the component correctly. This means figuring out what source code has to be written to (a) establish a connection to the component and (b) invoke the component with the right parameters and calling conventions.

Having committed to inventorying components, the next logical step is to automate the usage of each component. In essence this means associating with the component some wizard directives that can drive a dialog with a prospective user and translate the developer's responses into the source code required to achieve the desired objective. The degrees of freedom and the calling conventions of a component can, and should, be available from the component itself.

If we are going to the trouble of designing and building well-defined components, the incremental effort to capture the usage rules is relatively small. Substantial gains are realized because:

Anyone who has been watching closely will have noted that in making it easy for developers to use a component, an abstraction layer is created around the use of the component. The existence of this abstraction layer affords the ability to evolve the component at no risk to a user of the component. Many people who have tried to upgrade a component have noticed that the biggest constraint is the retrofit of users of the component.

Making the component easy for a developer to use also solved, in advance, the problem of retrofitting the use of changing components.

In the end, by being careful up front about the domain of a component, clear about the definition of input and output data fields and not considering the component development to be complete until its usage by a developer has been automated, productivity was more than doubled throughout the development life cycle.