Intermediary Architecture

Interposing middleware services and ilities between web client and server

Craig Thompson, Paul Pazandak, Venu Vasudevan, Frank Manola, Mark Palmer, Gil Hansen, Steve Ford
Object Services and Consulting, Inc.
November 21, 1997


This position paper describes what we are calling an Intermediary Architecture (IA), a specific infrastructure architecture for combining web and ORB/service architectures.  The idea is to interpose intermediary middleware plug-in services between the web client and server.  The architecture consists of a generic URL request interceptor and one or more plug-in services.  Services can be dynamically installed and placed on the client, server, or in between for different purposes.  Services may share a common service support infrastructure:  a system management policy governs the behavior of a service; services typically store their metadata in a local or remote, possibly shared repositories; and traders can be used to discover and federate repositories.  In our implementation (in progress), intermediaries are implemented as light weight proxies; services can be put in place with minimal changes to existing web infrastructure; they can be implemented in Java, C++, or other languages or using distributed programming architectures such as CORBA, DCOM, or ActiveX; and they may optionally modify the client browser or editor.  We describe examples that use the IA:  an annotation service, a personal web performance monitoring service, and ideas for others.  The Intermediary Architecture appears to be a generally useful way to integrate web architectures with middleware services.  Remaining challenges include composition of services, scaling, and federation of the IA infrastructure.


The overall thesis of our DARPA project Scaling Object Services Architectures to the Internet (also see Scaling OSAs architectural presentation) is that the web and object technology bases need to be better integrated to realize the benefits of both. We are working on several aspects of the problem including how to extend OMG's Object Management Architecture to accommodate architectural properties like composability, scaling, and evolution (OMA-NG), what a web object model might look like (Web Object Model),  and new infrastructures for web object middleware integration (the subject of this paper).  In a related DARPA project we are working on Survivability of Object Services Architectures.  We are long-time active members of OMG, currently chairing its Internet SIG, relatively recent members of W3C, and active as architects or reviewers in several other projects including National Industrial Information Infrastructure Protocols Consortium (NIIIP), MCC Object Infrastructure Project (OIP), and DARPA Advanced Information Technology Services (AITS).


Today the Web consists of thin clients, which provide universal browser interfaces, and web servers, which provide back-end access to global information resources (e.g., that's where the data is).  Meanwhile ORBs provide access to suites of middleware services (e.g., that's where the management is).  At present, ORBs are not really connected well to control access to web data sources.  Without better integration, the web community will build protocol extensions (e.g., WEBDAV) parallel to but different than the ORB community, and enterprise architects will be forced to make idiosyncratic tradeoffs in constructing systems that need to use both architectures together.

Web clients and servers today leave certain hooks for extensibility:

It would be desirable if it were easy for end-users and developers to plug in modules from component software libraries to extend web architectures.  Middleware ORB-based service architectures provide candidate component libraries.  So a next questions is, how can we combine web and ORB architectures.  Two approaches to web-ORB integration are already in common use: A third web-ORB integration architecture, which we are calling an Intermediary Architecture, also appears to be broadly useful in that it interposes intermediary middleware services directly between web client request and web server response. There are others as well:  WAI enables one to associate a (CORBA) service with a URL.

From a somewhat higher vantage point, we (like others before us) are noticing that Web and ORB architectures are naturally growing into each other's turf.  We are going through an architectural exercise in comparing the two architectures and then performing an operation on architectures by extending one with capabilities of the other by splicing OMG ORB-like service architectures between web client and server, in effect creating URL brokers.

Intermediary Architecture Examples

The design pattern of interposing intermediary behavior between components is not new. Wrappers do this in an ad hoc way.  More generally, reflective architectures (e.g., CLOS) provide expansion joints where new behaviors can be interposed.  The DARPA Open OODB system [Wells, Blakeley, Thompson. "Architecture of an Open Object-Oriented Database Management System."  IEEE Computer, October 1992] used sentries to install new behaviors like persistence and versioning and treated querying as a service.  OMG's security architecture uses (nearly) general purpose interceptors to overload the CORBA dispatch mechanism and install security into OMG's distributed object architecture.  The OMG Portable Object Adapter specification provides before-after filters. Several groups (ours, MCC OIP, others) are exploring a general intermediary design pattern for installing QoS -ilities in communication paths in ORB and now web architectures.

On the web, most work has focused on single-purpose, hard-wired intermediaries, called proxy servers.  A number of http servers can act as proxy servers to receive a URL from a client and pass it on to a server,  first performing some intermediary operation, then returning a page through the proxy to the client, possibly side-effecting the page transfer.  Common uses for proxies have been page caching mechanisms and security firewalls.  Web platform vendors are increasingly bundling functionality that could have been added by intermediaries, so thin clients are getting fatter:

Some of these also require client-side access to browser or editor/composer - a lesson we've learned is that the APIs to these components are deficient in some popular web products. Few have explored general-purpose, open intermediary architectures:  Server-side includes support feature insertion -- pages are scanned before being sent to the client for embedded instructions (html tokens) which can direct further execution.  Sun's JavaServer Toolkit and The Open Group's WAIBA are closest to our own. The W3C Protocol Extension Protocol (PEP) is addressing the same need (see more below). NS Enterprise Server's NSAPI enables one to attach code to steps in the request-response cycle

In order to better understand the design space for an intermediary architecture, we prototyped some plug-in services and the IA support infrastructure and at present are integrating the services into the infrastructure.  The plug-ins we have built include:

Others have developed what appear to be candidate IA services usually for dedicated purposes and minus a compositional service infrastructure.  Some potential examples of services include:

Intermediary Architecture Design Overview

A longer paper documenting our detailed design and interfaces is in preparation and should be available 1Q98 on our web site document log.

The idea of the Intermediary Architecture is conceptually to interpose intermediary middleware plug-in services between the web client and web server. The main components of the architecture are:

ORBs permit the request and response of an object invocation to be intercepted and manipulated on either the client or the server side. But commercial web clients (like Netscape Navigator) and servers so far do not directly support client side or server side filtering.  The experimental Jigsaw server does support server-side filters.  The normal architecture of the web is for a client to issue a GET request referencing a URL and a server to return the associated resource (page).  Intermediaries provide a general way to augment or overload this access behavior to add new behaviors.  One can draw a wrapper-like picture of client-IA-server where client is red and server is green and IA has a green-red layer so the client sees IA as a server and the server sees IA as a client.  This is like mating male and female leads.

One of our design goals was to avoid making changes to existing web infrastructure and still install the IA architecture.  We closely examined the Netscape browser client API to locate a hook for URL interceptors but found the API to be pretty minimal and the available hooks to be insufficient.  So in our current prototype (in progress), intermediaries are implemented as proxies.  At first we avoided this solution because proxy servers are generally heavy weight, but a very thin pass-through proxy can server as a URL interceptor. This allows IA to be portable across existing web infrastructure.  Commercial browsers and servers might in the future incorporate IA capabilities or implement extensible protocols like PEP (see below) to allow IA openness supported directly in clients or servers.  But it appears that both might effectively just be incorporating (bundling) IA, and there will still be a need for intermediary  implementations separate from client and server as well as a need for improved client and server APIs.

Intermediaries can be placed on the client, on the server, or in between.  It makes sense to place intermediaries on the

The above placement considerations are not the only ones. You may want the service on the server-side if the operation needs to be protected (e.g., it's proprietary and the owner wants to charge for its use, even if the operation is tailored for a specific client, e.g., by a client profile on the server). Also, you may want to locate services on client or server based on performance tradeoffs (reduce data shipped over the network by processing at the server, or offload processing from the server by moving the responsibility to the client), or security considerations.

Brief Comparison of IA with PEP

W3C's Protocol Extension Protocol (PEP) appears to share the same motivation as IA. PEP requires web servers to change to install the PEP protocol seamlessly where IA can make use of existing browsers but requires clients to explicitly redirect their browser to an IA proxy - really this is a small difference, we use proxies as a way to intercept URLs and if PEP-enabled servers provided this, we could use PEP. Both PEP and IA provide for sequencing of intermediate side-effecting services and dynamic loading of new services. At this level, both assume services that are protocol independent. PEP reasonably stops short of suggesting much about an infrastructure for services (policies, federated repositories, traders) nor is it exploring ORB services as plug ins or client-side extension mechanisms.

Next Steps

Our next steps are:


Work on component-based architectures is in its infancy. The IA presented here appears to be a productive architecture pattern though there is still much to learn before we can confirm this. It appears to offer a convenient way to deliver new services to end users in a way that may be amenable to visual application assembly from a potential smorgasbord of standard services that could be made available by third parties, opening the door a little wider for a component economy. The idea is not entirely new, so we can learn from related work on ORB interceptors and MOPs.

There remain several unexplored questions about IA architectures:

If we are successful in providing a widely useful architecture for web plus object integration, we will help to provide "objects for the masses" in the form of an understandable abstract framework for scaling, composing, and federating services to enable rapid application development.

Related OBJS papers include OMA-NG, Survivability, Web Object Model, Trader, Annotations Service, Performance Monitor Service, and Query Augmentor Service, and others.

This research is sponsored by the Defense Advanced Research Projects Agency and managed by the U.S. Army Research Laboratory under contract DAAL01-95-C-0112. The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied of the Defense Advanced Research Projects Agency, U.S. Army Research Laboratory, or the United States Government.

© Copyright 1997 Object Services and Consulting, Inc. Permission is granted to copy this document provided this copyright statement is retained in all copies. Disclaimer: OBJS does not warrant the accuracy or completeness of the information in this paper.

Last revised: December 8, 1997.  Send comments to Craig Thompson.