Object Services Architectures

Trader Service

Project Summary

Venu Vasudevan and Tom Bannon
Object Services and Consulting Inc.
 
September 15, 1998
 

Executive Summary

We are seeing tremendous growth in web-enabled services, without any support in the web infrastructure for service location and service composition. The dynamic repository federation problem of OSA/Annotations and the dynamic behavior location problem in OSA/ISA are two  problems that demonstrate the need for such an infrastructure within the OSA/Scaling context. In the former, an annotation system needs to dynamically discover and access  annotation repositories (i.e. internet-accessible repositories that contain value-added annotation information) . In the latter, intermediaries that augment a web interaction with one or more mediating services, must dynamically locate the internet component  that provides this service (e.g. applet, CGI script, servlet). Traders that have been proposed in other frameworks to address the dynamic service location problem. However, they have not been demonstrated to work with internet services. Furthermore, they do not address situations where none of the available service instances matches the  client request. We believe such mismatches between client needs and available service offers can be better handled by a trader that composes multiple service instances to provide the client-requested service. Doing so requires a trader that understands service composition.

As part of the  OSA/Trader task, we define a design space for trading, identify the requirements for internet services trading, and motivate the need for compositional trading. Prototyping efforts demonstrate a native internet services trader, and the adaptation of an OMG trader to perform internet services trading. The latter identifies changes required in the OMG trader specification to support internet services trading.

Motivation and Problem

Web computing and e-commerce are leading to an explosive growth in web-based interactive services, and application services that can be accessed by other programs.  Location of and access to such services remains a problem, as the web does not provide a search facility that is tailored to service location and access. Conventional search engines work for document searches but are inadequate for service searches. Service location services (aka traders) have been developed in the CORBA and Agent frameworks, but  work only within the frameworks they were developed for. A trading capability that caters to web services needs to be demonstrated. Such a web trading service will provide the end-user with effective access to interactive web services (e.g. for online shopping), and can be used by programs (e.g. comparison shopping bots) to provide higher-level services that take advantage of a smart service location capability. More generally, the OSA/Trader facilitates a dynamic application architecture, whereby components of an application can bind to other components at run-time. Trader-based binding can be used by applications to reconfigure themselves to changing locations (i.e. application migration) or changing user context. The dynamic binding capability provide by OSA/Trader is useful as a core service in several OSA/Scaling project components. The OSA/Annotations Service and OSA/Network Performance Monitor Service both could use a trader to locate group or public caches of metadata related to their service.

Objective

The near-term objective of this task was to implement a trader for web-based services based on well known trading models. The longer term objective was to identify an augmented trading model that allows the trader to assemble more complex bindings that consist of multiple services and are subject to QoS constraints.

Limitations of Related Work

As part of the preliminary investigation, we surveyed existing trader implementations and published a report on a A Reference Model for Trader-Based Distributed Systems Architectures. This report defined a design space characterizing  currently published trader implementations. The taxonomy revealed that the OMG trader supports only part of the design space out of the box, and that it might be a heavyweight solution to the web trading problem. The OMG Trading model's choice of service description language was IDL, which is suitable for CORBA but not likely to be a popular choice for web service descriptors. XML provided equivalent or better capabilities to IDL, and was much more likely to be accepted by the web community. The OMG trader worked on a push model, where services (or third-parties) pushed service advertisements to the trader. A pull scheme is much more likely to succeed in the web context, whereby robots or spiders scour the web for services, and pull the advertisements out of web pages or by inspecting the service. The pull scheme has already proven to be successful in conventional search engines. The most serious mismatch between the OMG trading model and the needs of the web was in the federation approach. OMG adopted  direct federation, over repository-based federation. The direct federation approach requires traders to directly communicate to (and know of) the traders they federate with. While this is a secure and controlled scheme, the communication overhead in such a scheme grows proportionally to the number of federating traders. An internet-scale federation with hundreds of globally distributed traders is hard to set up if large numbers of communication paths need to be established to make the federation operational. A more scalable approach is one where federating traders share service information via a commonly accessible service repository .  Traders wishing to share a service advertisement would publish the advertisement to the service repository. Traders wishing to expand the services they trade with would periodically access this service repository and obtain meta-data about new services.  The communication complexity in repository-based federation is independent of the number of traders involved in the federation. From the point of view of a trader in this federation, other traders are anonymous providers of service advertisements. Since these advertisements are published to the service repository, a trader can use the service meta-data gathered by other traders, without directly having to communicate with them. The challenging problem of supporting a globally available service repository has already been solved in the internet architecture, as internet search engines naturally provide such a repository.

Current traders focus on one-to-one matches between client request and available service instances. This level of matchmaking is inadequate in a number of situations, as there are format, QoS and performance mismatches between the client request and the closest matching service instance.  A compositional trader can address this problem by composing such a service instance with format translators and performance monitors to reduce the mismatch.

Approach

We decided to pursue two concurrent  implementations of web-based trading to address the above questions.  A WebTrader built from the ground-up using web technologies (Java, XML and search engines) would investigate the feasibility of building a lightweight, scalable trader of and for the web. The MWTrader implementation is based on JTrader (an OMG-compliant trader) would address whether an OMG trader implementation could be modified to trade with web services, and to support repository-based federation. It would also provide insight into the reusability of services across frameworks, and the  interoperability problems in using an ORB service in a web context.

As part of the compositional trader research, we examined work in QoS and reconfigurable distributed systems for examples of QoS-driven service composition. Many of these efforts used proprietary architectures, but provided examples that could be handled by an augmented trader.  The report  on Augmenting OMG traders to handle service composition  discusses augmentations to the OMG trading model  that would allow traders to provide continuous and end-to-end guarantees about the services it trades with.

Results

WebTrader

The initial WebTrader prototyping effort resulted in an implementation built using Java, XML and search engine technologies.  As part of this prototype, we developed an XML-based service description format that matches the expressiveness of  OMG's service description  format (IDL + properties). The XML format offers the additional advantages of being ASCII and human readable, and able to deal with multi-portal services (services that export themselves via CORBA, CGI, RMI and other portals simultaneously).  In contrast, OMG compliant traders only support CORBA service advertisements. XML service advertisements can also be embedded in HTML pages, and shared by traders using search engine based page retrieval. The prototype demonstrates an end-to-end trading capability where advertisements are published to a search engine (or located by a crawler/spider), imported (pulled in) by a trader, and provided as the result of a client query.  As mentioned earlier, the pull model supported by WebTrader allows it  to discover new services, without the services having to advertise themselves to the trader.

Further work on the WebTrader explored recursive trading, whereby traders are themselves objects (with associated metadata) that are traded by other traders.  Such scaling of the trading concept is important in an internet scenario with thousands of  specialized traders  (e.g. battlefield image trading, a trader that is good at finding the best travel deals) that will be available on the internet. Recursive trading either supports trade shipping (i.e. the shipping of a trader query to other traders) or the inclusion of trader object references in the result set that is returned to the client. The latter is a scaled down implementation of the former, with the trade shipping responsibility handled by the client.  The idea of recursive trading is related to the direct trader federation ideas proposed by OMG, and to the STARTS meta-search architecture proposed for federating document search engines.  Our goal is to produce a lightweight scheme for dynamic trader federations, in contrast the OMG scheme which uses heavyweight, highly secure  contractual mechanisms to set up federations. STARTS is geared more towards large-grained document search engines (e.g. Excite, Yahoo), and is not directly applicable to recursive trading with large number of finer-grained traders.

MWTrader

The first phase of the MWTrader prototyping effort involved getting  a stripped down  JTrader to  work with OrbixWeb, and understanding its code. The code review was instructive in pointing out some quirks in the OMG Trader Specification that were not obvious from reading it. The service type descriptions which are part of an ORB's type repository  are also cached in the trader's type repository. The OMG specification does not provide a rationale for such caching, given that type information is fairly small, fairly static data and does not benefit greatly from caching schemes. Also unexplained is the fact that the OMG trading service has its own notion of properties, and does not use the OMG Property Service. It is notable that  the JTrader query engine (that accepts a client "matchmaking query") is almost 40% of the trader code and is not ORB-specific. This indicates that duplication of the OMG trader's constraint language in a web trader will be a significant task.

The second phase of MWTrader prototyping focussed on enabling it to :  a) operate in pull mode, b) trade with foreign (non-ORB) services, c) support the "soft" (approximate) matchmaking needed for QoS trading, and d) support compositional trading. As pointed out in an earlier semi-annual progress report, given the time constraints on the OSA/Scaling project, we expected to identify approaches to the above and identify code insertion points without being able to demonstrate all these ideas in a single prototype. We prototyped IDLBuilder, an XML-to-IDL translator to demonstrate that XML advertisements published to a search engine could be imported by OMG traders in much the same way as the WebTrader. This is an important architectural point, as it shows how to replace the heavyweight direct federation mechanism proposed in the OMG trader specification with repository based federation without being inconsistent with the OMG specification.  We defined an MWTrader API for the registration, lookup and matchmaking of foreign services (which had a URL as a handle instead of an ORB reference). While the code and API additions are relatively small, they require extensions to the OMG Trader data structures to support foreign handles, and LookupForeign and RegisterForeign APIs.  There will likely be other API additions as well, to allow the OMG trader to be accessed via the web, and to return matching service advertisements in a XML or other web-friendly formats.

We identified the interface between the matchmaker component in JTrader and the rest of the JTrader code where MWTrader would introduce soft matchmaking and compositional matchmaking function. We concluded that the mechanics of plugging-in a new matchmaker with either soft or compositional matchmaking semantics was fairly straightforward. However, the current OMG trader API needs to be augmented with a soft matchmaking API, as there is no way to pass distance functions to a trader in the current OMG trader specification.  We did not attempt to prototype a compositional trader, as this required significant additions to the OMG trading model. Details of these augmented trading model are covered in Augmenting OMG traders to handle service composition.

We briefly explored the notion of servicebots, agents that explore the internet for unadvertised services and extract their service advertisements. Given that there are thousands of services on the net which nobody has developed advertisements for, we see a bot that automatically generates advertisements as a useful adjunct to a trader. Discovering the structure of web services is a significant research challenge, and an area of active research in the AI and machine learning communities.  Work on AI-based information agents such as ShopBot (now Jango from Excite), ILA, BargainFinder, Information Manifold  begins to address service structure discovery for web services, but does not integrate it with trading. These are technologies that can be used as part of a trader's servicebot.  Building on their work, and relating these ideas to standards-based traders is a promising future research direction.

 


This research is sponsored by the Defense Advanced Research Projects Agency and managed by the U.S. Army Research Laboratory under contract DAAL01-95-C-0112. The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied of the Defense Advanced Research Projects Agency, U.S. Army Research Laboratory, or the United States Government.

© Copyright 1998 Object Services and Consulting, Inc. Permission is granted to copy this document provided this copyright statement is retained in all copies. Disclaimer: OBJS does not warrant the accuracy or completeness of the information in this survey.

Last revised: September 15, 1998.  Send comments to Venu Vasudevan and Tom Bannon.