Visit to Dartmouth
May 6, 1999
Object Services and Consulting
I visited Dartmouth College to give a talk on my CoABS-related work, at
the invitation of Dave Kotz, PI of Dartmouth's D'Agents project.
In addition to giving the talk, I spent some time talking with members
of the project, and some other students.
Dave briefly went over the work they were doing, and showed me some
of their computing facilities. We discussed the development of CoABS
Grid ideas and how that work was coming together. I also told him
generally what the idea of technical exchange with ALP was all about.
The D'Agents project has built information access agents that interface
with the Smart full-text information retrieval system. For each query,
an agent visits a sequence of sites; at each site, it interacts with
a local Smart agent to search a local collection of documents. Smart
uses a vector-space model to measure the textual similarity between documents.
The agent fuses the results the returns to the user site to display the
results. One of the result displays is as an interactive graph showing
clusters of related documents and inter-document similarities. Katya
is working on improved clustering techniques in this application.
I asked her what she thought about developing these techniques so that
they could be better-integrated with mechanisms for searching more structured
information in distributed systems, noting that there was a spectrum of
available information which might be considered to have full-text retrieval
systems at one end, relational databases at the other, and semi-structured
Web information somewhere in the middle. It would be nice to be able
to tweak her techniques so they could use, e.g., ontology/thesaurus information
where that was available (e.g., to distinguish between the same term used
in two information repositories which was known to have distinct meanings
in those repositories). She said it was an interesting idea, but
she hadn't considered those other kinds of information sources.
Daniela is a co-investigator on D'Agents. She described her interest
both in information access, and robotics. The general idea behind
using mobile agents for information access is bringing the computation
(search in this case) to the data. I noted the need to also consider
arguments such as those used by the people who built page-server OODBMS
architectures, namely that off-loading computation from servers to clients
created better (and often fairer) load distribution (particularly when
the clients are PCs, which are becoming more powerful all the time.
One of the advantages of an architecture supporting highly flexible mobility
(including agent capabilities) is that it would allow all sorts of load
adjustments based on whatever the situation required, rather than imposing
a single model. We discussed some other scenarios in which agents
would be useful, e.g., mobile platforms with intermittent connections,
where there were fewer alternatives to agents.
George Cybenko is the TIE coordinator for the CoABS Scalability TIEs.
He is very interested in technology exchange with ALP, and feels that ALP
can provide potential scenarios and issues for his TIE. ALP is particularly
relevant because of the way it uses clusters to model organizations.
He has noted that a hierarchical command structure is a realistic basis
for estimating potential agent deployment scaling needs, since the command
structure helps pre-determine the number of agents, and the number of agents
each other agent might have to talk to. He expressed interest
in any projections ALP had as to the ultimate numbers of clusters, plugins,
etc. to be developed. I showed him the configuration of the clusters
for the 1998 demonstrations (which I was using in my talk), and said that
I'd heard some other numbers, but couldn't recall them offhand. When
I returned from the trip, I contacted Marshall Brinn, who sent me some
numbers, which I forwarded to George. [The figures are shown below.
Marshall said they may be off by a factor of two or so, but not by orders
FY97 FY98 FY99 FY00 FY01
Site 3 5 7 10 15
Cluster 12 76 125 170 250
Plugins/Generic 2 15 30 45 60
Plugins/Specific 24 87 1200 1700 2500]
George also talked about the work they are doing in mobile computing
in the D'Agents project. One of the systems they are using is called
RoamAbout, which bridges local wireless networks to a wired Ethernet LAN,
with PCMCIA cards providing the wireless LAN connections for laptops.
They are talking about buying some radio spectrum to do more intensive
experiments. I also spoke with a number of George's students, who
described the work they were doing.
My talk (basically to the D'Agents team) was entitled "Scalability in Large-Scale
Agent Systems". Since Dave had originally asked me to talk about
"what I was working on", after a brief overview of the various components
of the Agility project, and what we thought agents were (objects with additional
capabilities of various kinds), the talk broke down into two main sections:
the Grid, and the CoABS-ALP technology exchange work. I briefly surveyed
the various sources of "Grid" ideas (e.g., the computational grid and DoD
grid ideas described in my Grid
white paper), and some target capabilities for the CoABS Grid (from
the initial look-ahead paper). I noted that the actual work on the
CoABS Grid was pursuing a general line that might be described as an "agentized
[OMG] Object Management Architecture" (agents + services), and that it
needed to be thought of in terms of multiple layers of grid-like things
(the computational, data, object, and agent grids described in the white
paper). This would, for example, provide the agents with access to
the computational infrastructure they needed for such things as load-balancing,
as well as the access to large-scale data sources (databases, the Web)
they needed to be effective ("The Internet gives agents something to do").
I pointed folks to the Grid papers and other material on our Web site.
I then gave a brief overview of ALP's approach to building a multiple agent
system (covering the logistics problem, cluster organization, and both
inter- and intra-cluster interactions), and a comparison of ALP to a Grid.
I also noted some ways in which ALP could be enhanced to become more "agentized",
reviewing some of the topics mentioned in my Technical Exchange Area proposals.
Finally, I made some general remarks about lessons that could be learned
about building multiple agent systems from having looked at the Grid and
ALP, most especially:
Agent systems need to explicitly consider how they will provide access
to data (databases, the Internet) and "conventional" distributed object
systems (since agents won't do everything, and need to be able to interoperate
with these other systems in order to be effective).
Real systems have a lot of built-in structure. For example, ALP illustrates
the fact that real organizations have already worked out how they are going
to do the work they spend most of their time doing, and so people building
agent systems should not so concentrate on the dynamic aspects (e.g., matchmaking)
that they overlook facilities for doing this other stuff well.
There is a need to look carefully at the detailed assumptions built into
agents. For example, ALP clusters don't assume there is such a thing
as a matchmaker or trader component in the society (architecture), and
so it is not enough to simply add such a component to a society and provide
a way for clusters to communicate with it; the other clusters would
simply ignore it. Instead, logic has to be built into the clusters
to make use of matchmaking services (at points which the cluster deems
appropriate). In many other agent architectures, all agents assuming
there is such a component. Deciding what these built-in
universal assumptions are is part of, for example, defining the Grid, since
such assumptions implicitly define a minimum set of services that must
D'Agents project information can be obtained at http://agent.cs.dartmouth.edu/.