This is a description, as of 12/31/96, of how OBJS uses electronic support for collaboration and decision making, why we did it that way, how it actually worked, and what we would like to see different in the future. It is told somewhat from my personal perspective, particularly the "how well it worked" part; others may have different opinions. A lot of this sounds like "common sense", but it is useful to document this as a confirmation or refutation of "conventional wisdom", particularly since our virtual office made us rely almost exclusively on electronic distributed collaboration. As expected, there are both positives and negatives. What was unexpected, at least to me, was how much the use of electronic collaboration exaggerates existing tendencies.
We wanted to support distributed:
It was important that the different activities be able to either be supported by the same tools, or that there be a relatively smooth transition back and forth between the tools when changing activities. This seemed to imply the use of either a single suite of tools for all activities, several more or less coordinated sets of tools, or a common information format that could be interpreted by different tools in a consistent, but task-specific way. We ended up using a common set of tools for all tasks in the initial attempt.
It was also important that any technology work on the platforms we had or were likely to have. These were:
To get ourselves going quickly, we wanted to use tools that either had a very short learning curve or with which we (at least most of us) were already familiar. Related to this, we wanted to use tools that were either the same or inter-operated with other tools we used for other purposes. In particular, we did not want to adopt collaboration tools that required us to adopt that tool as the center of our universe. There were three reasons for this: we were not convinced from our readings that any existing tool was good enough to justify that level of commitment, the more popular tools (e.g., Lotus Notes) were very expensive and required a big learning and support commitment to use effectively, and we needed to be able to share with others who had not adopted such a tool. We also wanted to initially try to use tools that either we already had, were free, or were low cost, since we were not exactly sure what we wanted.
We currently use a combination of phone, email, Web pages, and MS Office products including Word, Excel, and PowerPoint. These were used as described below, although not in a completely consistent fashion; i.e., we mostly did it this way, but there was no rigid control of the process and no penalty for operating outside this model.
The primary communication is via email. Most people are using the email reader that comes with NetScape; others use Eudora Pro or roughly equivalent readers. Everyone keeps his/her own email archive; there is no central repository of email.
Email subject lines are assigned in reasonable, but not well-defined fashion. For threads of responses, the subject carries over from message to message, giving the email reader a way to track message threads by comparing subject line strings. OIDs are embedded in messages as part of "respond-to", so you can traverse backwards up a message chain, but this is invisible from the "thread" display in the summary listing. Of course it is not possible to go forward to messages that respond-to the current message.
When messages are responded to, the original message is included in the response. Sometimes parts of the message are elided, but this is not done consistently. Included messages are indented by the mailer.
Attachments to email are mostly Word documents, although sometimes HTML, Excel or PowerPoint are also attached. URLs to interesting Web sites are often embedded in messages and can be directly traversed from there. URLs in attachments are handled less regularly, since some people wrap them in HTML (either directly or by using an add-in like Internet Assistant), while others just send strings. We make fairly extensive use the Revisions Tool in Word to differentiate edits of various participants in the creation of a Word document.
Completed documents eventually end up as Web pages linked to the OBJS home page.
This section identifies what worked and what did not.
The use of email for brainstorming about a technical work plan worked very well. In a 6 day period (including a weekend when only some people were active), 86 messages related to the brainstorming topic were sent, of which 64 were substantive technical contributions and 22 were "procedural". Technical messages fell into several categories: new ideas, pointers to possible related work, resurrections of our existing papers and ideas, and elaborations and rebuttals of previous messages. As such, it was not a "classical" brainstorming, since ideas were taken considerably further than one would expect in a brainstorming. However, the distributed, asynchronous nature of the interaction meant that the group discussion did not stop while someone was thinking or looking something up. This was a major advantage in getting a lot of relatively well conceived and documented ideas out quickly, and perhaps is a better model for distributed brainstorming than the traditional model, which is targeted at groups collocated in time and space where events are sequential and documentation access is limited.
There was a tendency toward rambling and divergence. It is not clear what to do about it or even if something should be done about it as it occurs in a real brainstorming too. Perhaps the asynchronous aspect of the brainstorming makes it worse, since there is no immediate feedback that it is happening.
When moving from brainstorming to analysis, the collection of email messages was sorted manually into several "topic areas" determined by a manual, coarse reading of the individual messages. For each topic area, a list of messages falling into that category was made. In the lists, messages were identified by their sender and message timestamp, and a summary of the main topics of each message was appended. No analysis took place at this time. Messages could reside in more than one category, although most did not. Reading the messages for content was an inevitable time cost; manually making the lists and extracting the timestamps was cumbersome. The whole process took about 2 man-days. This categorization by itself added no new knowledge, since it was simply a rough sort of messages everyone had already seen. Its intent was to partition the discussion into manageable sized groups and topics, with the useful result being the eventual production of a coherent presentation of the topic by the group. As such, it would have been better to have started with a tool that supported email with fields and a reader which could have multiple views and sort orders. Given that it would be easy to build "brain-storming" forms and views. Notes and Exchange both do this.
Email worked best when generating ideas or references, and progressively less well for evaluation or document production. This was primarily because generating ideas does not create deeply nested sets of responses, whereas discussion does. Deeply nested discussions in email threads are hard to follow. It is hard to keep track of who said what, since depth of nesting is the only cue presented as to the author in at least the standard mail readers we used. Merging of branched threads are not supported at all in the tools we use, so if there are asynchronous responses to a message, no message ends up with all comments in it due to the branch. The protocol of embedding previous messages into a response message gives all context with a message, but creates very long messages in which it is tedious to find the new material. It also tends to clog mailboxes and create download problems if messages get too large or deeply nested. Some people try to overcome this by eliding parts of previous messages, but this creates problems for someone newly arrived to the thread, since they will not have access to older messages.
Attachments in something like Word with a Revisions feature somewhat alleviate this problem, since it becomes possible to see both the current document and the revision history by flipping modes. The revisions feature also gives much richer visual cues like color, strike-through, change bars, etc. However, this does not solve the problem entirely. Multiple consecutive revisions by the same person are not automatically coded with different colors, so they become indistinguishable. Also, two revisions by the same author appear completely unrelated if there has been an intervening edit by someone else; there is no color cueing. The problem is that color codings are assigned based on revision order rather than consistently by person (it would make more sense for David to always edit in green rather than green simply being the third person to edit a document). Of course, because the documents are attached to email, the branching version problem from email exists here as well. There is some support for merging versions, but we have not used it so far. In any event, the merging is whatever MS offers, which is not open to different merging policies.
Movement between messages in a thread is not all one could hope for. With the NetScape mail reader, there are two ways to move between messages: by moving around in a thread as shown in the mail summary window (when threading is turned on), or by traversing links to messages embedded within the mail header (visible when full header details are shown). Threads are determined by text in the subject line, so all messages in a thread must have the same subject. This makes it difficult to decide which message in a thread to look at, since in the summary they all look more or less the same. This also makes it impossible to create a clearly identified new thread off of an existing thread. For example, suppose a discussion thread about "Queries" develops a sizable discussion about "Indexing". The options are to either leave the two related discussions together in the same message stream, which is very unwieldy and certainly does not scale, or to create a new thread with a new name, in which case the "Indexing" thread appears unrelated to the "Query" thread where the initial part of the discussion resides. This effectively prunes that part of the discussion. The alternative of traversing by embedded links has the disadvantage that only backward traversal is possible because messages obviously have no links embedded in them to future messages. Links point to specific messages by a kind of "message UID", which is unrelated to the subject line information used to display threads. This could support multiple threads based on connectivity defined in various interesting ways by the user or mail system, but this does not appear to be the case at present.
Attachments are not automatically forwarded as part of a response. This makes them somewhat second class, and they are sometimes omitted. Revisions to attachments are not handled gracefully, since they must be saved locally and then reattached to the outgoing message. In this process, they appear to be a different file when they return to the original sender, leading to version control problems in the file systems on the various machines. Naming conventions with respect to these various files are insufficient to keep them differentiated, yet related.
Because so many documents contain HTML links, it would be preferable if they were uniformly treated as HTML rather than sometimes as strings. The problem at least in Word, is that to get HTML-mode with Internet Assistant, requires giving up a lot of the formatting and graphics available in Word. In addition, not all the tools that work with standard Word format work in HTML format; in particular, the Revision Tool does not work with HTML mode. Thus, one must have decided fairly early whether the primary display medium for a given document will be as a Web page or as a Word document in order to get the desired handling.
An organizational problem enabled by the use of email is for all discussions to go to all people. There is somewhat of a feeling that if the information is available, it is unprofessional to not look at it, as it may be important. The result is the electronic equivalent of a group of people sitting in a room all talking at the same time about slightly different topics. The use of email for discussion exacerbates this because it arrives when sent; thus you either get it right away or not at all. The mail readers most of us use do not support filtering or even nested folders, both of which would reduce the problem. The issue also occurs when registered on mailing lists, in that such "mailing list" mail arrives and is displayed with the same priority as time-critical mail.
Our use of email creates several types of information overload other than the obvious one that a lot of information being discovered and disseminated, The lack of good filtering in the mail reader means that every message is potentially relevant and must at least be looked at. This bothers some people a lot more than others; I am particularly sensitive to the fragmentation it causes. Our current lack of standard mail topics (e.g., ADMIN) and our use of email for practically everything means that technical discussions show up in a mailbox along with lots of other mail with different priorities and values. Combined with our lack of targeted mailing lists for particular topics due to the desire of many people not to miss potentially relevant discussions means that not only does almost every message get delivered to every person, but the messages must be sorted more or less manually. At the same time, because there is no email "archive", if you do not get the message initially, there is no convenient way to go back and get it if it later appears to be relevant.
Documentation of discussion helped me a lot. Not only am I a lousy note taker, but I find that generally when there are multiple sets of notes taken, unless they are reconciled soon after the meeting, they represent different, and often contradictory, "facts" about what happened. The electronic trail keeps a record of what people actually said, rather than what someone else wrote down about what they think was said.
Phone conversations are a good augmentation. My personal experience is that I most frequently call those people with whom I have some kind of social contact outside of the job. It would be interesting to see if others have the same experience. It would be consistent with what is reported on the JTF/ATD about how military planners actually work when they need to get something done quickly (presentation at IC&V PI Meeting , Oct 1996). A big problem is getting notes of side channels such as phone or face-to-face conversations into the log. Because so much is documented via email and document exchange, it is often assumed that everything is, so the lack of written documentation is more critical than usual when it is known that a lot of material exists that is not written down.
We tried NetMeeting, CoolTalk, and CU-SeeMe Internet conference systems. These are still more of a novelty than a reality. Audio quality over the Internet is still terrible. Video, at the level we have experienced, is simply inoperable. "Talk" still works well and now there are good programs for cooperative sketching over the Internet, but email is not much slower than talk and it is more reliable and more flexible.
At a somewhat more generic level, lack of good searching and sorting tools is a problem as it is with everyone else. Much of this is documented elsewhere as part of the discussion of the Semi-Structured Information Space Search Service (S2IS3) and is not repeated here. However, it should be noted that the capabilities of the S2IS3 should be applicable to the information we produce during brainstorming and discussion in the same way it applies to Web searching.
The asynchronous nature of electronic collaboration has many advantages regardless of the particular tools used. Of considerable value is that the built-in delay provides an opportunity to gracefully reflect before making a response. Thinking is always useful. One can temper remarks that may be taken either the wrong way or out of context, look up corroborative detail, and talk to others who are not directly part of the discussion. The flip side of this is of course that it is hard to bound a delay or know why it is occurring. A protocol for ensuring that responses are (or are not) forthcoming is particularly important in a distributed setting since it is not always clear that a message has been delivered, or if delivered, actually read. (A lot of mailers support return receipts.)
The downside of built-in delays is that it makes it harder to reach closure on an issue, since the added time tempts people to defer decisions until ALL the facts are in and everyone's individual comfort level has been reached. This can also be a problem in face-to-face meetings, but I think it is worse in our situation. Part of the problem is organizational. Technology could help by having a way to better track outstanding items and pester people to respond.
Being in a physically different place allows one to keep track of a discussion and still do other work. Ignoring irrelevant discussion helps me a lot, since I frequently don't need a lot of context-setting since I've been working in this area for a long time. The ability to let everyone spend as much time coming up to speed as they individually need is helpful. A problem is that we have no good facility for asking public questions of the "speaker" electronically. I don't know how to achieve this since a public question in a room is heard by all, even those not directly participating, and they may comment or learn some new angle. But in general, I find the ability to gracefully ignore things to be a good trade-off. Another difference between a "live" question and its electronic equivalent is that live questions go away once answered, while the electronic equivalents persist, drawing responses long after they have either been answered or have become irrelevant.
I don't find lack of social contact a problem, but then I never got my social life from work anyway; for others, this may be more of a problem, and VO is probably not for them. A more concrete problem is the lack of spontaneous discussion and cross-fertilization that occurs when people have casual conversations about their work or their problems. Perhaps this is not a problem for many people, but I find I do not carry on casual conversations on the phone or by email. Perhaps I am a closet Luddite, but I do find a difference. I do not know if high quality video would help. This is not the same as collaboration on a specific task, which I find works quite well in a distributed setting.
The distance in time and place between people I work with has pluses and minuses that have been discussed widely, mostly speculatively. I don't think it is the case that you can have all the pluses without some of the minuses no matter what the setting. My personal take is that they weigh in a little bit on the positive side for virtual offices, and could be tipped further that direction with a bit of better organization.
This section identifies next steps that we can take based on available tools or simple extensions.
A reader that made more effective use of existing message UIDs would be useful, in that it would allow much more effective threading of messages. Multiple threads could be supported. To really make use of this would require some kind of user interface for manually linking messages similar to the way Web pages are linked. These UIDs should be exposed outside the mailer so that messages could be linked to and from other kinds of documents.
A central archive with the ability to download messages selectively via either server-side filters (queries) or UIDs would go a long way to solving many of the information overload problems caused by our present system. This would allow local caches of mail to be maintained for user convenience, but would also allow messages to be obtained whenever needed, including very long after they were sent if someone wanted to join a discussion or look at backup material for a paper that exists in the form of email. It would enable "retractions" of messages ("oops, the message got out without its attachment"), time sensitive messages ("I will be gone this afternoon"), and "anti-messages" ("the meeting has been rescheduled") to be handled appropriately depending on when they are actually read.
Client-side filters would add the ability to categorize messages in multiple, flexible ways. Hierarchical mail folders would support much better arrangement of mail than we currently have. It would definitely be advantageous to be able to filter based on message content as well as headers. Searching within a folder, categorizing into folders via filters, and selectively retrieving via filters should be supported with the same interface and same power of predicates. Note: the next generation of mail readers apparently will provide much of this; Eudora Pro may provide some of it already.
The separation between email, attachments, Web pages, and the rest of the file system is an artificial barrier that should ultimately be eliminated. Regardless of how attachments are actually transmitted, it should be possible in the archive to have references to documents created using other tools, rather than having the document embedded in a message. This would allow a single copy of the document to be accessed from multiple places, would take care of a lot of our gratuitous branching version problem (artifacts of the embeddings rather than real branches that have semantic or operational meaning). If references to stored documents were treated as attachments by the mailer or mail reader, it should be possible to reconcile the UID of the document so that if it passed through several people it would not lose its identity (although it might create new versions). This would also eliminate the need to explicitly store the document locally in order to manipulate it; it could be reached in a variety of ways supported by an Internet OSA.
Another aspect of unifying the various document styles is that tools should be uniformly applicable; for example, the Spell Checker should be able to be used on email messages, including headers. This would probably be harder to do, since such tools have more knowledge about document format, but is a worthy goal. (This works for Exchange, but then that is within the same product line)
Periodic coalescing of discussion threads would be of immense value. It would eliminate deep nesting of discussion which is hard to follow, would provide entry points into discussions for new joiners, and would serve as intermediate steps toward a final document. This seems to indicate that the coalescings should be in a form other than email, since they will reflect more thought and are intended for a wider audience than the message traffic itself. At the same time, it should be possible to link back to the message thread(s) that were used in creating the coalescing, and it should be easy for new threads to emanate from specific places within the document. This again argues that mail messages should be first class objects on an equal footing with other objects in the file system.
All of this argues that email is simply another way to populate and view a complex document space, with the added characteristic that when in the "mail view", information can be pushed as well as pulled. What we eventually want is some sort of Semantic File System (or Object File System), would still be compatible with existing mailers and readers, would be compatible with users outside its domain who had to rely on current practice, and would allow us to clear up most of the problems we encountered. Clearly a full-blown SFS is not necessary to support the functionality described above. Even without anything fancy like a SFS, a central repository would help.
To meet our most immediate needs and to better understand our long term needs, by January 1997 we had established an OBJS-wide news server. Newsgroups and mailing lists are reciprocally (dual) gatewayed, so that messages sent to a gatewayed mailing list end up with the news server. Thus news can be posted using the same interface as sending email. Messages sent to individuals are delivered as usual, as are messages sent to mailing lists; in addition, messages sent to mailing lists that are gatewayed to newsgroups appear on the news server. In combination, this makes it possible to get messages immediately or deferred based on personal preference, and to read archived messages that were not originally addressed to you. Threading is the same in news as it is in the email reader.
Because news is centrally served, individual email files can be much smaller than they previously were. My personal experience with this is that because email is so critical to every phase of OBJS operations, my mail file reached 29MB, which, since my mail reader loads the entire file at once, caused severe paging problems. A central facility should alleviate this problem. (Note that so would stronger support for folders and filtering, since that would partition the mail into smaller, more manageable chunks. Additional memory on my personal machine would also solve the paging problem for a while, but unbounded mail file sizes can still exceed available memory, and some machines, such as laptops, have limits on the amount of memory they can accept. Cost and power consumption are also issues when adding memory.)
To take full advantage of the capabilities offered by the system described above requires upgrading to the newest generation of news readers which support retrieval by query/filter as well as message ID, and local caches. The ability to maintain local caches of interesting news is important in order to mirror what we current have with local email archives. At our level of connectivity, reading from a central server is not really a problem, but would be significant if there were frequent communications outages or a lot of travel that put people at the end of a low bandwidth line.
Even with the ability to retrieve by query/filter, the choice of newsgroups and subject lines for email messages may be important, since otherwise getting the desired messages in a timely fashion turns into a generalized information retrieval problem instead of being under the control of whoever posts messages. We currently plan on having newsgroups for a small number of topics, to agree on a labeling scheme for time-critical messages (e.g., Subject: **READ NOW**), and to be judicious in how many messages are sent to groups of people. We have no experience yet with how well this will work, either in terms of how to split up newsgroups, desirable labeled message categories, or how well people will adhere to the protocols being established.
This research is sponsored by the Defense Advanced Research Projects Agency and managed by the U.S. Army Research Laboratory under contract DAAL01-95-C-0112. The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied of the Defense Advanced Research Projects Agency, U.S. Army Research Laboratory, or the United States Government.
© Copyright 1996 Object Services and Consulting, Inc. Permission is granted to copy this document provided this copyright statement is retained in all copies. Disclaimer: OBJS does not warrant the accuracy or completeness of the information in this survey.
This page was written by David Wells. Send questions and comments about it to wells@objs.com.
Last updated: 1/9/97 sjf
Back to Internet Tool Survey -- Back to OBJS