Object Services Architectures

Natural Language Interface

Project Summary

Gil Hansen and Craig Thompson Object Services and Consulting, Inc. September 15, 1998

Executive Summary

Accessing task specific data in command and control information systems has always been hampered by having to hand translate the request into system specific, complex query and command languages. The need to learn these languages and perform the hand translation can be eliminated by using a natural language system that directs the user in the composition of only legal queries and commands and then translates them to the target query/command languages. The NLI project provides this missing capability.

In the OSA project we used and improved the NLI prototype to access metadata repositories. It has been applied to the OSA/Weather project to query a local performance metadata repository. This demonstrates a high level user friendly interface to service specific metadata and can be viewed as an optional but useful component in the service support infrastructure of the OSA Internet Services Architecture.

Problem Statement

Accessing data in enterprise and command and control information systems requires forming the query or command in a specialized language that must be learned. A more natural approach is to express the query or command in English and automate the translation process. Many forms of user interface technology are in everyday use (e.g., command lines, menus, GUIs) but natural language has always been waiting in the wings, the modality of choice for people-to-people communication but very ineffective in man-machine interfaces. This in not for want of trying – many experimental NLI interface systems have been built, and a few have been marketed, but none are in wide-spread use. A variety of interface technologies exist for asking simple queries, filling in forms, filtering email, directly manipulating objects graphically, and even constructing applications using drag and drop components. But there is a significant gap when it comes to constructing complex queries and commands. SQL for databases and scripting languages for various applications make this capability possible. But there is so far no good way to make this easy for non-expert users. Humans use natural language to communicate such needs.

Conventional natural language interface technology has not succeeded in delivering an easy to use, cost effective, componentized capability. Problems with conventional natural language as an interface technology are:

End-users cannot easily learn the coverage of conventional NLI systems and so they very often type in (or speak) queries or commands that overshoot or undershoot the NLI system's capabilities. Overshooting means asking queries that the system is not programmed to understand; undershooting means not realizing that the system can answer questions about areas the system knows about (e.g., requests to view maps or simulations). Lexical, syntactic, semantic and pragmatic mismatches occur very often in even the best conventional NLI systems.
Conventional NLI systems are expensive to develop and maintain because they must cover many ways a user might paraphrase a question or command.

Objectives

This project addresses the problem of assisting the end user in specifying effective, intuitive information queries and commands targeting multiple information systems. The resulting system can be used as a general purpose natural language interface component, but is specifically applied in the OSA to provide a way for end users to access service specific metadata in an Internet Services Architecture.

Approach

The basic approach of the NLI prototype is to use the idea of completion and a look-ahead parser that predicts next legal words or phrases and to (optionally) show these to the user using menus or some similar display, at the same time checking that if the user types input that simultaneously the input is checked against the grammar to insure the system understands the input. When well-formed queries or commands are entered, they can be translated to some query or command language and executed.

Results

The current NLI prototype is an extension of an earlier one, which removes several limitations:

The user interface was rewritten to use cascaded menus instead of a constellation of menus to conserve screen real estate.
A query/command editing facility was added. In the NLI, a user can type in a phrase at any point within an incomplete query/command. This required adding phrase verification which checks if the (partial) phrase is incorrect or incomplete. Incomplete phrases may be completed by using a popup menu which only lists the legal phrases that would make it complete. Incorrect phrases must be corrected by editing or erased.
Much of the current system is written in Java so it is more portable than before. The earlier system depended on UNIX and X-Windows and to Oracle DBMS systems. To make NLI portable to all platforms it was written in Java and the Java Foundation Classes were used to implement the user interface. To access any database, JDBC was used.
The ability to retarget user interfaces quickly was demonstrated by interfacing NLI to the metadata repository of a service in the Internet Services Architecture.
The Lisp interpreter, originally written in C, was re-written in Java. However, because of time limitations, the parser was not resulting in a hybrid system consisting of Java and native executable code. Also because of time, only a type-in expert was implemented, but other experts can easily be added.
NLI also allows custom interfaces to a particular database schema to be built using a interface definition specification and the same underlying grammar.
Sockets replace an earlier file-based scheme for interprocess communication (i.e., passing Lisp s-expression back and forth). This has the added advantage that the user interface and parser no longer need to reside on the same machine. With some additional work, a remote parser could service multiple users.

Enabling Technology

NLI is based on an earlier prototype system written mostly in C. The user interface of the current NLI was written from scratch and presents end-users with a series of cascaded menus each of which contains the set of next legal phrases. The Lisp interpreter and parser modules of the earlier system were utilized. Development of the current prototype was in Java and C. The following software was used in the development of NLI:

Java code is compiled using Sun's JDK 1.1.6 and debugged using IBM's Jikes debugger.
C code is compiled and debugged using GNU-Win32 b19, a UNIX development environment that sits on top of NT and generates native executables.
The NLI GUIs are implemented using Sun's Java Foundation Classes (JFC 1.1 to be released in JDK 1.2) which is based on the Java AWT and is also known as Swing.
All of the code executes under Windows NT 4.0, SP3 and relies on windows sockets. NLI will NOT currently execute under Windows 95, because the socket code causes the native code to core dump.

Limitations and Next Steps

The existing prototype demonstrates the ideas behind NLI and insures a basic capability can be demonstrated immediately, but capability extensions are needed in the following areas:

Extend NLI to mixed modality interfaces.
Extend the coverage.
Large DBMS Interfaces.
One interface to multiple data sources.
Cooperative Response.

Work is needed on human factors to quantify and remove limitations of predictive natural language interface technology. Limitations can accrue from large menus (users can get lost) or from the left-to-right completion order. Also, the technology can be used for asking very complex questions that might take a database hours to respond to, so means of restricting grammars via performance constraints will have to be developed for large databases. This is actually a DBMS technology limitation but one that continued NLI technology development can address.

Impact

NLI provides a basis for further research in the use of a natural language interface for enterprise and command and control information systems. It is a fully functional prototype. If its capabilities are extended as stated in the Next Steps, it can, for example, assist the commander in the next generation Command Posts in specifying effective, intuitive information queries and commands targeting multiple applications and data sources. NLI can assist in attaining the goal of small, mobile command posts.

This research is sponsored by the Defense Advanced Research Projects Agency and managed by the U.S. Army Research Laboratory under contract DAAL01-95-C-0112. The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied of the Defense Advanced Research Projects Agency, U.S. Army Research Laboratory, or the United States Government.

© Copyright 1998 Object Services and Consulting, Inc. Permission is granted to copy this document provided this copyright statement is retained in all copies. Disclaimer: OBJS does not warrant the accuracy or completeness of the information in this survey.

Last revised: September 15, 1998. Send comments to Gil Hansen.