Object Services Architectures
Natural Language Interface
Gil Hansen and Craig
Object Services and Consulting,
September 15, 1998
Accessing task specific data in command and control information systems
has always been hampered by having to hand translate the request into system
specific, complex query and command languages. The need to learn these
languages and perform the hand translation can be eliminated by using a
natural language system that directs the user in the composition of only
legal queries and commands and then translates them to the target query/command
languages. The NLI project provides this missing capability.
In the OSA project we used and improved the NLI prototype to access
metadata repositories. It has been applied to the OSA/Weather project
to query a local performance metadata repository. This demonstrates
a high level user friendly interface to service specific metadata and can
be viewed as an optional but useful component in the service support infrastructure
of the OSA Internet Services Architecture.
Accessing data in enterprise and command and control information systems
requires forming the query or command in a specialized language that must
be learned. A more natural approach is to express the query or command
in English and automate the translation process. Many forms of user
interface technology are in everyday use (e.g., command lines, menus, GUIs)
but natural language has always been waiting in the wings, the modality
of choice for people-to-people communication but very ineffective in man-machine
interfaces. This in not for want of trying – many experimental NLI interface
systems have been built, and a few have been marketed, but none are in
wide-spread use. A variety of interface technologies exist for asking simple
queries, filling in forms, filtering email, directly manipulating objects
graphically, and even constructing applications using drag and drop components.
But there is a significant gap when it comes to constructing complex queries
and commands. SQL for databases and scripting languages for various applications
make this capability possible. But there is so far no good way to make
this easy for non-expert users. Humans use natural language to communicate
Conventional natural language interface technology has not succeeded
in delivering an easy to use, cost effective, componentized capability.
Problems with conventional natural language as an interface technology
End-users cannot easily learn the coverage of conventional NLI systems
and so they very often type in (or speak) queries or commands that overshoot
or undershoot the NLI system's capabilities. Overshooting means asking
queries that the system is not programmed to understand; undershooting
means not realizing that the system can answer questions about areas the
system knows about (e.g., requests to view maps or simulations). Lexical,
syntactic, semantic and pragmatic mismatches occur very often in even the
best conventional NLI systems.
Conventional NLI systems are expensive to develop and maintain because
they must cover many ways a user might paraphrase a question or command.
This project addresses the problem of assisting the end user in specifying
effective, intuitive information queries and commands targeting multiple
information systems. The resulting system can be used as a general
purpose natural language interface component, but is specifically applied
in the OSA to provide a way for end users to access service specific metadata
in an Internet Services Architecture.
The basic approach of the NLI prototype is to use the idea of completion
and a look-ahead parser that predicts next legal words or phrases and to
(optionally) show these to the user using menus or some similar display,
at the same time checking that if the user types input that simultaneously
the input is checked against the grammar to insure the system understands
the input. When well-formed queries or commands are entered, they
can be translated to some query or command language and executed.
The current NLI prototype is an extension of an earlier one, which removes
The user interface was rewritten to use cascaded menus instead of a constellation
of menus to conserve screen real estate.
A query/command editing facility was added. In the NLI, a user
can type in a phrase at any point within an incomplete query/command.
This required adding phrase verification which checks if the (partial)
phrase is incorrect or incomplete. Incomplete phrases may be completed
by using a popup menu which only lists the legal phrases that would make
it complete. Incorrect phrases must be corrected by editing or erased.
Much of the current system is written in Java so it is more portable than
before. The earlier system depended on UNIX and X-Windows and to
Oracle DBMS systems. To make NLI portable to all platforms it was
written in Java and the Java
Foundation Classes were used to implement the user interface. To access
any database, JDBC
The ability to retarget user interfaces quickly was demonstrated by interfacing
NLI to the metadata repository of a service in the Internet Services Architecture.
The Lisp interpreter, originally written in C, was re-written in Java.
However, because of time limitations, the parser was not resulting in a
hybrid system consisting of Java and native executable code. Also because
of time, only a type-in expert was implemented, but other experts can easily
NLI also allows custom interfaces to a particular database schema to be
built using a interface definition specification and the same underlying
Sockets replace an earlier file-based scheme for interprocess communication
(i.e., passing Lisp s-expression back and forth). This has the added
advantage that the user interface and parser no longer need to reside on
the same machine. With some additional work, a remote parser could service
NLI is based on an earlier prototype system written mostly in C.
The user interface of the current NLI was written from scratch and presents
end-users with a series of cascaded menus each of which contains the set
of next legal phrases. The Lisp interpreter and parser modules of the earlier
system were utilized. Development of the current prototype was in
Java and C. The following software was used in the development of
Java code is compiled using Sun's JDK
1.1.6 and debugged using IBM's Jikes
C code is compiled and debugged using GNU-Win32
b19, a UNIX development environment that sits on top of NT and generates
The NLI GUIs are implemented using Sun's Java Foundation Classes (JFC
1.1 to be released in JDK 1.2) which is based on the Java
AWT and is also known as Swing.
All of the code executes under Windows NT 4.0, SP3 and relies on windows
sockets. NLI will NOT currently execute under Windows 95, because the socket
code causes the native code to core dump.
Limitations and Next Steps
The existing prototype demonstrates the ideas behind NLI and insures a
basic capability can be demonstrated immediately, but capability extensions
are needed in the following areas:
Work is needed on human factors to quantify and remove limitations of predictive
natural language interface technology. Limitations can accrue from large
menus (users can get lost) or from the left-to-right completion order.
Also, the technology can be used for asking very complex questions that
might take a database hours to respond to, so means of restricting grammars
via performance constraints will have to be developed for large databases.
This is actually a DBMS technology limitation but one that continued NLI
technology development can address.
Extend NLI to mixed modality interfaces.
Extend the coverage.
Large DBMS Interfaces.
One interface to multiple data sources.
NLI provides a basis for further research in the use of a natural language
interface for enterprise and command and control information systems. It
is a fully functional prototype. If its capabilities are extended as stated
in the Next Steps, it can, for example, assist the commander in the next
generation Command Posts in specifying effective, intuitive information
queries and commands targeting multiple applications and data sources.
NLI can assist in attaining the goal of small, mobile command posts.
is sponsored by the Defense Advanced Research Projects Agency and managed
by the U.S. Army Research Laboratory under contract DAAL01-95-C-0112. The
views and conclusions contained in this document are those of the authors
and should not be interpreted as necessarily representing the official
policies, either expressed or implied of the Defense Advanced Research
Projects Agency, U.S. Army Research Laboratory, or the United States Government.
© Copyright 1998 Object Services and Consulting,
Inc. Permission is granted to copy this document provided this copyright
statement is retained in all copies. Disclaimer: OBJS does not warrant
the accuracy or completeness of the information in this survey.
Last revised: September 15, 1998. Send comments
to Gil Hansen.