Skip to main content

Full text of "DTIC ADA414209: Metareasoning for More Effective Human-Computer Dialogue"

See other formats


REPORT DOCUMENTATION PAGE 


Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching t 
the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions 
Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork F 


AFRL-SR-AR-TR-03- 

; onC 


1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE , I REPORT TYh_ „,.p vnrts COVERED" 

__I_I_ 01 JUL 00-31 DEC 02 

4. TITLE AND SUBTITLE “ S. FUNDING NUMBERS 

METAREASONING FOR MORE EFFECTIVE HUMAN-COMPUTER DIALOGUE F49620-00-1-0336 


I 3. REPORT TYh „ „,.o unrts COVERED 


6. AUTHOR(S) 
DON PERLIS 


7. PERFORMING ORGANIZATION NAMEIS) AND ADDRESS(ES) 
UNIVERSITY OF MARYLAND 
COLLEGE PARK, MD 20742 


PERFORMING ORGANIZATION 
REPORT NUMBER 


9. SPONSORINGIMQNITORING AGENCY NAME® AND ADDRESS(ES) 
AFOSR/NM 

4015 Wilson Blvd, Room 713 
Arlington, VA 22203-1954 


10. SPONSORINGIMONITORIN 
AGENCY REPORT NUMBER 

F49620-00-1-0336 


11. SUPPLEMENTARY NOTES “ 


12a. DISTRIBUTION AVAILABILITY STATEMENT ~ 

1 12b. DISTRIBUTION CODE 


APPROVED FOR PUBLIC RELEASE, DISTRIBUTION UNLIMITED 


13. ABSTRACT (Maximum 200 words) ~~~ 

The research project explores specific meta-dialogue behaviors in terms of both how a system could be made to perform 
them, and to what extent they can increase overall system performance. We focus on two types of metadialoglle capabilities: 
ability to detect and recover from anomalous dialogue patterns in simple exchanges, and on-line extensions or changes to 
working vocabulary. 

Our main method involves detailed representation of the dialogue context, separating domain, language, and dialogue specific 
aspects, and significant amounts of meta-reasoning about the system's processing of these representations. An existing logical 
inference system, ALMA/CARNE, developed as part of a pilot study, is being used in an implementation phase of this work. 
We are also engaged in a study of existing dialogue corpora to investigate the range and frequency of meta-dialogue 
expressions in different task domains. 


14. SUBJECT TERMS 


17, SECURITY CLASSIFICATION 
OF REPORT 


20030520 


15, NUMBER OF PAGES 
7 


18. SECURITY CLASSIFICATION 
OF THIS PAGE 


SECURITY CLASSIFICATION 
OF ABSTRACT 



Standard Form 298 (Rev. 2-89 {EG 

Prescribed by ANSI Std, 239,18 

Designed using Perform Pro, WHSfOIOR, Oct 94 





FINAL REPORT 


GRANT NUMBER: F496200010336 

TITLE: Metareasoning for More Effective Human-Computer Dialogue 
PRINCIPAL INVESTIGATORS: 

Don Perlis and Mike Anderson 
University of Maryland 
College Park, MD 20742 


OBJECTIVES: 

This study will foster new understanding of error-repair in 
intelligent systems, of very broad applicability. Such repair is 
necessary where there is complex and noisy data, as is prevalent in 
military settings. 


STATUS OF EFFORT: 

See Accomplishments, below. The grant period ended December, 2002. 


SUMMARY: 

The research project explores specific meta-dialogue behaviors in 
terms of both how a system could be made to perform them, and to what 
extent they can increase overall system performance. We focus on 
two types of metadialogue capabilities: ability to detect and recover 
from anomalous dialogue patterns in simple exchanges, and on-line 
extensions or changes to working vocabulary. 

Our main method involves detailed representation of the dialogue 
context, separating domain, language, and dialogue specific aspects. 


1 


and significant amounts of meta-reasoning about the system's 
processing of these representations. An existing logical inference 
system, ALMA/CARNE, developed as part of a pilot study, is being used 
in an implementation phase of this work. We are also engaged in a 
study of existing dialogue corpora to investigate the range and 
frequency of meta-dialogue expressions in different task domains. 


ACCOMPLISHMENTS: 

1. We designed and implemented extensions of the existing active logic 
engine and dialogue manager, to be able to engage in a variety of 
action directive subdialogues, which include: 

(a) a request for action performance by the user 
Cb) performance of an action by the system 

(c) feedback ("followup") from the user (which might be positive 
indicating acceptance of the action, or negative, possibly 
Coupled with a replacement action). 

2. We also designed a more powerful architecture that will allow us to 
process new words and phrases and implicit and explicit quotation, 

in order to adapt to a variety of users and contexts. As a 
result, users will be able to instruct the system in exchanges 
such as: 

(d) I use "transportation" to refer to any vehicle, not just a car. 

(e) Peking is the old name for Beijing. 

3. Our most recent work has allowed us to implement a system (ALFRED: 
Active Logic for Reason Enhanced Dialog) that can, in simple cases, 

(i) recognize the need to for meaning clarification, (ii) generate a 
request for such clarification, and (iii) process a direct supplying 
of that clarification, as in: 

(f) A: Shoot the Chicago train to Boston. 

B: What does "shoot" mean? 

A: Shoot means send. 

B: OK [and sends the Chicago train to Boston] 


2 




This system was demonstrated at the University of Maryland Research 

Review Day (March 2003). A more advanced version is slated for demonstration 

at IJCAI 2003. 

Implementing the more powerful architecture above involved integrating 
a new parser into the system, as well as recoding several other 
modules. We are now begining to extend the system, allowing it not 
only to learn new words and phrases as in (d-e-f) above, but also 
learn novel concepts. 

In addition, isolating the metareasoning components involved in 
understanding language from the other parts of the system allows the 
dialog system to be easily oriented to different domains. 

4. These capabilities advance the state of the art for current 
natural language dialogue systems, and represent a major step toward 
our long-term goal of "conversational adequacy" as described in our 
earlier papers. 

5. We have also recently begun a corpus study of different kinds of 
meta-language, their form, frequency, and effect. Results from this 
study will help to implement more sophisticated reasoning mechanisms for 
dealing with the various kinds of misunderstanding in dialog. 

6. Summary of Dissertation. 

Khemdut Purang, Systems that Detect and Repair their own Mistakes’', 
University of Maryland 2001. 

Making mistakes is an inescapable aspect of everyday life. We 
constantly make mistakes, recognize them and try to correct 
them. Mistakes are inevitable because of the incompleteness of our 
knowledge of the world, its inherent uncertainty and its being in a 
constant state of change. We can never know for sure that what we know 
is true and the actions that we take based on these beliefs can 
therefore be misguided. Sooner or later we act based on some false 
belief or the world changes in an unexpected way and we fail to 


3 



achieve our goal. But the fact that we can recognize and repair these 
errors mitigates their effects. Software systems face the same 
problems. The difference is that they do not ususally have as robust a 
capability as we have to detect and respond to their mistakes. This is 
part of what makes them seem brittle and user-unfriendly. This problem 
is not likely to get any better as the systems exhibit more complex 
behaviors in more realistic domains. Our work begins to address that 
problem by focusing on the computational capabilities required of 
software systems for them to be able to autonomously recognize and 
respond to their own mistakes. We study in particular, mistaken 
beliefs, intentions and actions in agents that have some goals to 
achieve. Intuitively enough, the basic capabilities required are an 
ability to inspect their past behavior and computations and the past 
states of the world and to use that to determine their future 
behavior. These abilities are not typically available in software 
systems. We have implemented a general logical framework in which one 
can specify the behavior of an agent that supports this kind of 
representation and computation . We have implemented agents that 
detect and respond appropriately to their mistakes in some aspects of 
language processing. We have also implemented a system that handles 
its mistaken beliefs in any domain that can be described using the 
language of non-monotonic logic. This system was tested on a test 
suite that we compiled from examples of non-monotonic reasoning in the 
literature. We finally provide a design of the representations and 
algorithms for handling mistakes in an agent that acts in the world 
and has mistaken beliefs, intentions and actions. Implementing such an 
agent is the next step in this work. 


4 



PERSONNEL SUPPORTED 


D Perlis (PI) 

M Anderson (co-PI) 
K Purang (GRA) 

Y Chong (GRA) 

Y Okamoto (GRA) 

D Josyula (GRA) 

A Rogriguez (GRA) 

G Lapizco (GRA) 


TRANSITIONS (to DoD, Government, Industry): 
None, 


PUBLICATIONS: 


1 . Representations of Dialogue Statue for Domain and Task Independent 
Meta-Dialogue. David Traum, Carl Andersen, Yuan Chong, Darsana Josyula, 
Yoshi Okamoto, Khemdut Purang, Michael 0’Donovan-Anderson and Don Perlis 
Electronic Transactions on AI 6, 2002. 

2. The Use-Mention Distinction and its importance to HCI. Michael 

L. Anderson, Yoshi Okamoto, Darsana Josyula, and Don Perlis. Proceedings 
of the Sixth Workshop on the Semantics and Pragmatics of Dialog. 2002. 

3. Meta-reasoning for Conversational Adequacy. Michael L. Anderson 
Invited talk, Deutsches Forschungszentrum fr Knstliche Intelligenz, 2002 

4. Seven Days in the Life of a Robotic Agent. Waiyian Chong, Mike 
Q ! Donovan-Anderson, Yoshi Okamoto and Don Perlis. 

First GSFC/JPL Workshop on Radical Agent Concepts, 

2001, NASA Goddard Space Flight Center, Greenbelt, MD, USA 


5 


5. Meta-reasoning for Intelligent Dialog Repair, Khemdut Purang, David Traum, 
Darsana V, Purushothaman, Waiyian Chong, Yoshi Okamoto and Don Perlis, 
November 2000. 

6. Time-Situated Agency: Active Logic and Intention Formation. 

Cognitive Agents Workshop, 2002. With M. Anderson, 

D. Josyula, Y. Okamoto. 

7. Handling Uncertainty with Active Logic, Manjit Bhatia, Paul Chi, Waiyian 
Chong, Darsana P. Josyula, Yoshi Okamoto, Don Perlis and Khemdut Purang. 
March 2001. To be presented at the 2001 AAAI Fall Symposium Series in North 
Falmouth, MA. 

8. Zi at the Interface and Beyond, Yoshi Okamoto. 

Presented at the L0T-NWCL Summer School. Utrecht, the Netherlands: 
Universiteit Utrecht, 2001. 

9. Feature Analysis of Zibun, Yoshi Okamoto. April 2000. Presented at 45th 
Annual Conference of the International Linguistic Association. Washington, 
DC; Georgetown University. 

10. Split Antecedents <E0> la Movement, Yoshi Okamoto. July 

2000. Snippets, Issue 2. Milano, Italy: LED - Edizioni Universitarie 
di Lettere Economia Diritto. 

11. Agents that recognize and repair their own mistakes. PhD 
dissertation. Khemdut Purang. August 2001. 

12. D. Perlis. What does it take to refer? (Invited paper.) 

J. of Consciousness Studies, 2000. 

13. Symbol Systems. With Michael L. Anderson. 

Encyclopedia of Cognitive Science. 2002. 

14. Invited one-hour talk and paper on self-reference, presented 
at PhiLog, Copenhagen, October 2002. 


6 



INTERACTIONS: 

1. invited talk at New Mexico University 

2. contact and discussions with ARL (Dr. John Gurney) 

3. invited talk at PhiLog (Logic Colloquium in Denmark) 

4. invited talk at DFKI. 

Inventions: none 
Awards: none 


7