中国科学院软件研究所机构知识库
Advanced  
ISCAS OpenIR  > 中科院软件所  > 中科院软件所
题名:
多通道用户界面开发方法和环境
作者: 张高
答辩日期: 1999
专业: 计算机应用技术
授予单位: 中国科学院软件研究所
授予地点: 中国科学院软件研究所
学位: 博士
关键词: 多通道用户界面 ; 开发方法 ; 开发环境 ; 原型方法 ; 交互原语 ; 多通道整合 ; 多Agent软件模型 ; 多通道用户界面评估
摘要: 多通道用户界面的研究是为了消除当前图形用户界面、多媒体用户界面输入和输出带宽不平衡的瓶颈,综合采用语音、手势、视线等新的交互通道、设备和交互技术,使用户可以利用多个通道以自然、并行、协作的方式进行人机对话:通过整合来自多个通道的精确的和不精确的输入来捕捉用户的交互意图,提高人机交互的自然性和高效性。多通道用户界面已经成为了国内外的一个重要研究方向,本文作者参加了国家 自然科学基金重点项目“多通道用户界面研究”和863项目“基于PC机的多通道用户界面开发环境”,对多通道用户界面的关键技术和软件结构作了较深入的研究和开发工作。在项目的研究和开发中,我们得出了以下结论:针对图形用户界面而建立起来的面向对象的用户界面开发方法已经不能很好地适应多通道用户界面的特点了,多通道用户界面所带来的用户界面非精确性和并发性等特点,需要有新的开发方法和技术来支持。本文的目标是在充分讨论多通道用户界面的特点和关键技术的基础上,探讨适应于多通道用户界面特点的开发方法和相关技术。围绕这一目标,本文就以下方面展开了讨论:1.多通道用户界面的研究概况 国内外的研究人员对多通道的提法和定义有一些不同的理解,多通道用户界面的定义和性质是讨论其开发方法和环境的基础。本文总结了国内外相关研究的主要结论,并给出了具有代表性的多通道用户界面的定义和性质描述。2.多通道用户界面的开发方法 结合对多通道用户界面特点的分析,本文从软件工程的角度探讨了适合于多通道用户界面开发的方法。从软件重用的角度,我们对比了多Agent技术和多通道用户界而的特点,指出多Agent技术能够更好地适应多通道用户界面的要求。在此基础上,本文提出了面向Agent的多通道用户界面开发方法,对基于其上的多通道用户界面丌发流程进行了描述,并分析了面向Agent的技术和面向对象技术的区别。3.多通道用户界面中的关键技术 多通道交互设备管理和多通道整合是多通道用户界面研究中的两个关键技术,对它们的深入研究是进一步探讨面向Agent开发方法的技术实现的前提。在多通道交互设备管理方面,本文根据交互原语的设备无关性和应用无关性,提出了以交互原语作为多通道交互设备分类管理的科学依据。在多通道整合技术方面,本文总结了现有的较为典型的整合算法,并根据通道信息的分层模型,提出了基于任务的多通道分层整合算法。该算法比传统的整合算法更加系统化,并较好地解决了多通道整合中的指称语义反馈问题。4.多通道用户界面的多Agent软件模型 建立多通道用户界面的多Agent软件模型是面向Agent开发方法的技术实现,本文根据多通道用户界面的特点和关键技术,分析了目前用户界面领域中常用的 PAC Agent及其改进Agent模型的不足,并提出了基于整合的多通道用户界面的多Agent软件模型。该模型利用交互Agent来管理多通道交互设备,利用词法整合Agent、语法整合Agent和语义整合Agent来体现分层整合算法的思想,利用语义反馈Agent来增强对话独立后的语义反馈。为了充分体现Agent的可重用性,本文提出了利用静态的知识来代替动态过程的思想,给出了建立在知识和规则之上的新的Agent体系结构。利用BNF和LOTOS相结合的方法,我们对多通道用户界面的多Agent软件模型中的各个Agent作了形式化的描述。5.多通道用户界面评估 多通道用户界面评估是多通道用户界面原型开发中的重要环节,用户界面的评 估方法可以分为形式化方法和经验方法,经验方法是支持多通道用户界面开发的更实用、更有效的方法。本文总结了三种典型的经验评估方法:观察法、咨询法、实验方法,并将方差分析的方法引入到对不同通道结合方式的比较评估中,为评估多通道用户界面的交互效率提供了定量的依据。6.多通道用户界面开发环境及其应用实例 多通道用户界面的开发方法和软件模型要通过具体的开发环境来发挥作用,本文介绍了一个我们开发的多通道用户界面支撑平台,并对进一步建立可视化的开发环境给出了设计。结合多通道地图查询系统的开发,本文用实例说明了多通道用户界面开发中从需求分析、Agent设计、Agent的实现到界面评估的全过程。通过以上的讨论,本文在以下方面做出了创新的工作: 1.在多通道用户界面开发方法方面,提出了面向Agent的多通道用户界面开发方法,并对相应的开发流程进行了描述。2.深入研究了多通道用户界面的整合算法,提出了更加系统的基于任务的多通道分层整合算法,较好地解决了多通道整合中的指称语义反馈问题。3.分析了目前用户界面领域中常用的Agent模型的不足,提出了基于整合的多通道用户界面的多Agent软件模型,并给出了建立在知识和规则之上的Agent体 系结构。4.将方差分析的方法引入到了多通道用户界面的评估中,为评估多通道用户界面的交互效率提供了定量的依据。
英文摘要: By combining multiple interaction modalities, such as speech, gesture, eye-gazing etc., Mulitmodal User Interface(MMI) is aiming to balance the input bandwidth with the great output bandwidth provided by Multimedia User Interface. In MMI, User can interact with computer in a more natural, parallel, cooperative way. By integrating multiple accurate and inaccurate input information from the user, MMI can improve more nature and intuitive interaction. Multimodal User Interface has been set up as an important research direction in China. The author of this paper has took part in several related projects, such as "Study of Multimodal User Interface" supported by National Natural Science Foundation, "PC Based development environment of Multimodal User Interface" supported by 863 project. In these projects, we have done a lot of research and development on the key technologies and the software structure of MMI. The result of these projects showed that the Object-oriented development method based on tradiontal GUI could not meet the need of Multimoda Use Interface. MMI needs the new development method and technology to support its inaccuracy and parallel. The goal of this paper is to deeply discuss the characteristics and key technologies of MMI and explore the suitable development method and technology for MMI. Around this goal, this paper has discussed the following issues: 1. Survey of Multimodal User Interface Study There are some different definitions and understandings about Multimodal in this study field, and the definition and characteristics of MMI are the base of the further discussion. This paper surveyed the related classic research results and gave out the representative definition and characteristics description. 2. Development Method of Multimodal User Interface This paper discussed the suitable development method for MMI from the view of Software Engineering. This paper compared the characteristics of Multi-agent technology and Multimodal UI and asserted that the Multi-agent technology is more suitable for MMI in development for the greater reusability. This paper presented the Agent-oriented development method for MMI, and compared the differences of Agent-oriented method and Object-oriented method. 3. The Key Technologies in Multimodal User Interface Multiple interaction equipment management and multimodal integration are two key technologies in the study of MMI, and they are the basis of Agent-oriented development method. By studying the independence of equipment and application of Interaction Primitive, this paper presented to manage the multiple interaction equipment by use of Interaction Primitive. This is a more reasonable management method. By studying the existed integration algorithms, this paper presented Delamination Integration Algorithm based on the task. This algorithm was a more powerful than existed ones and gave some ways to solve the semantic feedback problem in the integration. 4. Multi-agent Based Software Architecture of MMI The Multi-agent based Software Architecture of MMI is the implementation of Agent-oriented development method. This paper presented the Integration-based Multi-agent software architecture of MMI. In this architecture, Interaction Agent was used to manage the interaction equipment; Lexcial Agent, Syntax Agent, Semantic Agent were used to perform the integration; and Feedback Agent was used to provide semantic feedback among these Agents. This paper also gave out the new Agent structure by replacing dynamic procedures with static knowledge and rules. We gave out the formal description of each Agent in the Multi-agent software architecture. 5. Evaluation of Multimodal User Interface Multimodal User Interface evaluation is very important to support the prototyping development. This paper asserted that the experiment-based methods were more suitable for MMI. This paper introduced the ANOVA (Analysis of Variance) to evaluate the different interaction efficiency of different modality combination modes. This method provided the mensurable foundation for multimodal user interface evaluation. 6. Development Environment and Application Cases of MMI The Agent-oriented method helps development by used of development environment. This paper gave out a development supporting platform and the design of visualized development tools. Through some multimodal interaction cases, this paper showed how to develop a MMI by use of Agent-oriented method. From the above discussion, this paper drew some creative conclusions as following: 1. Having presented the Agent-oriented development method for MMI; 2. Having presented the task-based delamination integration algorithm, and gave out some ways to solve the reference semantic feedback problem; 3. Having presented the integration-based multi-agent Software architecture for MMI, and gave out the structure of Agent; 4. Having introduced ANOVA into multimodal User interface evaluation.
语种: 中文
内容类型: 学位论文
URI标识: http://ir.iscas.ac.cn/handle/311060/6890
Appears in Collections:中科院软件所

Files in This Item:
File Name/ File Size Content Type Version Access License
LW002849.pdf(3408KB)----限制开放-- 联系获取全文

Recommended Citation:
张高. 多通道用户界面开发方法和环境[D]. 中国科学院软件研究所. 中国科学院软件研究所. 1999-01-01.
Service
Recommend this item
Sava as my favorate item
Show this item's statistics
Export Endnote File
Google Scholar
Similar articles in Google Scholar
[张高]'s Articles
CSDL cross search
Similar articles in CSDL Cross Search
[张高]‘s Articles
Related Copyright Policies
Null
Social Bookmarking
Add to CiteULike Add to Connotea Add to Del.icio.us Add to Digg Add to Reddit
所有评论 (0)
暂无评论
 
评注功能仅针对注册用户开放,请您登录
您对该条目有什么异议,请填写以下表单,管理员会尽快联系您。
内 容:
Email:  *
单位:
验证码:   刷新
您在IR的使用过程中有什么好的想法或者建议可以反馈给我们。
标 题:
 *
内 容:
Email:  *
验证码:   刷新

Items in IR are protected by copyright, with all rights reserved, unless otherwise indicated.

 

 

Valid XHTML 1.0!
Copyright © 2007-2017  中国科学院软件研究所 - Feedback
Powered by CSpace