中国科学院软件研究所机构知识库
Advanced  
ISCAS OpenIR  > 中科院软件所  > 中科院软件所
题名:
多通道用户界面开放模型的研究及应用
作者: 李茂贞
答辩日期: 1997
专业: 计算机软件
授予单位: 中国科学院软件研究所
授予地点: 中国科学院软件研究所
学位: 博士
关键词: 用户界面 ; 用户界面概念模型 ; 对话独立性 ; 用户界面模型 ; 多通道用户界面 ; 交互风格原语 ; 交互概念原语 ; 交互任务原语 ; 应用原语 ; 原语映射 ; 通道整合 ; 用户界面评估
摘要: 近年来,计算机输入输出装置在数量和能力上迅速增加,但计算机加工和呈现视觉、听觉信息的能力和人机交互能力是不相称的,从计算机到用户的通讯带宽要比从用户到计算机的通讯带宽大得多。这种不相称来自于硬件的限制和人的交互通道与对象操作间的不匹配。另外,虚拟现实技术、科学计算可视化对计算机系统的人机交互提出了高效、三维和非精确的要求,而现有的交互方式能胜任二维操作但缺乏空间中进行非精确直接操作对象的能力。要从根本上改变人机交互现状,必须发展从用户到计算机之间的快速、高效的多通道用户界面(Multi-modal Interface, MMI)。目前,MMI 已成为用户界面研究的重点。MMI 采用视线跟踪、语音识别、手势输入等新的交互技术,允许用户利用多个交互通道并行、非精确方式与计算机系统进行交互,旨在提高人机交互的自然性的高效性。MMI 是继命令行界面(Command LIne Inteface, CLI) 、WIMP (窗口、图符、菜单、指点)界面之后的新一代用户界面。论文围绕多通道用户界面这一课题进行了研究,论文的主要工作包括以下几个方面:1.用户界面的概念模型 本文提出了CLI、WIMP界面及MMI这三类用户界面的概念模型,并对它们进行了比较。比较结果表明,无论从用户到计算机的输入带宽,还是计算机到用户的输出带宽,MMI比CLI、WIMP的人机通讯带宽都提高了很多。因此MMI是用户界面的发展趋势。2.多通道界面开放模型PBM_MMI PAC模型和Arch 模型分别为基于Agent的用户界面模型和结构化界面模型的典型代表。本文将 PAC 模型中的PAC Agent 嵌入到Arch模型中的各个部件中,提出一种基于交互原语的多通道界面开放模型PBM_MMI。该模型能很好地满足MMI的各方面要求,体现了交互系统的纵向角度、横向角度分解的思想,从而简化交互系统的开发。3.PBM_MMI中原语的使用 原语是知识的一种表示方法,其好处在于表达方式的简洁性及语义的完整性。在PBM_MMI中使用了四级原语即交互风格原语ISP、交互概念原语ICP、ITP到DAP, 交互系统能逐步理解用户的交互意图。因为不同的ISP可以映射为同一个ICP,因此PBM_MMI易于扩充交互通道。同时关联ICP的使用使得多个ICP语义整合成为可能。由于PBM_MMI具有各个ICP的语义信息及整合规则,因此PBM_MMI可以支持多个交互通道的使用及信息的非精确性处理。而原语使用者PAC Agent使得PBM_MMI可以支持信息的并发处理。这样,PBM_MMI允许用户使用多个交互通道以并行、非精确的方式与系统进行交互。4.PBM_MMI中的PAC Agent PBM_MMI中使用了几类PAC Agent。 每类PAC Agent都赋予了一定的处理能力,各个PAC Agent之间通过消息传递进行通讯,PAC Agent只接收自己感兴趣的消息,因此通讯效率得到提高。PAC Agent是独立的运行单元,多个PAC Agent可以并发处理。这样,一个交互系统就可以分解为多个PAC Agent的运行,本文利用BNF范式对进行了定义,为PAC Agent的实现提供了一个框架结构。5.PBM_MMI中原语映射表的实现 PBM_MMI使用了几类信息表用于实现原语之间的映射。按照本论文中提出的操作语法定义,可以对这些信息表进行一定的操作如增加、删除、查询、修改,以便使它们适应不同的环境。6.MMI的评估机制 由于MMI主要体现了界面表现及人机交互两个方面的内容,因此本文从这两个方面出发,并结合MMI的特点,提出一种MMI的评估机制。该机制能够对整个MMI进行定量评估。7.PBM_MMI的应用 基于PBM_MMI,我们设计了具有多通道界面特点的原型系统 VoiceDraw。这是一个语音驱动的绘图代理。在VoiceDraw中,用户可以使用手势和语音两个交互通道以并行、非精确性方式和系统进行交互。VoiceDraw体现了用户和计算机系统交互的自然性和高效性,同时也验了PBM_MMI的有效性。并且利用本文提出的MMI评估机制。对VoiceDraw进行了定量评估。本论文的研究是国家自然科学基金重点项目“多通道用户界面研究”中的一部分工作。
英文摘要: People have been faced an explosion in both the number and capabilities of human-computer interactive devices in recent years. However, there is an asymmetry between the ability of computer to process and present visual and audio information and the ability for people to process and interact with this information. The communication bandwidth of computer-human is much more large than that of human-computer. This kind of asymmetry comes from hardware constrains and the mismatch between the manipulation of objects and interactive modality of human. In addition, much of the human-computer interaction of VR or scientific visualization system involves three-dimensional and usually inaccurate manipulations of graphic objects. Existing interactive mode lacks the ability of manipulate objects in three-dimensional space in a direct, inaccurate and intuitive way while it is good enough to two-dimensional operation. Multimodal Interface (MMI) is a promising technology to solve this problem. MMI has been the research emphasis of user interface (UI) recently. With new interactive technologies such as eye-tracking, voice recognition, gesture et al., MMI allow user to use multiple interactive modalities to interact with the computer system. in a parallel and inaccurate way. The goal of MMI is to improve the efficiency and intuition of human-computer interaction. Following Command Line Interface (CLI) and WIMP (Window, Icon, Menu, Pointing) Interface, MMI is a new generation user interface. This paper studies some issues in MMI, the main research work in this paper involves the following aspects. 1. The concept model of user interface This paper puts forth the concept models of CLI, WIMP interface and MMI. The comparison among them is given. The communication bandwidth of human-computer and computer-human of MMI is much more wide than that of CLI and WIMP interface. Therefore, MMI is the trend of user interface. 2. Primitive_based open model of MMI(PBM_MMI) PAC model is a typical model of Agent-based user interface model, and Arch model is a typical model of structured interface model. Embedding the PAC Agents into components of Arch model, a primitive-based open model of MMI(PBM_MMI) is given. The PBM_MMI can meet all of the requirements of MMI and serves as a good base for the vertical and horizontal decomposition of interactive system. 3. Interactive primitives used in PBM_MMI Primitive is a kind of presentation method of knowledge. The advantages of primitive are succinctness and semantic completeness. Four level of interactive primitives, Interactive Style Primitive (ISP), Interactive Concept Primitive (ICP), Interactive Task Primitive (ITP) and Domain Application Primitive (DAP) are used in PBM_MMI. The abstraction degrees of each level primitive are different. From ISP, ICP, ITP to DAP, interactive system can grasp the user interactive intention steply. Since different ISP can be mapped into the same ICP, the PBM_MMI is easy to expand interactive modalities. In addition, integrating semantic information of multiple ICPs which are related, therefore, the PBM_MMI can support multiple interactive modalities and inaccurate information process. PAC Agent makes PBM_MMI support information concurrent process. In summary, PBM_MMI allows user using multiple interactive modalities to interact with computer system in a parallel and inaccurate way. 4. PAC Agents in PBM_MMI There are several kinds of PAC Agents in PBM_MMI. Each kind of Agents is granted with certain process capabilities. They communicate through message passing. Agents only accept the information which they are interested in. Therefore the communication efficiency is high. In addition, Agents are stand-alone processing unit, multiple Agents can implement concurrent process of information. Thus, a large interactive system can be divided into multiple Agents, which simplifies the development of interactive system. The BNF definition of Agent is given in this paper, which can provide a framework for Agent implementation. 5. The implementation of primitive-mapping tables in PBM_MMI With several information tables, PBM_MMI can map different primitives. Based on operation definition syntactic given in this paper, we can perform some operations on these information tables such as addition, deletion, query and modification to make them fit for different situations. 6. An evaluation mechanism of MMI Based on the interface presensation and human-computer interaction, an evaluation mechanism is given in this paper. the mechanism can evaluate MMI in a quantitative way. 7. The application of PBM_MMI Based on PBM_MMI, we have implemented a interactive system called VoiceDraw, which is characteristic of MMI. In VoiceDraw, user can use gesture and voice to interact with computer system. VoiceDraw shows the efficiency and intuition of human-computer interaction, in the meantime, it also verifies the validation of PBM_MMI. Using the evaluation mechanism in this paper, we evaluate VoiceDraw quantitatively. This paper is part work of the key project of National Natural Science Foundation of China, Multimodal User Interface Research (MUIR).
语种: 中文
内容类型: 学位论文
URI标识: http://ir.iscas.ac.cn/handle/311060/7246
Appears in Collections:中科院软件所

Files in This Item:
File Name/ File Size Content Type Version Access License
N98830.pdf(3181KB)----限制开放-- 联系获取全文

Recommended Citation:
李茂贞. 多通道用户界面开放模型的研究及应用[D]. 中国科学院软件研究所. 中国科学院软件研究所. 1997-01-01.
Service
Recommend this item
Sava as my favorate item
Show this item's statistics
Export Endnote File
Google Scholar
Similar articles in Google Scholar
[李茂贞]'s Articles
CSDL cross search
Similar articles in CSDL Cross Search
[李茂贞]‘s Articles
Related Copyright Policies
Null
Social Bookmarking
Add to CiteULike Add to Connotea Add to Del.icio.us Add to Digg Add to Reddit
所有评论 (0)
暂无评论
 
评注功能仅针对注册用户开放,请您登录
您对该条目有什么异议,请填写以下表单,管理员会尽快联系您。
内 容:
Email:  *
单位:
验证码:   刷新
您在IR的使用过程中有什么好的想法或者建议可以反馈给我们。
标 题:
 *
内 容:
Email:  *
验证码:   刷新

Items in IR are protected by copyright, with all rights reserved, unless otherwise indicated.

 

 

Valid XHTML 1.0!
Copyright © 2007-2017  中国科学院软件研究所 - Feedback
Powered by CSpace