Abstract: The EasyLiving project is concerned with development of an architecture and technologies for intelligent environments which allow the dynamic aggregation of diverse I/O devices into a single coherent user experience. Components of such a system include middleware (to facilitate distributed computing), world modelling (to provide location-based context), perception (to collect information about world state), and service description (to support decomposition of device control, internal logic, and user interface). This paper describes the current research in each of these areas, highlighting some common requirements for any intelligent environment.

Abstract: A multiple view system uses two or more distinct views to support the investigation of a single conceptual entity. Many such systems exist, ranging from computer-aided design (CAD) systems for chip design that display both the logical structure and the actual geometry of the integrated circuit to overview-plus-detail systems that show both an overview for context and a zoomed-in-view for detail. Designers of these systems must make a variety of design decisions, ranging from determining layout to constructing sophisticated coordination mechanisms. Surprisingly, little work has been done to characterize these systems or to express guidelines for their design. Based on a workshop discussion of multiple views, and based on our own design and implementation experience with these systems, we present eight guidelines for the design of multiple view systems.

Abstract: A user interface software tool helps developers design and implement the user interface. Research on past tools has had enormous impact on today's developers—virtually all applications today are built using some form of user interface tool. In this article, we consider cases of both success and failure in past user interface tools. From these cases we extract a set of themes which can serve as lessons for future work. Using these themes, past tools can be characterized by what aspects of the user interface they addressed, their threshold and ceiling, what path of least resistance they offer, how predictable they are to use, and whether they addressed a target that became irrelevant. We believe the lessons of these past themes are particularly important now, because increasingly rapid technological changes are likely to significantly change user interfaces. We are at the dawn of an era where user interfaces are about to break out of the “desktop” box where they have been stuck for the past 15 years. The next millenium will open with an increasing diversity of user interface on an increasing diversity of computerized devices. These devices include hand-held personal digital assistants (PDAs), cell phones, pages, computerized pens, computerized notepads, and various kinds of desk and wall size-computers, as well as devices in everyday objects (such as mounted on refridgerators, or even embedded in truck tires). The increased connectivity of computers, initially evidenced by the World Wide Web, but spreading also with technologies such as personal-area networks, will also have a profound effect on the user interface to computers. Another important force will be recognition-based user interfaces, especially speech, and camera-based vision systems. Other changes we see are an increasing need for 3D and end-user customization, programming, and scripting. All of these changes will require significant support from the underlying user interface sofware tools.

Abstract: We address the problems of virtual object interaction and user tracking in a table-top augmented reality (AR) interface. In this setting there is a need for very accurate tracking and registration techniques and an intuitive and useful interface. This is especially true in AR interfaces for supporting face to face collaboration where users need to be able to easily cooperate with each other. We describe an accurate vision-based tracking method for table-top AR environments and tangible user interface (TUI) techniques based on this method that allow users to manipulate virtual objects in a natural and intuitive manner. Our approach is robust, allowing users to cover some of the tracking markers while still returning camera viewpoint information, overcoming one of the limitations of traditional computer vision based systems. After describing this technique we describe its use in prototype AR applications.

Abstract: To create successful interactive systems, user interface designers need to cooperate with developers and application domain experts in an interdisciplinary team. These groups, however, usually miss a common terminology to exchange ideas, opinions, and values.This paper presents an approach that uses pattern languages to capture this knowledge in software development, HCI, and the application domain. A formal, domain-independent definition of design patterns allows for computer support without sacrificing readability, and pattern use is integrated into the usability engineering life cycle.As an example, experience from building an award-winning interactive music exhibit was turned into a pattern language, which was then used to inform follow-up projects and support HCI education.

Abstract: The invention enables creation of grammar networks that can regulate, control, and define the content and scope of human-machine interaction in natural language voice user interfaces (NLVUI). More specifically, the invention concerns a phrase-based modeling of generic structures of verbal interaction and use of these models for the purpose of automating part of the design of such grammar networks.

Abstract: A distributed electronic entertainment method and apparatus are described. In one embodiment, a central management resource is coupled to multiple out-of-home venues through a wide area network (WAN). The central management resource stores content and performs management, monitoring and entertainment content delivery functions. At each venue at least one entertainment unit is coupled to the WAN. Multiple entertainment units in a venue are coupled to each other through a local area network (LAN). In one embodiment, an entertainment unit includes a user interface that comprises at least one graphical user interface (GUI). The entertainment unit further comprises a local memory device that stores entertainment content comprising music, a peripheral interface, and a user input device. A plurality of peripheral devices are coupled to the at least one entertainment unit via the peripheral interface, wherein a user, through the user input device and the user interface, performs at least one activity from a group comprising, playing music, playing electronic games, viewing television content, and browsing the Internet.

Abstract: Modern window-based user interface systems generate user interface events as natural products of their normal operation. Because such events can be automatically captured and because they indicate user behavior with respect to an application's user interface, they have long been regarded as a potentially fruitful source of information regarding application usage and usability. However, because user interface events are typically voluminos and rich in detail, automated support is generally required to extract information at a level of abstraction that is useful to investigators interested in analyzing application usage or evaluating usability. This survey examines computer-aided techniques used by HCI practitioners and researchers to extract usability-related information from user interface events. A framework is presented to help HCI practitioners and researchers categorize and compare the approaches that have been, or might fruitfully be, applied to this problem. Because many of the techniques in the research literature have not been evaluated in practice, this survey provides a conceptual evaluation to help identify some of the relative merits and drawbacks of the various classes of approaches. Ideas for future research in this area are also presented. This survey addresses the following questions: How might user interface events be used in evaluating usability? How are user interface events related to other forms of usability data? What are the key challenges faced by investigators wishing to exploit this data? What approaches have been brought to bear on this problem and how do they compare to one another? What are some of the important open research questions in this area?

Abstract: Systems and methods are provided for dynamic distribution of audio signals at a site based on defined zones within the site. A plurality of addressable audio devices are coupled to a local network for the site which are configured to receive a designated digital audio stream over the local network and to output the received digital audio stream to audio equipment located at the site. A zone manager defines a plurality of zones for the site which may include a plurality of the addressable audio devices. The zone manager defines a relationship between a characteristic of the audio signal for a reference audio device and for the addressable audio devices in the zones. An audio interface receives digital audio streams and outputs the digital audio streams on the local network addressed to selected ones of the audio devices based on the defined zones, the defined relationship between a characteristic of the audio signal for a reference audio device and for the addressable audio devices and a control input associated with the characteristic. A user interface is provided which is configured to receive a user designation of the control input. Systems and methods for dynamic aggregation of audio equipment in zones are also provided.

Abstract: A challenge for human-computer interaction researchers and user interf ace designers is to construct information technologies that support creativity. This ambitious goal can be attained by building on an adequate understanding of creative processes. This article offers a four-phase framework for creativity that might assist designers in providing effective tools for users: (1)Collect: learn from provious works stored in libraries, the Web, etc.; (2) Relate: consult with peers and mentors at early, middle, and late stages, (3)Create: explore, compose, evaluate possible solutions; and (4) Donate: disseminate the results and contribute to the libraries. Within this integrated framework, this article proposes eight activities that require human-computer interaction research and advanced user interface design. A scenario about an architect illustrates the process of creative work within such an environment.

Abstract: Methods and systems for providing graphical user interfaces are described. To provide greater access and consolidation to frequently used items in the graphical user interface, a userbar is established which includes a plurality of item representations. To permit a greater number of items to reside in the userbar, a magnification function can be provided which magnifies items within the userbar when they are proximate the cursor associated with the graphical user interface.

Abstract: A method of informing a first network user of activity by other network users includes receiving information identifying television programming viewed by at least one other network user and displaying the information to the first network user on a user interface.

Abstract: In the last several years, large multi-dimensional databases have become common in a variety of applications such as data warehousing and scientific computing. Analysis and exploration tasks place significant demands on the interfaces to these databases. Because of the size of the data sets, dense graphical representations are more effective for exploration than spreadsheets and charts. Furthermore, because of the exploratory nature of the analysis, it must be possible for the analysts to change visualizations rapidly as they pursue a cycle involving first hypothesis and then experimentation. The authors present Polaris, an interface for exploring large multi-dimensional databases that extends the well-known Pivot Table interface. The novel features of Polaris include an interface for constructing visual specifications of table based graphical displays and the ability to generate a precise set of relational queries from the visual specifications. The visual specifications can be rapidly and incrementally developed, giving the analyst visual feedback as they construct complex queries and visualizations.

Abstract: A Cooperative Help Assistance (CHA) system and method provide real-time user assistance for one or more windows-based Graphic User Interface (GUI) applications or a single application's different subsections such as web pages, running concurrently in any operating system. The CHA System enables the development of an informative assistance object independently from the original source code or development environment of the target Host Application. The assistance object can be selected by any number of user interfaces from sophisticated inference driven interactive interface search tools or categorized lists. By intercepting and monitoring user actions on a Host Application, the CHA system performs intelligent assistance in the context of the target host application program. Utilizing a Host Application Model, the CHA System and method dynamically assemble many elements in real-time or just-in-time to produce assistance sequences or elements very efficiently without having to code every interface path permutation. Paths can be dynamically generated from the Host Application Model, which enables a real-time module to offer intelligent, contextual assistance as well as real-time construction of automated, accelerated CHA Sequences or Procedures that require little or no user interaction. All assistance and information are processed and expressed by an extensive multitasking, multimedia subsystem for two dimensional (2D) and real-time three-dimensional (3D) application interfaces, which greatly enhances and extends the effectiveness of any explanation or material expression. The production of Assistant Sequences is facilitated by the Host Application Model and 2D and 3D GUI “drag and drop” interface tools.

Abstract: Advancements in computer technology have allowed the development of human-appearing and -behaving virtual agents. This study examined if increased richness and anthropomorphism in interface design lead to computers being more influential during a decision-making task with a human partner. In addition, user experiences of the communication format, communication process, and the task partner were evaluated for their association with various features of virtual agents. Study participants completed the Desert Survival Problem (DSP) and were then randomly assigned to one of five different computer partners or to a human partner (who was a study confederate). Participants discussed each of the items in the DSP with their partners and were then asked to complete the DSP again. Results showed that computers were more influential than human partners but that the latter were rated more positively on social dimensions of communication than the former. Exploratory analysis of user assessments revealed that some features of human–computer interaction (e.g. utility and feeling understood) were associated with increases in anthropomorphic features of the interface. Discussion focuses on the relation between user perceptions, design features, and task outcomes.

Abstract: Since the 1960s, the rapid growth of information systems has led to the wide development of research on human-computer interaction (HCI) that aims at the designing of human-computer interfaces presenting ergonomic properties, such as friendliness, usability, transparency, etc. Various work situations have been covered--clerical work, computer programming, design, etc. However, they were mainly static in the sense that the user fully controls the computer. More recently, public and private organizations have engaged themselves in the enterprise of managing more and more complex and coupled systems by the means of automation. Modern machines not only process information, but also act on dynamic situations as humans have done in the past, managing stock exchange, industrial plants, aircraft, etc. These dynamic situations are not fully controlled and are affected by uncertain factors. Hence, degrees of freedom must be maintained to allow the humans and the machine to adapt to unforeseen contingencies. A human-machine cooperation (HMC) approach is necessary to address the new stakes introduced by this trend. This paper describes the possible improvement of HCI by HMC, the need for a new conception of function allocation between humans and machines, and the main problems encountered within the new forms of human-machine relationship. It proposes a conceptual framework to study HMC from a cognitive point of view in highly dynamic situations like aircraft piloting or air-traffic control, and concludes on the design of 'cooperative' machines.

Abstract: Service robotics is currently a highly active research area in robotics, with enormous societal potential. Since service robots directly interact with people, finding “natural” and easy-to-use user interfaces is of fundamental importance. While past work has predominately focussed on issues such as navigation and manipulation, relatively few robotic systems are equipped with flexible user interfaces that permit controlling the robot by “natural” means. This paper describes a gesture interface for the control of a mobile robot equipped with a manipulator. The interface uses a camera to track a person and recognize gestures involving arm motion. A fast, adaptive tracking algorithm enables the robot to track and follow a person reliably through office environments with changing lighting conditions. Two alternative methods for gesture recognition are compared: a template based approach and a neural network approach. Both are combined with the Viterbi algorithm for the recognition of gestures defined through arm motion (in addition to static arm poses). Results are reported in the context of an interactive clean-up task, where a person guides the robot to specific locations that need to be cleaned and instructs the robot to pick up trash.

Abstract: A method of navigating within a plurality of bit-maps through a client user interface, comprising the steps of displaying at least a portion of a first one of the bit-maps on the client user interface, receiving a gesture at the client user interface, and in response to the gesture, altering the display by substituting at least a portion of a different one of the bit-maps for at least a portion of the first bit-map

Abstract: The present invention describes a revolutionary new device bridging the gap between the virtual multimedia-based Internet world and the real world, best exemplified by print media. More particularly, the invention relates to communicating multimedia information using a scanner for machine-readable code containing a link information corresponding to a provider information depicted on the printed medium, a user interface for obtaining user input information corresponding to the provider information, a communications bridge for sending the link information and the user input information via the network, a receiver in communication with the scanner, capable of receiving the link information and user input information, and further capable of receiving and playing a multimedia information sequence, and a portal server in communication with the scanner via the network capable of selecting a multimedia information sequence corresponding to the link information and the user input information.

Abstract: A wireless communication system that utilizes a remote voice recognition server system to translate voice input received from serviced mobile devices into a symbolic data file (e.g. alpha-numeric or control characters) that can be processed by the mobile devices. The translation process begins by establishing a voice communication channel between the serviced mobile device and the voice recognition server. A user of the mobile device then begins speaking in a fashion that may be detected by the voice recognition server system. Upon detecting the user's speech, the voice recognition server system translates the speech into a symbolic data file, which is then forwarded to the user through a separate data communication channel. The user, upon receiving the symbolic data file at the mobile device, reviews and edits the content and further utilizes the file as desired.

Abstract: A user interface device and system for providing a shared GTM and CDN (collectively Universal Distribution Network) for a service fee, where the customer or user does not need to purchase significant hardware and/or software features. The present interface device and system allows a customer to scale up its Web site, without a need for expensive and difficult to use hardware and/or software. In a preferred embodiment, the customer merely pays for a service fee, which can be fixed, variable, lump some, or based upon a subscription model using the present system. The present device and system are preferably implemented on a system including a novel combination of global traffic management and content distribution.

Abstract: An alternate display content controller provides a technique for controlling a video display separately from and in addition to the content displayed on the operating system display surface. Where the display is a computer monitor, the alternate display content controller interacts with the computer utility operating system and hardware drivers to control allocation of display space and create and control one or more parallel graphical user interfaces adjacent the operating system desktop. An alternate display content controller may be incorporated in either hardware or software. As software, an alternate display content controller may be an application running on the computer operating system, or may include an operating system kernel of varying complexity ranging from dependent on the utility operating system for hardware system services to a parallel system independent of the utility operating system and capable of supporting dedicated applications. The alternate display content controller may also include content and operating software delivered over the internet or any other LAN. The alternate display content controller may also be included in a television decoder/settop box to permit two or more parallel graphical user interfaces to be displayed simultaneously.

Abstract: In an interactive network-based personal video recording system, a user interface is provided that allows viewers to search a database of program guide information according to program title, indexed program attributes such as actor and director, and key word. Viewer interaction is by way of a remote control unit.

Abstract: Multiple coordinated visualizations enable users to rapidly explore complex information. However, users often need unforeseen combinations of coordinated visualizations that are appropriate for their data. Snap-Together Visualization enables data users to rapidly and dynamically mix and match visualizations and coordinations to construct custom exploration interfaces without programming. Snap's conceptual model is based on the relational database model. Users load relations into visualizations then coordinate them based on the relational joins between them. Users can create different types of coordinations such as: brushing, drill down, overview and detail view, and synchronized scrolling. Visualization developers can make their independent visualizations snap-able with a simple API.Evaluation of Snap revealed benefits, cognitive issues, and usability concerns. Data savvy users were very capable and thrilled to rapidly construct powerful coordinated visualizations. A snapped overview and detail-view coordination improved user performance by 30-80%, depending on task.

Abstract: A messaging application user interface has an input element for receiving electronic messages and an output element for displaying electronic messages. The messaging application user interface can be implemented so as to maintain a subset of the plurality of potential message recipients, the subset being determined using a predetermined criterion. The messaging application user interface also can be implemented so as to auto-complete a partially entered address based on the subset of potential message recipients. A set of user-selectable signals that modify auto-completion behavior also can be used with the messaging application user interface. The messaging application user interface can be used in an instant messaging system to allow a user to send instant messages to and exchange comments with a plurality of other users without having to switch among several instant message windows.

Abstract: A method on an information processing unit performing steps for creating a user interface (UI) to assemble a document that conforms to a particular document type definition. The method hides the specific syntax of document type definitions such as DTDs and schemas from the user. The method begins with a selection from a user for a document type or an existing document. Once the document type is selected or determined from the existing document the document type definitions are retrieved. The document type definitions include one or more elements. The method parses the elements which are subsequently mapped to one or more interface controls such as icons, pull-down menus, buttons, selection boxes, progress indicators, on-off checkmarks, scroll bars, windows, window edges for resizing the window, toggle buttons, forms, and UI widgets. UI can be GUIs or interactive voice response systems. A UI editor is presented by assembling the one or more interface controls without presenting specific document type definition syntax to a user. The UI editor permits the user to create and edit the content objects that are associated with the interface controls. The content objects are aggregated in an XML compatible format and ready to be checked in for further processing. The method permits specific UI interfaces to be created for specific publishing environments and at the same time permit the creation of reusable content objects.

Abstract: An internet radio for portable applications and uses such as in an automobile enables providing navigational services to be provided to a wireless communication device, and comprises the steps of providing a user interface in said wireless communication device for providing navigational services to a user; downloading updated navigational data from a remote network; and providing navigational services based upon said updated navigational data. Customized information is also communicated to the radio such as stock quotes, travel information, advertising, and e-mail. Onboard global positioning allows for channel updating by location, traffic information, geographic advertising and available similar content.