Abstract: Understanding why machine learning models behave the way they do empowers both system designers and end-users in many ways: in model selection, feature engineering, in order to trust and act upon the predictions, and in more intuitive user interfaces. Thus, interpretability has become a vital concern in machine learning, and work in the area of interpretable models has found renewed interest. In some applications, such models are as accurate as non-interpretable ones, and thus are preferred for their transparency. Even when they are not accurate, they may still be preferred when interpretability is of paramount importance. However, restricting machine learning to interpretable models is often a severe limitation. In this paper we argue for explaining machine learning predictions using model-agnostic approaches. By treating the machine learning models as black-box functions, these approaches provide crucial flexibility in the choice of models, explanations, and representations, improving debugging, comparison, and interfaces for a variety of users and models. We also outline the main challenges for such methods, and review a recently-introduced model-agnostic explanation approach (LIME) that addresses these challenges.

Abstract: FIT2D is one of the principal area detector data reduction, analysis and visualization programs used at the European Synchrotron Radiation Facility and is also used by more than 400 research groups worldwide, including many other synchrotron radiation facilities. It has been developed for X-ray science, but is applicable to other structural techniques and is used in analysing electron diffraction data and microscopy, and neutron diffraction and scattering data. FIT2D works for both interactive and `batch'-style data processing. Calibration and correction of detector distortions, integration of two-dimensional data to a variety of one-dimensional scans, and one- and two-dimensional model fitting are the main uses. Many other general-purpose image processing and image visualization operations are available. Commands are available through a `graphical user interface' and operations common to certain types of analysis are grouped within `interfaces'. Executable versions for most workstation and personal computer systems, and web page documentation, are available at http://www.esrf.eu/computing/scientific/FIT2D.

Abstract: With increasing use of computers in our daily lives, lately there has been a rapid increase in the efforts to develop a better human computer interaction interface. The need of easy to use and advance types of human-computer interaction with natural interfaces is more than ever. In the present framework, the UI (User Interface) of a computer allows user to interact with electronic devices with graphical icons and visual indicators, which is still inconvenient and not suitable for working in virtual environments. An interface which allow user to communicate through gestures is the next step in the direction of advance human computer interface. In the present paper author explore different aspects of gesture recognition techniques.

Abstract: Current telehealth usability questionnaires are designed primarily for older technologies, where telehealth interaction is conducted over dedicated videoconferencing applications. However, telehealth services are increasingly conducted over computer-based systems that rely on commercial software and a user supplied computer interface. Therefore, a usability questionnaire that addresses the changes in telehealth service delivery and technology is needed. The Telehealth Usability Questionnaire (TUQ) was developed to evaluate the usability of telehealth implementation and services. This paper addresses: (1) the need for a new measure of telehealth usability, (2) the development of the TUQ, (3) intended uses for the TUQ, and (4) the reliability of the TUQ. Analyses indicate that the TUQ is a solid, robust, and versatile measure that can be used to measure the quality of the computer-based user interface and the quality of the telehealth interaction and services.

Abstract: The popularity of mobile devices has been steadily growing in recent years. These devices heavily depend on software from the underlying operating systems to the applications they run. Prior research showed that mobile software is different than traditional, large software systems. However, to date most of our research has been conducted on traditional software systems. Very little work has focused on the issues that mobile developers face. Therefore, in this paper, we use data from the popular online Q&A site, Stack Overflow, and analyze 13,232,821 posts to examine what mobile developers ask about. We employ Latent Dirichlet allocation-based topic models to help us summarize the mobile-related questions. Our findings show that developers are asking about app distribution, mobile APIs, data management, sensors and context, mobile tools, and user interface development. We also determine what popular mobile-related issues are the most difficult, explore platform specific issues, and investigate the types (e.g., what, how, or why) of questions mobile developers ask. Our findings help highlight the challenges facing mobile developers that require more attention from the software engineering research and development communities in the future and establish a novel approach for analyzing questions asked on Q&A forums.

Abstract: An electronic device with a touch-sensitive surface, a display, and one or more sensors to detect intensity of contacts: displays a plurality of user interface objects in a first user interface; detects a contact while a focus selector is at a location of a first user interface object; and, while the focus selector is at the location of the first user interface object: detects an increase in a characteristic intensity of the contact to a first intensity threshold; in response, visually obscures the plurality of user interface objects, other than the first user interface object, while maintaining display of the first user interface object; detects that the characteristic intensity of the contact continues to increase above the first intensity threshold; and, in response, dynamically increases the amount of visual obscuring of the plurality of user interface objects, other than the first user interface object.

Abstract: This paper introduces swarm user interfaces, a new class of human-computer interfaces comprised of many autonomous robots that handle both display and interaction. We describe the design of Zooids, an open-source open-hardware platform for developing tabletop swarm interfaces. The platform consists of a collection of custom-designed wheeled micro robots each 2.6 cm in diameter, a radio base-station, a high-speed DLP structured light projector for optical tracking, and a software framework for application development and control. We illustrate the potential of tabletop swarm user interfaces through a set of application scenarios developed with Zooids, and discuss general design considerations unique to swarm user interfaces.

Abstract: The Soil and Water Assessment Tool (SWAT) model is a robust watershed modeling tool. It typically uses the ArcSWAT interface to create its inputs. ArcSWAT is public domain software which works in the licensed ArcGIS environment. The aim of this paper was to develop an open source user interface for the SWAT model. The interface, QSWAT, is written in the Python programming language and uses various functionalities of the open source geographic information system, QGIS. The current interface performs similar functions to ArcSWAT, but with additional enhanced features such as merging small subbasins and static and dynamic visualization of outputs. The interface is demonstrated through a case study in the Gumera watershed in the Lake Tana basin of Ethiopia, where it showed a successful performance. QSWAT will be a valuable tool for the SWAT scientific community, with improved availability and functionality compared with other options for creating SWAT models. Open source software called QSWAT was developed on the QGIS platform.The software creates input data and executes the SWAT model.It can provide static and dynamic visualization of outputs.A case study utilizing the software demonstrated a very good performance.The tool is beneficial to users in developing countries.

Abstract: Image analysis tools for brain magnetic resonance imaging (MRI) have become increasingly important for computer-aided diagnosis that involves large amounts of medical image data. The authors of this article have endeavored to develop software tools to serve the clinical research community, starting with a stand-alone executable, hybrid local computation model for today's modern architecture of cloud services, which they call MRICloud. MRICloud provides a high-throughput neuroinformatics platform for automated brain MRI segmentation and analytical tools for quantification via distributed remote computation and Web-based user interfaces. There are several key, inherent advantages to a cloud-based software as a service--in particular, how it improves the efficiency of software implementation, upgrades, and maintenance. The client-server model is also ideal for high-performance computing, allowing for distribution of computational servers across the world. This article introduces the basic functions and utilities of MRICloud, its developmental history and future perspectives, its infrastructures, and the benefits of this cloud service framework.

Abstract: User research is a broad term that encompasses many methodologies, such as usability testing, surveys, questionnaires, and site visits, that generate quantifiable outcomes. Usability testing is a central activity in user research and typically generates the metrics of completion rates, task times, errors, satisfaction data, and user interface problems. You can quantify data from small sample sizes and use statistics to draw conclusions. Even open-ended comments and problem descriptions can be categorized and quantified.

Abstract: Personal drones are becoming part of every day life. To fully integrate them into society, it is crucial to design safe and intuitive ways to interact with these aerial systems. The recent advances on User-Centered Design (UCD) applied to Natural User Interfaces (NUIs) intend to make use of human innate features, such as speech, gestures and vision to interact with technology in the way humans would with one another. In this paper, a Graphical User Interface (GUI) and several NUI methods are studied and implemented, along with computer vision techniques, in a single software framework for aerial robotics called Aerostack which allows for intuitive and natural human-quadrotor interaction in indoor GPS-denied environments. These strategies include speech, body position, hand gesture and visual marker interactions used to directly command tasks to the drone. The NUIs presented are based on devices like the Leap Motion Controller, microphones and small size monocular on-board cameras which are unnoticeable to the user. Thanks to this UCD perspective, the users can choose the most intuitive and effective type of interaction for their application. Additionally, the strategies proposed allow for multi-modal interaction between multiple users and the drone by being able to integrate several of these interfaces in one single application as is shown in various real flight experiments performed with non-expert users.

Abstract: The field of automotive user interfaces has developed rapidly over the last several years. To date, the field has primarily focused on creating user interfaces that promote safe driving, including when the driver is engaged in a secondary task in addition to operating the vehicle. However, researchers now need to prepare for a major change in the automotive domain: the automated driving revolution. The authors argue for a new research agenda that focuses on four challenges for automotive user interfaces: assuring safety in the age of automation, transforming vehicles into places for productivity and play, taking advantage of new mobility options made possible by automated vehicles, while throughout all this preserving user privacy and data security. This article is part of a special issue on smart vehicle spaces.

Abstract: The Internet of Things envisions integration, coordination, communication, and collaboration of real-world objects in order to perform daily tasks in a more intelligent and efficient manner. To comprehend this vision, this article studies the design of a large-scale IoT system for smart grid application, which constitutes a large number of home users and has the requirement of fast response time. In particular, we focus on the messaging protocol of a universal IoT home gateway, where our cloud enabled system consists of a back-end server, a unified home gateway (UHG) at the end users, and a user interface for mobile devices. We discuss the features of such an IoT system to support a large-scale deployment with a UHG and real-time residential smart grid applications. Based on the requirements, we design an IoT system using XMPP and implemented in a testbed for energy management applications. To show the effectiveness of the designed testbed, we present some results using the proposed IoT architecture.

Abstract: There is a rapid growth in the use of voice-controlled intelligent personal assistants on mobile devices, such as Microsoft's Cortana, Google Now, and Apple's Siri. They significantly change the way users interact with search systems, not only because of the voice control use and touch gestures, but also due to the dialogue-style nature of the interactions and their ability to preserve context across different queries. Predicting success and failure of such search dialogues is a new problem, and an important one for evaluating and further improving intelligent assistants. While clicks in web search have been extensively used to infer user satisfaction, their significance in search dialogues is lower due to the partial replacement of clicks with voice control, direct and voice answers, and touch gestures. In this paper, we propose an automatic method to predict user satisfaction with intelligent assistants that exploits all the interaction signals, including voice commands and physical touch gestures on the device. First, we conduct an extensive user study to measure user satisfaction with intelligent assistants, and simultaneously record all user interactions. Second, we show that the dialogue style of interaction makes it necessary to evaluate the user experience at the overall task level as opposed to the query level. Third, we train a model to predict user satisfaction, and find that interaction signals that capture the user reading patterns have a high impact: when including all available interaction signals, we are able to improve the prediction accuracy of user satisfaction from 71% to 81% over a baseline that utilizes only click and query features.

Abstract: Personalization and adaptivity can promote motivated usage, increased user acceptance, and user identification in serious games. This applies to heterogeneous user groups in particular, since they can benefit from customized experiences that respond to the individual traits of the players. In the context of games, adaptivity describes the automatic adaptation of game elements, i.e., of content, user interfaces, game mechanics, game difficulty, etc., to customize or personalize the interactive experience. Adaptation processes follow an adaptive cycle, changing a deployed system to the needs of its users. They can work with various techniques ranging from simple threshold-based parameter adjustment heuristics to complex evolving user models that are continuously updated over time. This chapter provides readers with an understanding of the motivation behind using adaptive techniques in serious games and presents the core challenges around designing and implementing such systems. Examples of how adaptability and adaptivity may be put into practice in specific application scenarios, such as motion-based games for health, or personalized learning games, are presented to illustrate approaches to the aforementioned challenges. We close with a discussion of the major open questions and avenues for future work.

Abstract: Applications and systems are constantly faced with decisions that require picking from a set of actions based on contextual information. Reinforcement-based learning algorithms such as contextual bandits can be very effective in these settings, but applying them in practice is fraught with technical debt, and no general system exists that supports them completely. We address this and create the first general system for contextual learning, called the Decision Service. Existing systems often suffer from technical debt that arises from issues like incorrect data collection and weak debuggability, issues we systematically address through our ML methodology and system abstractions. The Decision Service enables all aspects of contextual bandit learning using four system abstractions which connect together in a loop: explore (the decision space), log, learn, and deploy. Notably, our new explore and log abstractions ensure the system produces correct, unbiased data, which our learner uses for online learning and to enable real-time safeguards, all in a fully reproducible manner. The Decision Service has a simple user interface and works with a variety of applications: we present two live production deployments for content recommendation that achieved click-through improvements of 25-30%, another with 18% revenue lift in the landing page, and ongoing applications in tech support and machine failure handling. The service makes real-time decisions and learns continuously and scalably, while significantly lowering technical debt.

Abstract: This paper presents a smart home management system in which a community broker role is used for integrating community services, thereby reducing the workload of community management staff, providing electronic information services, and deepening the community's integration with the surrounding environment. At the home end, a home intranet was created by integrating a fixed touch panel with a home controller system and various sensors and devices to deliver, for example, energy, scenario information, and security functions. The community end comprises a community server and community personal computers, and connects to devices (e.g., video cameras and building automation devices) in other community systems and to the home networks. Furthermore, to achieve multiple inhome displays, standard interface devices can be employed to separate the logic and user interfaces. This study also determined that the message queuing telemetry transport protocol can provide optimal home control services in smart home systems, whereas hypertext transfer protocol is optimal for delivering location-based information integration services.

Abstract: An area of intensive research under the umbrella of the Internet of Things (IoT) has resulted in intensive proliferation of globally deployed sensor devices that provide a basis for the development of different use-case applications working with real-time data and demanding a rich user interface. Overcoming the lack of the standard HTML platform, HTML5 specifications WebSocket and Canvas graphics strongly supported the development of rich real-time applications. Such support has been offered by browser plug-ins such as Adobe Flash and Microsoft Silverlight for years. In order to provide a deep insight into IoT Web application performance, we implemented two test applications. In the first application, we measured latencies induced by different communication protocols and message encodings, as well as graphics rendering performance, while comparing the performance of different Web platform implementations. In the second application, we compared Web performance of IoT messaging protocols such as MQTT, AMQP, XMPP, and DDS by measuring the latency of sensor data message delivery and the message throughput rate. Our tests have shown that although Adobe Flash has the best performance at the moment, HTML5 platform is also very capable of running real-time IoT Web applications, whereas Microsoft Silverlight is noticeably behind both platforms. On the other hand, MQTT is the most appropriate messaging protocol for a wide set of IoT Web applications. However, IoT application developers should be aware of certain MQTT message broker implementation shortcomings that could prevent the usage of this protocol.

Abstract: We present BitDrones, a toolbox for building interactive real reality 3D displays that use nano-quadcopters as self-levitating tangible building blocks. Our prototype is a first step towards interactive self-levitating programmable matter, in which the user interface is represented using Catomic structures. We discuss three types of BitDrones: PixelDrones, equipped with an RGB LED and a small OLED display; ShapeDrones, augmented with an acrylic mesh spun over a 3D printed frame in a larger geometric shape; and DisplayDrones, fitted with a thin-film 720p touchscreen. We present a number of unimanual and bimanual input techniques, including touch, drag, throw and resize of individual drones and compound models, as well as user interface elements such as self-levitating cone trees, 3D canvases and alert boxes. We describe application scenarios and depict future directions towards creating high-resolution self-levitating programmable matter.

Abstract: A method include placing a first processor in a sleep operating mode and running a second processor that is operative to wake the first processor from the sleep operating mode in response to a speech command phrase. The method includes identifying, by the second processor, a speech command phrase segment and performing a control operation in response to detecting the segment in detected speech. The control operation is performed while the first processor is maintained in the sleep operating mode.

Abstract: Methods and systems are provided for generating automatic program recommendations based on user interactions. In some embodiments, control circuitry processes verbal data received during an interaction between a user of a user device and a person with whom the user is interacting. The control circuitry analyzes the verbal data to automatically identify a media asset referred to during the interaction by at least one of the user and the person with whom the user is interacting. The control circuitry adds the identified media asset to a list of media assets associated with the user of the user device. The list of media assets is transmitted to a second user device of the user.

Abstract: Tangible User Interfaces offer new ways of interaction with virtual objects, yet little research has been conducted on their learner-friendly design in the context of spatial learning. Although frameworks such as Embodied Cognition stress the importance of sensory perception and movement, studies have found that high interactivity can be overwhelming and may lead to a lower learning performance. In a 2?×?2 factorial design participants (n?=?96) learned heart anatomy using a 3D model that was either controlled using a mouse or a tangible object, i.e. a motion tracked plastic model of the virtual heart. Secondly, we varied the interaction mode featuring either a selective pointing mode in which only the label that the user currently activated was displayed or permanent display of all labels. Retention performance, cognitive load scores, and motivation measures indicate that the tangible object leads to significantly higher learning outcomes. The effect of the label display mode is different for the two input devices: The performance with selective pointing in the mouse condition is better than the performance with permanent display in the mouse condition; in the TUI condition this is exactly the other way around. Based on these results, we propose extensions for Embodied Cognition and Cognitive Load Theory. Multisensory learning is investigated using a tangible user interface.An interactive selective pointing of labels is contrasted with permanent display.Higher interactivity leads to worse learning performance with the tangible interface.We introduce Embodied Cognitive Load Theory as extension for Cognitive Load Theory.Interactivity and haptic perception are discussed in terms of a cost-benefit model.

Abstract: A content segmentation, categorization and identification method on consumer devices (clients) is described. Methods for content tracking are illustrated that are suitable for large scale deployment and applications such as broadcast monitoring, novel content publishing and interaction. Time-aligned (synchronous) applications such as multi-language selection, customized advertisements, second screen services and content monitoring applications can be economically deployed at large scales. The client performs fingerprinting, scene change detection, audio turn detection, and logo detection on incoming video and gathers database search results, logos and text to identify and segment video streams into content, promos, and commercials. A learning engine is configured to learn rules for optimal identification and segmentation at each client for each channel and program. Content sensed at the client site is tracked with reduced computation and applications are executed with timing precision. A method and user interface for time-aligned publishing of content and subsequent usage and interaction on one or more displays is described.

Abstract: Electronic devices with improved methods and interfaces for messaging are disclosed, including improved ways to: acknowledge messages; edit previously sent messages; express what a user is trying to communicate; display private messages; synchronize viewing of content between users; incorporate handwritten inputs; quickly locate content in a message transcript; integrate a camera; integrate search and sharing; integrate interactive applications; integrate stickers; make payments; interact with avatars; make suggestions; navigate among interactive applications; manage interactive applications; translate foreign language text; combine messages into a group; and flag messages.

Abstract: Internet-of-Things (IoTs) envisions to integrate, coordinate, communicate, and collaborate real-world objects in order to perform daily tasks in a more intelligent and efficient manner. To comprehend this vision, this paper studies the design of a large scale IoT system for smart grid application, which constitutes a large number of home users and has the requirement of fast response time. In particular, we focus on the messaging protocol of a universal IoT home gateway, where our cloud enabled system consists of a backend server, unified home gateway (UHG) at the end users, and user interface for mobile devices. We discuss the features of such IoT system to support a large scale deployment with a UHG and real-time residential smart grid applications. Based on the requirements, we design an IoT system using the XMPP protocol, and implemented in a testbed for energy management applications. To show the effectiveness of the designed testbed, we present some results using the proposed IoT architecture.

Abstract: Information technology (IT) capabilities are increasing at an impressive pace, but users’ cognitive abilities are not developing at the same speed. Thus, there is a gap between users’ abilities and available IT. Handbooks or online help functions such as ‘‘F1 help’’ try to close this gap by providing explanatory information for the IT capabilities at hand. However, there is strong empirical evidence that traditional support structures are not as effective as intended (Sykes 2015); on the contrary, they distract users from their work (Barrett et al. 2004), which results in decreased efficiency and effectiveness as well as lower job satisfaction. Initial attempts to support users with more comprehensive integrated assistance functions failed miserably. A well-known example of such a dismal failure is ‘‘Clippy, the paperclip’’, a cartoon character developed by Microsoft that automatically popped up to assist users of Microsoft Office. However, instead of supporting the user with clear and precise guidance, studies show that Clippy ‘‘was considered to be annoying, impolite, and disruptive of a user’s workflow’’ (Veletsianos 2007, p. 374). In the end, Clippy, the ‘‘non-intelligent artificial intelligence assistant’’, was so despised that even Microsoft made fun of it. However, more recent attempts to support users, primarily in the private-life context (on mobile devices), are becoming more sophisticated and have been met with a more favorable response (e.g., Apple’s Siri and Google’s Google Now). Moreover, Microsoft has integrated its personal assistant, Cortana, into the latest version of the operating system Windows 10, which is available for private and business environments. One domain that is far more mature with regard to ‘‘user’’ support is the automotive sector. For more than 30 years there has been research into assistance systems that proactively support drivers (Bengler et al. 2014). Early driver assistance systems (DAS) only measured the parameters inside the car, for example with regard to vehicle stabilization (electronic stability control). Later on, sensors also captured the car’s external environment. The use of the collected data, navigation systems, adaptive cruise control, and parking assistance can assist drivers in avoiding hazardous situations and increasing driver comfort. Advanced DAS, considered to be the third phase of DAS evolution, are about to become commercialized as Accepted after three revisions by Prof. Dr. Sinz.