Abstract: The Rosetta software for macromolecular modeling, docking and design is extensively used in laboratories worldwide. During two decades of development by a community of laboratories at more than 60 institutions, Rosetta has been continuously refactored and extended. Its advantages are its performance and interoperability between broad modeling capabilities. Here we review tools developed in the last 5 years, including over 80 methods. We discuss improvements to the score function, user interfaces and usability. Rosetta is available at http://www.rosettacommons.org.

Abstract: Currently, new technologies have enabled the design of smart applications that are used as decision-making tools in the problems of daily life. The key issue in designing such an application is the increasing level of user interaction. Mixed reality (MR) is an emerging technology that deals with maximum user interaction in the real world compared to other similar technologies. Developing an MR application is complicated, and depends on the different components that have been addressed in previous literature. In addition to the extraction of such components, a comprehensive study that presents a generic framework comprising all components required to develop MR applications needs to be performed. This review studies intensive research to obtain a comprehensive framework for MR applications. The suggested framework comprises five layers: the first layer considers system components; the second and third layers focus on architectural issues for component integration; the fourth layer is the application layer that executes the architecture; and the fifth layer is the user interface layer that enables user interaction. The merits of this study are as follows: this review can act as a proper resource for MR basic concepts, and it introduces MR development steps and analytical models, a simulation toolkit, system types, and architecture types, in addition to practical issues for stakeholders such as considering MR different domains.

Abstract: There is a growing body of research in the field of interaction between automated vehicles and other road users in their vicinity. To facilitate such interactions, researchers and designers have explored designs, and this line of work has yielded several concepts of external Human-Machine Interfaces (eHMI) for vehicles. Literature and media review reveals that the description of interfaces is often lacking in fidelity or details of their functionalities in specific situations, which makes it challenging to understand the originating concepts. There is also a lack of a universal understanding of the various dimensions of a communication interface, which has impeded a consistent and coherent addressal of the different aspects of the functionalities of such interface concepts. In this paper, we present a unified taxonomy that allows a systematic comparison of the eHMI across 18 dimensions, covering their physical characteristics and communication aspects from the perspective of human factors and human-machine interaction. We analyzed and coded 70 eHMI concepts according to this taxonomy to portray the state of the art and highlight the relative maturity of different contributions. The results point to a number of unexplored research areas that could inspire future work. Additionally, we believe that our proposed taxonomy can serve as a checklist for user interface designers and researchers when developing their interfaces.

Abstract: Mental health applications hold great promise as interventions for addressing common mental issues. Although many people with mental health issues use mobile app interventions, their adherence level remains low. Low engagement affects the effectiveness of mobile interventions. However, there is still a dearth of research to explain the reasons for low engagement. User experience and usability are two factors that determine the adoption and usage of apps. Analyzing user reviews of mobile apps for mental health issues reveals user experience and what features users liked and disliked in the apps and hence informs future app design and refinements. This research aims to analyze user reviews of publicly available mental health applications to uncover their strengths, weaknesses, and gaps, hence revealing why users are likely to cease using these applications. We mined reviews of 106 mental health apps retrieved from Apple's App Store and Google Play and employed thematic analysis on 13,549 reviews. The review analysis shows that users placed more emphasis on the user interface and the user-friendliness of the app. Users also appreciated apps that present them with a variety of options, functionalities, and content that they can choose. Again, apps that offer adaptive functionalities that allow users to adapt some app features also received high ratings. In contrast, poor usability emerged as the most common reason for abandoning mental health apps. Other pitfalls include lack of a content variety, lack of personalization, lack of customer service and trust, and security and privacy issues.

Abstract: Industrial standards define safety requirements for Human-Robot Collaboration (HRC) in industrial manufacturing. The standards particularly require real-time monitoring and securing of the minimum protective distance between a robot and an operator. This paper proposes a depth-sensor based model for workspace monitoring and an interactive Augmented Reality (AR) User Interface (UI) for safe HRC. The AR UI is implemented on two different hardware: a projector-mirror setup and a wearable AR gear (HoloLens). The workspace model and UIs are evaluated in a realistic diesel engine assembly task. The AR-based interactive UIs provide 21–24% and 57–64% reduction in the task completion and robot idle time, respectively, as compared to a baseline without interaction and workspace sharing. However, user experience assessment reveal that HoloLens based AR is not yet suitable for industrial manufacturing while the projector-mirror setup shows clear improvements in safety and work ergonomics.

Abstract: Robots are important in high-mix low-volume manufacturing because of their versatility and repeatability in performing manufacturing tasks. However, robots have not been widely used due to cumbersome programming effort and lack of operator skill. One significant factor prohibiting the widespread application of robots by small and medium enterprises (SMEs) is the high cost and necessary skill of programming and re-programming robots to perform diverse tasks. This paper discusses an Augmented Reality (AR) assisted robot programming system (ARRPS) that provides faster and more intuitive robot programming than conventional techniques. ARRPS is designed to allow users with little robot programming knowledge to program tasks for a serial robot. The system transforms the work cell of a serial industrial robot into an AR environment. With an AR user interface and a handheld pointer for interaction, users are free to move around the work cell to define 3D points and paths for the real robot to follow. Sensor data and algorithms are used for robot motion planning, collision detection and plan validation. The proposed approach enables fast and intuitive robotic path and task programming, and allows users to focus only on the definition of tasks. The implementation of this AR-assisted robot system is presented, and specific methods to enhance the performance of the users in carrying out robot programming using this system are highlighted.

Abstract: We present CAiRE, an end-to-end generative empathetic chatbot designed to recognize user emotions and respond in an empathetic manner. Our system adapts the Generative Pre-trained Transformer (GPT) to empathetic response generation task via transfer learning. CAiRE is built primarily to focus on empathy integration in fully data-driven generative dialogue systems. We create a web-based user interface which allows multiple users to asynchronously chat with CAiRE. CAiRE also collects user feedback and continues to improve its response quality by discarding undesirable generations via active learning and negative training.

Abstract: With the rapid progress of the semantic web, a huge amount of structured data has become available on the web in the form of knowledge bases (KBs). Making these data accessible and useful for end-users is one of the main objectives of chatbots over linked data. Building a chatbot over linked data raises different challenges, including user queries understanding, multiple knowledge base support, and multilingual aspect. To address these challenges, we first design and develop an architecture to provide an interactive user interface. Secondly, we propose a machine learning approach based on intent classification and natural language understanding to understand user intents and generate SPARQL queries. We especially process a new social network dataset (i.e., myPersonality) and add it to the existing knowledge bases to extend the chatbot capabilities by understanding analytical queries. The system can be extended with a new domain on-demand, flexible, multiple knowledge base, multilingual, and allows intuitive creation and execution of different tasks for an extensive range of topics. Furthermore, evaluation and application cases in the chatbot are provided to show how it facilitates interactive semantic data towards different real application scenarios and showcase the proposed approach for a knowledge graph and data-driven chatbot.

Abstract: We present a new problem: grounding natural language instructions to mobile user interface actions, and create three new datasets for it. For full task evaluation, we create PIXELHELP, a corpus that pairs English instructions with actions performed by people on a mobile UI emulator. To scale training, we decouple the language and action data by (a) annotating action phrase spans in HowTo instructions and (b) synthesizing grounded descriptions of actions for mobile user interfaces. We use a Transformer to extract action phrase tuples from long-range natural language instructions. A grounding Transformer then contextually represents UI objects using both their content and screen position and connects them to object descriptions. Given a starting screen and instruction, our model achieves 70.59% accuracy on predicting complete ground-truth action sequences in PIXELHELP.

Abstract: We present ConvLab-2, an open-source toolkit that enables researchers to build task-oriented dialogue systems with state-of-the-art models, perform an end-to-end evaluation, and diagnose the weakness of systems As the successor of ConvLab, ConvLab-2 inherits ConvLab’s framework but integrates more powerful dialogue models and supports more datasets Besides, we have developed an analysis tool and an interactive tool to assist researchers in diagnosing dialogue systems The analysis tool presents rich statistics and summarizes common mistakes from simulated dialogues, which facilitates error analysis and system improvement The interactive tool provides an user interface that allows developers to diagnose an assembled dialogue system by interacting with the system and modifying the output of each system component

Abstract: An emerging generation of visualization authoring systems support expressive information visualization without textual programming. As they vary in their visualization models, system architectures, and user interfaces, it is challenging to directly compare these systems using traditional evaluative methods. Recognizing the value of contextualizing our decisions in the broader design space, we present critical reflections on three systems we developed —Lyra, Data Illustrator, and Charticulator. This paper surfaces knowledge that would have been daunting within the constituent papers of these three systems. We compare and contrast their (previously unmentioned) limitations and trade-offs between expressivity and learnability. We also reflect on common assumptions that we made during the development of our systems, thereby informing future research directions in visualization authoring systems.

Abstract: Graphical User Interface (GUI) elements detection is critical for many GUI automation and GUI testing tasks. Acquiring the accurate positions and classes of GUI elements is also the very first step to conduct GUI reverse engineering or perform GUI testing. In this paper, we implement a User Iterface Element Detection (UIED), a toolkit designed to provide user with a simple and easy-to-use platform to achieve accurate GUI element detection. UIED integrates multiple detection methods including old-fashioned computer vision (CV) approaches and deep learning models to handle diverse and complicated GUI images. Besides, it equips with a novel customized GUI element detection methods to produce state-of-the-art detection results. Our tool enables the user to change and edit the detection result in an interactive dashboard. Finally, it exports the detected UI elements in the GUI image to design files that can be further edited in popular UI design tools such as Sketch and Photoshop. UIED is evaluated to be capable of accurate detection and useful for downstream works. Tool URL: http://uied.online Github Link: https://github.com/MulongXie/UIED

Abstract: The emergence of high throughput technologies that produce vast amounts of genomic data, such as next-generation sequencing (NGS) is transforming biological research. The dramatic increase in the volume of data, the variety and continuous change of data processing tools, algorithms and databases make analysis the main bottleneck for scientific discovery. The processing of high throughput datasets typically involves many different computational programs, each of which performs a specific step in a pipeline. Given the wide range of applications and organizational infrastructures, there is a great need for highly parallel, flexible, portable, and reproducible data processing frameworks. Several platforms currently exist for the design and execution of complex pipelines. Unfortunately, current platforms lack the necessary combination of parallelism, portability, flexibility and/or reproducibility that are required by the current research environment. To address these shortcomings, workflow frameworks that provide a platform to develop and share portable pipelines have recently arisen. We complement these new platforms by providing a graphical user interface to create, maintain, and execute complex pipelines. Such a platform will simplify robust and reproducible workflow creation for non-technical users as well as provide a robust platform to maintain pipelines for large organizations. To simplify development, maintenance, and execution of complex pipelines we created DolphinNext. DolphinNext facilitates building and deployment of complex pipelines using a modular approach implemented in a graphical interface that relies on the powerful Nextflow workflow framework by providing 1. A drag and drop user interface that visualizes pipelines and allows users to create pipelines without familiarity in underlying programming languages. 2. Modules to execute and monitor pipelines in distributed computing environments such as high-performance clusters and/or cloud 3. Reproducible pipelines with version tracking and stand-alone versions that can be run independently. 4. Modular process design with process revisioning support to increase reusability and pipeline development efficiency. 5. Pipeline sharing with GitHub and automated testing 6. Extensive reports with R-markdown and shiny support for interactive data visualization and analysis. DolphinNext is a flexible, intuitive, web-based data processing and analysis platform that enables creating, deploying, sharing, and executing complex Nextflow pipelines with extensive revisioning and interactive reporting to enhance reproducible results.

Abstract: Recent advances in information technology and improved access to the internet have led to a rapid increase in the adoption and ownership of electronic devices such as touch screen smartphones and tablet computers. This has also led to a renewed interest in the field of digital health also referred to as telehealth or electronic health (eHealth). There is now a drive to collect these PROs electronically using ePRO systems. However, the user interfaces of ePRO systems need to be adequately assessed to ensure they are not only fit for purpose but also acceptable to patients who are the end users. Usability testing is a technique that involves the testing of systems, products or websites with participants drawn from the target population. Usability testing can assist ePRO developers in the evaluation of ePRO user interface. The complexity of ePRO systems; stage of development; metrics to measure; and the use of scenarios, moderators and appropriate sample sizes are key methodological issues to consider when planning usability tests. The findings from usability testing may facilitate the improvement of ePRO systems making them more usable and acceptable to end users. This may in turn improve the adoption of ePRO systems post-implementation. This article highlights the key methodological issues to consider and address when planning usability testing of ePRO systems.

Abstract: The ability to combine digital information with the real world enables mixed reality (MR) technology to provide a better display of information, resulting in its increasing popularity in va...

Abstract: In order to achieve information visualization, realize good interaction between users and information, and meet the needs of users, this study first studied the interaction behavior of the user when using the smartphone was studied, and analyzed the visual factors of the smartphone interface were analyzed from the user sensory interaction level, and the user operation mode level, from the expression of visual form to the commonly used interface mode and User Interface (UI) component space. On this basis, the situational visual expression of the scene in different interaction scenarios was analyzed. Secondly, the basic theory of visual design of smartphone application interface was explained from the perspectives of aesthetics, semiotics and Gestalt psychology, In other words, the visual design of the application interface should be metaphorical, highlighting the key points in the overall visual style, and conforming to the user’s psychological model. At the same time, in order to meet the user’s personalized needs for control, it must add customized options. Finally, the model of the interface visual design method of the interdisciplinary “Shared Communication” system for the interface design of the mobile APP was constructed, and the case of Didi Chuxing was analyzed, which preliminarily confirmed the feasibility of the construction of the interface visual design method model of the “Shared Communication” system.

Abstract: The aim of this study is to examine adaptation elements and Intelligent Tutoring System (ITS) elements used in Adaptive Intelligent Tutoring Systems (AITSs), using meta-synthesis methods to analyze the results of previous research. Toward this end, articles appearing in the Web of Science, Google Scholar, Eric and Science Direct databases in 2000 and later were identified with the keyphrase “adaptive intelligent tutoring system.” Application of exclusion and inclusion procedures to the articles accessed in the search resulted in the selection of 32 articles, which were analyzed using meta-synthesis methods and then evaluated in the light of prespecified themes and elements used in AITSs were determined. According to results, the systems were designed for a wide range of fields such as Information Technologies, Mathematics, Science, Medicine, and Foreign Language Education. In these systems, content adaptation was generally used, based mostly on such criteria as feedback, student level, student learning and cognitive styles, and student performance. And besides 4 basic ITS modules (knowledge, student, teaching and user interface), some different modules such as guide module, strategy module, personal learning module, knowledge base module, communication module, system administrator module and messaging module were used. Finally, some suggestions were given for such studies in the future.

Abstract: A robot-assisted feeding system can potentially help a user with upper-body mobility impairments eat independently. However, autonomous assistance in the real world is challenging because of varying user preferences, impairment constraints, and possibility of errors in uncertain and unstructured environments. An autonomous robot-assisted feeding system needs to decide the appropriate strategy to acquire a bite of hard-to-model deformable food items, the right time to bring the bite close to the mouth, and the appropriate strategy to transfer the bite easily. Our key insight is that a system should be designed based on a user’s preference about these various challenging aspects of the task. In this work, we explore user preferences for different modes of autonomy given perceived error risks and also analyze the effect of input modalities on technology acceptance. We found that more autonomy is not always better, as participants did not have a preference to use a robot with partial autonomy over a robot with low autonomy. In addition, participants’ user interface preference changes from voice control during individual dining to web-based during social dining. Finally, we found differences on average ratings when grouping the participants based on their mobility limitations (lower vs. higher) that suggests that ratings from participants with lower mobility limitations are correlated with higher expectations of robot performance. CCS CONCEPTS • Human-centered computing $\rightarrow$ Empirical studies in accessibility; • Social and professional topics $\rightarrow$ People with disabilities; Assistive technologies; • Computer systems organization $\rightarrow$ Robotic autonomy. ACM Reference Format: Tapomayukh Bhattacharjee, Ethan K. Gordon, Rosario Scalise, Maria E. Cabrera, Anat Caspi, Maya Cakmak, and Siddhartha S. Srinivasa. 2020. Is More Autonomy Always Better? Exploring Preferences of Users with Mobility Impairments in Robot-assisted Feeding. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (HRI’20), March 23-26, 2020, Cambridge, UK. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3319502.3374818

Abstract: Writing or sketching on whiteboards is an essential part of collaborative discussions in business meetings, reading groups, design sessions, and interviews. However, prior work in collaborative virtual reality (VR) systems has rarely explored the design space of multi-user layouts and interaction modes with virtual whiteboards. In this paper, we present CollaboVR, a reconfigurable framework for both co-located and geographically dispersed multi-user communication in VR. Our system unleashes users’ creativity by sharing freehand drawings, converting 2D sketches into 3D models, and generating procedural animations in real-time. To minimize the computational expense for VR clients, we leverage a cloud architecture in which the computational expensive application (Chalktalk) is hosted directly on the servers, with results being simultaneously streamed to clients. We have explored three custom layouts – integrated, mirrored, and projective – to reduce visual clutter, increase eye contact, or adapt different use cases. To evaluate CollaboVR, we conducted a within-subject user study with 12 participants. Our findings reveal that users appreciate the custom configurations and real-time interactions provided by CollaboVR. We have open sourced CollaboVR at https://github.com/snowymo/CollaboVR to facilitate future research and development of natural user interfaces and real-time collaborative systems in virtual and augmented reality.

Abstract: A home automation system controls lighting, temperature, multimedia systems, and appliances. Since these devices and sensors are connected to common infrastructure, they form the Internet of Things. A home automation system links multiple controllable devices to a centralized server. These devices have a user interface for controlling and monitoring, which can be accessed by using a tablet or a mobile application, which can be accessed remotely as well. Ideally, anything that can be connected to a network can be automated and controlled remotely. Smart homes must be artificially intelligent systems that need to adapt themselves based on user actions and surroundings. These systems need to carefully analyze the user needs and the conditions of the surroundings in order to predict future actions and also minimizes user interaction. Traditional home automation systems that provide only remote access and control are not that effective in terms of being ‘smart’, so in this paper we put forward the use of concepts of different machine learning algorithms along with computer vision to shape together a smart learning automated system that controls lighting, sound and other devices based on the user’s emotion.

Abstract: As increasingly complex software is developed every day, a growing number of companies use static analysis tools to reason about program properties ranging from simple coding style rules to more advanced software bugs, to multi-tier security vulnerabilities While increasingly complex analyses are created, developer support must also be updated to ensure that the tools are used to their best potential Past research in the usability of static analysis tools has primarily focused on usability issues encountered by software developers, and the causes of those issues in analysis tools In this article, we adopt a more user-centered approach, and aim at understanding why software developers use analysis tools, which decisions they make when using those tools, what they look for when making those decisions, and the motivation behind their strategies This approach allows us to derive new tool requirements that closely support software developers (eg, systems for recommending warnings to fix that take developer knowledge into account), and also open novel avenues for further static-analysis research such as collaborative user interfaces for analysis warnings

Abstract: In this article we introduce mobileQ, which is a free, open-source platform that our lab has developed to use in experience sampling studies. Experience sampling has several strengths and is becoming more widely conducted, but there are few free software options. To address this gap, mobileQ has freely available servers, a web interface, and an Android app. To reduce the barrier to entry, it requires no high-level programming and uses an easy, point-and-click interface. It is designed to be used on dedicated research phones, allowing for experimenter control and eliminating selection bias. In this article, we introduce setting up a study in mobileQ, outline the set of help resources available for new users, and highlight the success with which mobileQ has been used in our lab.

Abstract: Language User Interfaces (LUIs) could improve human-machine interaction for a wide variety of tasks, such as playing music, getting insights from databases, or instructing domestic robots. In contrast to traditional hand-crafted approaches, recent work attempts to build LUIs in a data-driven way using modern deep learning methods. To satisfy the data needs of such learning algorithms, researchers have constructed benchmarks that emphasize the quantity of collected data at the cost of its naturalness and relevance to real-world LUI use cases. As a consequence, research findings on such benchmarks might not be relevant for developing practical LUIs. The goal of this paper is to bootstrap the discussion around this issue, which we refer to as the benchmarks' low ecological validity. To this end, we describe what we deem an ideal methodology for machine learning research on LUIs and categorize five common ways in which recent benchmarks deviate from it. We give concrete examples of the five kinds of deviations and their consequences. Lastly, we offer a number of recommendations as to how to increase the ecological validity of machine learning research on LUIs.

Abstract: Text entry is a common activity in virtual reality (VR) systems. There is a limited number of available hands-free techniques, which allow users to carry out text entry when users’ hands are busy such as holding items or hand-based devices are not available. The most used hands-free text entry technique is DwellType, where a user selects a letter by dwelling over it for a specific period. However, its performance is limited due to the fixed dwell time for each character selection. In this paper, we explore two other hands-free text entry mechanisms in VR: BlinkType and NeckType, which leverage users’ eye blinks and neck’s forward and backward movements to select letters. With a user study, we compare the performance of the two techniques with DwellType. Results show that users can achieve an average text entry rate of 13.47, 11.18 and 11.65 words per minute with BlinkType, NeckType, and DwellType, respectively. Users’ subjective feedback shows BlinkType as the preferred technique for text entry in VR. Index Terms: Human-centered computing—Human computer interaction (HCI)—Interaction paradigms—Virtual reality; Human-centered computing—Human computer interaction (HCI)—Interaction techniques—Text input