Abstract: Abstract The SAT solver CaDiCaL provides a rich feature set with a clean library interface. It has been adopted by many users, is well documented and easy to extend due to its effective testing and debugging infrastructure. In this tool paper we give a high-level introduction into the solver architecture and then go briefly over implemented techniques. We describe basic features and novel advanced usage scenarios. Experiments confirm that CaDiCaL despite this flexibility has state-of-the-art performance both in a stand-alone as well as incremental setting.

Abstract: Fault Localization (FL), in which a developer seeks to identify which part of the code is malfunctioning and needs to be fixed, is a recurring challenge in debugging. To reduce developer burden, many automated FL techniques have been proposed. However, prior work has noted that existing techniques fail to provide rationales for the suggested locations, hindering developer adoption of these techniques. With this in mind, we propose AutoFL, a Large Language Model (LLM)-based FL technique that generates an explanation of the bug along with a suggested fault location. AutoFL prompts an LLM to use function calls to navigate a repository, so that it can effectively localize faults over a large software repository and overcome the limit of the LLM context length. Extensive experiments on 798 real-world bugs in Java and Python reveal AutoFL improves method-level acc@1 by up to 233.3% over baselines. Furthermore, developers were interviewed on their impression of AutoFL-generated explanations, showing that developers generally liked the natural language explanations of AutoFL, and that they preferred reading a few, high-quality explanations instead of many.

Abstract: Teaching and learning in higher education require adaptation following students' inevitable use of AI chatbots. This study contributes to the empirical literature on students' use of AI chatbots and how they influence learning. The aim of this study is to identify how to adapt programming education in higher engineering education. A mixed-methods case study was conducted of a scientific computing course in a Mechanical Engineering Master's program at a University of Technology in [blinded for review]. Data consisted of 29 student questionnaires, a semi-structured group interview with three students, a semi-structured interview with the teacher, and 29 students' grades. Results show that students used ChatGPT for error checking and debugging of code, increasing conceptual understanding, generating, and optimizing solution code, explaining code, and solving mathematical problems. While students reported advantages of using ChatGPT, the teacher expressed concerns over declining code quality and student learning. Furthermore, both students and teacher perceived a negative influence from ChatGPT usage on pair programming, and consequently on student collaboration. The findings suggest that learning objectives should be formulated in more detail, to highlight essential programming skills, and be expanded to include the use of AI tools. Complex programming assignments remain appropriate in programming education, but pair programming as a didactic approach should be reconsidered in light of the growing use of AI Chatbots.

Abstract: When a bug is detected by testing a quantum program on a quantum computer, we want to determine its location to fix it. To locate the bug, the quantum program is divided into several segments, and each segment is tested. However, to prepare a quantum state that is input to a segment, it is necessary to execute all the segments ahead of that segment in a quantum computer. This means that the cost of testing each segment depends on its location. We can also locate a buggy segment only if it is confirmed that there are no bugs in all segments ahead of that buggy segment. Since a quantum program is tested statistically on the basis of measurement results, there is a tradeoff between testing accuracy and cost. Although these characteristics are unique to quantum programs and complicate locating bugs, they have not been investigated. We suggest for the first time that these characteristics should be considered to efficiently locate bugs. We are also the first to propose a bug-locating method that takes these characteristics into account. The results from experiments indicate that the bug-locating cost, represented as the number of executed quantum gates, can be reduced with the proposed method compared with naive methods.

Abstract: As the reconstruction of Genome-Scale Metabolic Models (GEMs) becomes standard practice in systems biology, the number of organisms having at least one metabolic model is peaking at an unprecedented scale. The automation of laborious tasks, such as gap-finding and gap-filling, allowed the development of GEMs for poorly described organisms. However, the quality of these models can be compromised by the automation of several steps, which may lead to erroneous phenotype simulations. Biological networks constraint-based In Silico Optimisation (BioISO) is a computational tool aimed at accelerating the reconstruction of GEMs. This tool facilitates manual curation steps by reducing the large search spaces often met when debugging in silico biological models. BioISO uses a recursive relation-like algorithm and Flux Balance Analysis (FBA) to evaluate and guide debugging of in silico phenotype simulations. The potential of BioISO to guide the debugging of model reconstructions was showcased and compared with the results of two other state-of-the-art gap-filling tools (Meneco and fastGapFill). In this assessment, BioISO is better suited to reducing the search space for errors and gaps in metabolic networks by identifying smaller ratios of dead-end metabolites. Furthermore, BioISO was used as Meneco's gap-finding algorithm to reduce the number of proposed solutions for filling the gaps.

Abstract: Part-prototype models are explainable-by-design image classifiers, and a promising alternative to black box AI. This paper explores the applicability and potential of interpretable machine learning, in particular PIP-Net, for automated diagnosis support on real-world medical imaging data. PIP-Net learns human-understandable prototypical image parts and we evaluate its accuracy and interpretability for fracture detection and skin cancer diagnosis. We find that PIP-Net’s decision making process is in line with medical classification standards, while only provided with image-level class labels. Because of PIP-Net’s unsupervised pretraining of prototypes, data quality problems such as undesired text in an X-ray or labelling errors can be easily identified. Additionally, we are the first to show that humans can manually correct the reasoning of PIP-Net by directly disabling undesired prototypes. We conclude that part-prototype models are promising for medical applications due to their interpretability and potential for advanced model debugging.

Abstract: Students learning computer science frequently struggle with debugging errors in their code. These struggles can have significant downstream effects—negatively influencing how students assess their programming ability and contributing to their decision to drop out of CS courses. However, debugging instruction is often an overlooked topic, and instructors report feeling unaware of effective approaches to teach debugging. Within the literature, research on the topic is sporadic, and though there are rigorous and insightful studies to be found, there is a need to synthesize instructional approaches for debugging. In this paper, we review research from 2010 to 2022 on debugging interventions. We summarize the common pedagogical approaches for learning and categorize how these target specific cognitive and non-cognitive debugging skills, such as self-efficacy and emotion regulation. We also present a summary of assessment methods and their outcomes in order to discuss intervention efficacy and directions for further research. Our sample displays a diverse variety of debugging interventions and pedagogical approaches, ranging from games to unplugged activities. An evaluation of paper results also presents encouraging findings, revealing several interventions that improved debugging accuracy and learning. Still, we notice gaps in interventions addressing non-cognitive debugging skills, and observe limited success in guiding students toward adopting systematic debugging strategies. The review concludes with a discussion of future directions and implications for researchers and instructors in the field.

Abstract: Steering wheel angle is an important and essential parameter of the navigation control of autonomous wheeled vehicles. At present, the combination of rotary angle sensors and four-link mechanisms is the main sensing approach for steering wheel angle with high measurement accuracy, which is widely adopted in autonomous agriculture vehicles. However, in a complex and challenging farmland environment, there are a series of prominent problems such as complicated installation and debugging, spattered mud blocking the parallel four-bar mechanism, breakage of the sensor wire during operation, and separate calibrations for different vehicles. To avoid the above problems, a novel dynamic measurement method for steering wheel angle is presented based on vehicle attitude information and a non-contact attitude sensor. First, the working principle of the proposed measurement method and the effect of zero position error on measurement accuracy and path tracking are analyzed. Then, an optimization algorithm for zero position error of steering wheel angle is proposed. The experimental platform is assembled based on a 2ZG-6DM rice transplanter by software design and hardware modification. Finally, comparative tests are conducted to demonstrate the effectiveness and priority of the proposed dynamic sensing method. Experimental results show that the average absolute error of the straight path is 0.057° and the corresponding standard deviation of the error is 0.483°. The average absolute error of the turning path is 0.686° and the standard deviation of the error is 0.931°. This implies the proposed dynamic sensing method can accurately realize the collection of the steering wheel angle. Compared to the traditional measurement method, the proposed dynamic sensing method greatly improves the measurement reliability of the steering wheel angle and avoids complicated installation and debugging of different vehicles. The separate calibrations for different vehicles are not needed since the proposed measurement method is not dependent on the kinematic models of the vehicles. Given that the attitude sensor can be installed at a higher position on the wheel, sensor damage from mud blocking and the sensor wire breaking is also avoided.

Abstract: Summary Power system monitoring relies on the reliability of state estimation (SE) results. SE plays a dominant role in data debugging if sufficient data is available. Criticality analysis (CA) integrates SE as a module in which measurements—taken one‐by‐one or in groups (tuples) of minimal cardinality—are designated crucial. The combinatorial nature of extensive CA (not restricted to identifying low‐cardinality critical tuples) characterizes its computational complexity and imposes challenging limits to go beyond. In simple terms, these limits are established by the number of measurements to be combined, the cardinality of tuples, and the computing time required to check the criticality condition. This paper proposes an innovative computational solution to expand CA limits found to date in the literature. A framework with multi‐threads designed cleverly on a graphics processing unit (GPU) parallel processing environment is built. The conceived architecture favors evaluating massive measurement combinations of diverse cardinality in extensive CA. Numerical results reveal significant speed‐ups with the proposed approach, contrasting with those reported in research efforts published so far.

Abstract: Large Language Models (LLMs) now excel at generative skills and can create content at impeccable speeds.However, they are imperfect and still make various mistakes.In a Computer Science education context, as these models are widely recognized as "AI pair programmers," it becomes increasingly important to train students on evaluating and debugging the LLM-generated code.In this work, we introduce HypoCompass, a novel system to facilitate deliberate practice on debugging, where human novices play the role of Teaching Assistants and help LLM-powered teachable agents debug code.We enable effective task delegation between students and LLMs in this learning-by-teaching environment: students focus on hypothesizing the cause of code errors, while adjacent skills like code completion are offloaded to LLM-agents.Our evaluations demonstrate that HypoCompass generates high-quality training materials (e.g., bugs and fixes), outperforming human counterparts fourfold in efficiency, and significantly improves student performance on debugging by 12% in the pre-to-post test.

Abstract: Abstract Automated debugging techniques have the potential to reduce developer effort in debugging. However, while developers want rationales for the provided automatic debugging results, existing techniques are ill-suited to provide them, as their deduction process differs significantly froof human developers. Inspired by the way developers interact with code when debugging, we propose Automated Scientific Debugging ( AutoSD ), a technique that prompts large language models to automatically generate hypotheses, uses debuggers to interact with buggy code, and thus automatically reach conclusions prior to patch generation. In doing so, we aim to produce explanations of how a specific patch has been generated, with the hope that these explanations will lead to enhanced developer decision-making. Our empirical analysis on three program repair benchmarks shows that AutoSD performs competitively with other program repair baselines, and that it can indicate when it is confident in its results. Furthermore, we perform a human study with 20 participants to evaluate AutoSD -generated explanations. Participants with access to explanations judged patch correctness more accurately in five out of six real-world bugs studied. Furthermore, 70% of participants answered that they wanted explanations when using repair tools, and 55% answered that they were satisfied with the Scientific Debugging presentation.

Abstract: Synthetic data created by differentially private (DP) generative models is increasingly used in real-world settings. In this context, PATE-GAN has emerged as a popular algorithm, combining Generative Adversarial Networks (GANs) with the private training approach of PATE (Private Aggregation of Teacher Ensembles). In this paper, we analyze and benchmark six open-source PATE-GAN implementations, including three by (a subset of) the original authors. First, we shed light on architecture deviations and empirically demonstrate that none replicate the utility performance reported in the original paper. Then, we present an in-depth privacy evaluation, including DP auditing, showing that all implementations leak more privacy than intended and uncovering 17 privacy violations and 5 other bugs. Our codebase is available from https://github.com/spalabucr/pategan-audit.

Abstract: Web applications are often built as Single Page Applications (SPA), for example applications offered by Google, Facebook, Twitter or Netflix. Users interact with SPAs through a single HTML page that is dynamically rewritten with new data from the web server (instead of a web browser that loads entire new HTML pages). Just like with any type of software system, debugging is a common activity during the development and maintenance of SPAs. In order to fix bugs observed during runtime, developers often try to reproduce the bug first to better understand it. However, research has shown that reproducing bugs is not always possible. In this paper we (i) develop a technique for Application Monitoring (AM) to collect data to support bug reproduction; and (ii) apply the monitoring technique in a SPA test bed as well as a real-world SPA application to show its feasibility. As part of our research we developed an initial version of the AM technique and implemented it in a prototype. Our evaluation using this prototype showed that it not only improves the efficiency of the bug reproduction process but also reduces information gaps caused by incomplete bug reports submitted by users. Additionally, compared to the information provided by users, data provided by AM is more accurate and detailed and covers a wider range of data. Future work includes deploying the AM framework in more SPAs and investigating how AM can be integrated into software developer workflows.

Abstract: Large language models (LLMs) have shown remarkable abilities to generate code. However, their ability to develop software for physical computing and embedded systems, which requires cross-domain hardware and software knowledge, has not been thoroughly studied. We observe through our experiments and a 15-user pilot study that even when LLMs fail to produce working code, they can generate helpful reasoning about embedded design tasks, as well as specific debugging suggestions for both novice and expert developers. These results highlight the potential to develop AI assistants to dramatically lower the barrier to entry for working with hardware. To evaluate the capabilities and limitations of LLMs, we develop an automated testbench to quantify LLM performance on embedded programming tasks and perform 450 trials. We leverage these findings to analyze how programmers interact with these tools including their productivity and sense of fulfillment and outline a human-AI collaborative workflow for developing and debugging embedded systems.

Abstract: Despite advancements in IDE tooling, code understanding, generation, and automated repair, debugging continues to present significant challenges. Existing debugging strategies available to developers in literature are often too mechanical and rigid for day-to-day issues. Recent advances in Large Language Models (LLMs) promise practical solutions that allow for more free-form debugging strategies. While LLMs offer satisfactory assistance in some cases, they often leap to action without sufficient context, making implicit assumptions and providing inaccurate responses. Moreover, the dialogue between developers and LLMs predominantly takes the form of question-answer pairs, placing the burden of formulating the correct questions and sustaining multi-turn conversations on the developer. We introduce Robin, a novel multi-agent conversational AI-assistant within GitHub Copilot Chat, specifically designed for debugging. Robin moves beyond the question-answer pairs by introducing the investigate & respond pattern, that focuses on using information gathered automatically from the IDE or gathered interactively from the developer before responding. Robin incorporates a general debugging strategy to systematically analyze bugs to sustain collaborative interactions while ensuring that the conversation does not deviate from the debugging task at hand. Through a within-subjects user study with 16 industry professionals, we find that equipping Robin to-(1) leverage the insert expansion interaction pattern, (2) facilitate turn-taking, and (3) utilize debugging strategies-leads to lowered conversation barriers, a 2.5 x improvement in bug localization and a substantial 3.5x improvement in bug resolution compared to AI-assisted debugging in Visual Studio prior to Robin.

Abstract: In recent years, there has been an exponential growth in the size and complexity of System-on-Chip designs targeting different specialized applications. The cost of an undetected bug in these systems is much higher than in traditional processor systems as it may imply the loss of property or life. The problem is further exacerbated by the ever-shrinking time-to-market and ever-increasing demand to churn out billions of devices. Despite decades of research in simulation and formal methods for debugging and verification, it is still one of the most time-consuming and resource intensive processes in contemporary hardware design cycle. In this work, we propose VeriBug, which leverages recent advances in deep learning to accelerate debugging at the Register-Transfer Level and generates explanations of likely root causes. First, VeriBug uses control-data flow graph of a hardware design and learns to execute design statements by analyzing the context of operands and their assignments. Then, it assigns an importance score to each operand in a design statement and uses that score for generating explanations for failures. Finally, VeriBug produces a heatmap highlighting potential buggy source code portions. Our experiments show that VeriBug can achieve an average bug localization coverage of 82.5% on open-source designs and different types of injected bugs.

Abstract: Autonomous driving (AD) system designers need methods to efficiently debug vulnerabilities found in control algorithms. Existing methods lack alignment to the requirements of AD control designers to provide an analysis of the parameters of the AD system and how they are affected by cyber-attacks. We introduce ADAssure, a methodology for debugging AD control system algorithms that incorporates automated mechanisms which support generation of assertions to guide the AD system designer to identify vulnerabilities in the system. Our evaluation of ADAssure on a real-world AD vehicular system using diverse cyber-attacks developed a set of assertions that identified weaknesses in the OpenPlanner 2.5 AD planning algorithm and its constituent planning functions. Working with an AD control system designer and safety validation engineer, the results of ADAssure identified remediation of the AD control system, which can support the implementation of a redundant observer for data integrity checking and improvements to the planning algorithm. The adoption of ADAssure improves autonomous system design by providing a systematic approach to enhance safety and reliability through the identification and mitigation of vulnerabilities from corner cases.

Abstract: 3D printing is a revolutionary technology that enables the creation of physical objects from digital models. However, the quality and accuracy of 3D printing depend on the correctness and efficiency of the numerical control programming language (specifically, G-code) that instructs 3D printers on moving and extruding material. Debugging G-code, a low-level programming language for 3D printing, is a challenging task that requires manual tuning and geometric reasoning. In this paper, we present the first extensive evaluation of numerous large language models (LLMs) for debugging G-code files for 3-axis 3D printing. We design effective prompts to enable pre-trained LLMs to understand and manipulate G-code and test their performance on various aspects of G-code debugging and manipulation, including detection and correction of common errors and the ability to perform geometric transformations. We compare different state-of-the-art LLMs and analyze their strengths and weaknesses. We also discuss the implications and limitations of using LLMs for G-code comprehension and suggest directions for future research.

Abstract: Information Retrieval Based Bug Localization (IRBL) techniques are well suited for large-scale software debugging with fewer external dependencies and lower execution costs. However, existing IRBL techniques have several challenges, including localization granularity and applicability. First, existing IRBL techniques have not yet achieved statement-level bug localization. Second, almost all studies are limited to Java-based projects, while their effectiveness for other popular programming languages (e.g., Python) is unknown. The reason for these deficiencies is that existing IRBL techniques mainly rely on conventional NLP techniques to analyze the bug reports and have not yet fully utilized the stack traces attached to the bug reports. To improve the IRBL technique, we propose a context-aware program simplification technique – COPS – that can localize defective statements in suspicious files by analyzing the stack traces in bug reports, enabling statement-level bug localization for Python-based projects. Our experiment is based on 948 bug reports, and the results show that COPS can effectively localize buggy statements. First, compared to the original stack traces, Top@10 is improved by 102.6%, MAP@10 by 56.2%, and MRR@10 by 95.6%. We found that actual buggy code entities are more likely to appear in the first five frames of the stack trace. Second, COPS can achieve equally good localization performance compared to state-of-the-art statement-level bug localization techniques and achieve 92% buggy statement coverage with a full-scope search. Finally, experiments found that the stack trace's first two-thirds of information is more conducive to localizing buggy statements.

Abstract: Engineering high-performance query execution engines is a challenging task. Query compilation provides excellent performance, but at the same time introduces significant system complexity, as it makes the engine hard to build, debug, and maintain. To overcome this complexity, we propose Nautilus, a framework that combines the ease of use of query interpretation and the performance of query compilation. On the one hand, Nautilus provides an interpretation-based operator interface that enables engineers to implement operators using imperative C++ code to ensure a familiar developer experience. On the other hand, Nautilus mitigates the performance drawbacks of interpretation by introducing a novel trace-based, multi-backend JIT compiler that translates operators into efficient code. As a result, Nautilus bridges the gap between compilation and interpretation and provides the best of both worlds, achieving high performance without sacrificing the productivity of engineers.

Abstract: Deep Learning models have become an integrated component of modern software systems. In response to the challenge of model design, researchers proposed Automated Machine Learning (AutoML) systems, which automatically search for model architecture and hyperparameters for a given task. Like other software systems, existing AutoML systems have shortcomings in their design. We identify two common and severe shortcomings in AutoML, performance issue (i.e., searching for the desired model takes an unreasonably long time) and ineffective search issue (i.e., AutoML systems are not able to find an accurate enough model). After analyzing the workflow of AutoML, we observe that existing AutoML systems overlook potential opportunities in search space, search method, and search feedback, which results in performance and ineffective search issues. Based on our analysis, we design and implement DREAM, an automatic and general-purpose tool to alleviate and repair the shortcomings of AutoML pipelines and conduct effective model searches for diverse tasks. It monitors the process of AutoML to collect detailed feedback and automatically repairs shortcomings by expanding search space and leveraging a feedback-driven search strategy. Our evaluation results show that DREAM can be applied on two state-of-the-art AutoML pipelines and effectively and efficiently repair their shortcomings.

Abstract: ChatGPT, a recently developed product by openAI, is successfully leaving its mark as a multi-purpose natural language based chatbot. In this paper, we are more interested in analyzing its potential in the field of computational biology. A major share of work done by computational biologists these days involve coding up bioinformatics algorithms, analyzing data, creating pipelining scripts and even machine learning modeling and feature extraction. This paper focuses on the potential influence (both positive and negative) of ChatGPT in the mentioned aspects with illustrative examples from different perspectives. Compared to other fields of computer science, computational biology has (1) less coding resources, (2) more sensitivity and bias issues (deals with medical data), and (3) more necessity of coding assistance (people from diverse background come to this field). Keeping such issues in mind, we cover use cases such as code writing, reviewing, debugging, converting, refactoring, and pipelining using ChatGPT from the perspective of computational biologists in this paper.