This paper presents a systematic literature review of the recent (2006--2010) development of automatic assessment tools for programming exercises. We discuss the major features that the tools support and the different approaches they are using both from the pedagogical and the technical point of view. Examples of these features are ways for the teacher to define tests, resubmission policies, security issues, and so forth. We have also identified a list of novel features, like assessing web software, that are likely to get more research attention in the future. As a conclusion, we state that too many new systems are developed, but also acknowledge the current reasons for the phenomenon. As one solution we encourage opening up the existing systems and joining efforts on developing those further. Selected systems from our survey are briefly described in Appendix A.

Review of recent systems for automatic assessment of programming assignments

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research.

/pdf/introductory-programming-a-systematic-literature-review-3yllvt7k18.pdf

Introductory programming: a systematic literature review

This Handbook describes the extent and shape of computing education research today. Over fifty leading researchers from academia and industry (including Google and Microsoft) have contributed chapters that together define and expand the evidence base. The foundational chapters set the field in context, articulate expertise from key disciplines, and form a practical guide for new researchers. They address what can be learned empirically, methodologically and theoretically from each area. The topic chapters explore issues that are of current interest, why they matter, and what is already known. They include discussion of motivational context, implications for practice, and open questions which might suggest future research. The authors provide an authoritative introduction to the field and is essential reading for policy makers, as well as both new and established researchers.

/pdf/the-cambridge-handbook-of-computing-education-research-4648mu3xgq.pdf

The Cambridge Handbook of Computing Education Research

Previous research in computer science education has demonstrated the importance of motivation for success in introductory programming. Theoretical constructs from self-regulated learning theory (SRL), which integrates several different types of metacognitive processes, as well as motivational constructs, have proved to be important predictors of success in most academic disciplines. These individual components of self-regulated learning (e.g., self-efficacy, metacognitive strategies) interact in complex ways to influence students' affective states and behaviors, which in turn influence learning outcomes. These elements have been previously examined individually in novice programmers, but we do not have a comprehensive understanding of how SRL constructs interact to influence learning to program. This paper reports on a study that examined the interaction of self-efficacy, intrinsic and extrinsic goal orientations, and metacognitive strategies and their impact on student performance in a CS1 course. We also report on significant gender differences in the relationships between SRL constructs and learning outcomes. We found that student performance had the expected motivational and SRL precursors, but the interactions between these constructs revealed some unexpected relationships. Furthermore, we found that females' self-efficacy had a different connection to programming performance than that of their male peers. Further research on success in introductory programming should take account of the unique and complex relationship between SRL and student success, as well as gender differences in these relationships that are specific to CS.

Learning to Program: Gender Differences and Interactive Effects of Students' Motivation, Goals, and Self-Efficacy on Performance

Traditional quantitative research methods of data collection in programming, such as questionnaires and interviews, are the most common approaches for researchers in this field. However, in recent years, eye-tracking has been on the rise as a new method of collecting evidence of visual attention and the cognitive process of programmers. Eye-tracking has been used by researchers in the field of programming to analyze and understand a variety of tasks such as comprehension and debugging. In this article, we will focus on reporting how experiments that used eye-trackers in programming research are conducted, and the information that can be collected from these experiments. In this mapping study, we identify and report on 63 studies, published between 1990 and June 2017, collected and gathered via manual search on digital libraries and databases related to computer science and computer engineering. Among the five main areas of research interest are program comprehension and debugging, which received an increased interest in recent years, non-code comprehension, collaborative programming, and requirements traceability research, which had the fewest number of publications due to possible limitations of the eye-tracking technology in this type of experiments. We find that most of the participants in these studies were students and faculty members from institutions of higher learning, and while they performed programming tasks on a range of programming languages and programming representations, we find Java language and Unified Modeling Language (UML) representation to be the most used materials. We also report on a range of eye-trackers and attention tracking tools that have been utilized, and find Tobii eye-trackers to be the most used devices by researchers.

/pdf/a-survey-on-the-usage-of-eye-tracking-in-computer-1ic4uu04yf.pdf

A Survey on the Usage of Eye-Tracking in Computer Programming

This paper presents an analysis of research papers about programming education that were published in computing education conferences in the years 2005 to 2008. We employed Simon's classification scheme to identify the papers of interest from the ICER, SIGCSE, ITiCSE, ACE, Koli Calling and NACCQ conferences. Having identified the papers, we analyzed the type of data collected, whether the analysis was qualitative, quantitative, or mixed, and the aims and outcomes being reported. The greatest number of papers employed quantitative research methods, investigated the ability, aptitude, or understanding of students, and were based in single courses. The theme of the research and the type of study conducted vary across the conferences, indicating the different nature and role of each conference. Papers that investigated student learning of programming in terms of established theories or models of learning were not common, indicating an area of research that deserves more attention.

Analysis of research into the teaching and learning of programming

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

S. Simon

Author Tools

Chat about Author

Papers

Analysis of research into the teaching and learning of programming