This paper presents a systematic literature review of the recent (2006--2010) development of automatic assessment tools for programming exercises. We discuss the major features that the tools support and the different approaches they are using both from the pedagogical and the technical point of view. Examples of these features are ways for the teacher to define tests, resubmission policies, security issues, and so forth. We have also identified a list of novel features, like assessing web software, that are likely to get more research attention in the future. As a conclusion, we state that too many new systems are developed, but also acknowledge the current reasons for the phenomenon. As one solution we encourage opening up the existing systems and joining efforts on developing those further. Selected systems from our survey are briefly described in Appendix A.

Review of recent systems for automatic assessment of programming assignments

OmniPHR: A distributed architecture model to integrate personal health records.

Using HL7 FHIR to achieve interoperability in patient health record.

Practical programming competencies are critical to the success in computer science (CS) education and go-to-market of fresh graduates. Acquiring the required level of skills is a long journey of discovery, trial and error, and optimization seeking through a broad range of programming activities that learners must perform themselves. It is not reasonable to consider that teachers could evaluate all attempts that the average learner should develop multiplied by the number of students enrolled in a course, much less in a timely, deep, and fair fashion. Unsurprisingly, exploring the formal structure of programs to automate the assessment of certain features has long been a hot topic among CS education practitioners. Assessing a program is considerably more complex than asserting its functional correctness, as the proliferation of tools and techniques in the literature over the past decades indicates. Program efficiency, behavior, and readability, among many other features, assessed either statically or dynamically, are now also relevant for automatic evaluation. The outcome of an evaluation evolved from the primordial Boolean values to information about errors and tips on how to advance, possibly taking into account similar solutions. This work surveys the state of the art in the automated assessment of CS assignments, focusing on the supported types of exercises, security measures adopted, testing techniques used, type of feedback produced, and the information they offer the teacher to understand and optimize learning. A new era of automated assessment, capitalizing on static analysis techniques and containerization, has been identified. Furthermore, this review presents several other findings from the conducted review, discusses the current challenges of the field, and proposes some future research directions.

Automated Assessment in Computer Science Education: A State-of-the-Art Review

2012 [AS13b, Wal12a]. 2013 [Alv13, Bar13, Cam13, Car13, CCS13, Cut13, Edi14, Swe13]. 2014 [CDS13, DN14, ND14, Rod14, Tym14a]. 2015 [DeL14a, DeL14b, She15b]. 2016 [AT16a, CCV16, CCV17, MR15, SDCT16a, SDCT16c, She16a]. 2017 [DG17, EC17, Fit18b, MJ18, TCSM17, TCM17]. 2018 [BG18b, DD17, Dec19b, Gol18, JI18, MKMP18, MMK18, PQH17, She18]. 2019 [Cut19, FI19, HPQ19, MP19a, MPRM19, MP19b, MS19, Rod18, She19a]. 2020 [DH19].

/pdf/a-complete-bibliography-of-publications-in-sigcse-bulletin-1aolnsl5m4.pdf

A Complete Bibliography of Publications in SIGCSE Bulletin: 2000{2009

Refined pivot selection for maximal clique enumeration in graphs

The complexity and resource requirements of professional IDEs mean that they are unsuitable for use in intermediate level programming courses. Jenuity is an efficient development environment for the Java programming language. Efficiency is essential as students often have outdated hardware unable to run mainstream development environments. This is of particular relevance in the context of a developing country. Jenuity provides advanced features usually associated with more resource intensive tools. It provides a simple and intuitive interface, which is well suited to intermediate level programming courses. Jenuity has been used successfully in the teaching of these courses at the authors' institution since 2004. The requirements, development and optimisation of this tool are discussed. Techniques used to optimise Jenuity for low specification student hardware, some of which are novel, are presented. Experiences using Jenuity in a university environment are also reported. The efficiency of Jenuity is also demonstrated by means of a comparison to mainstream development environments.

Jenuity: a lightweight development environment for intermediate level programming courses

Learning to write software requires much practice and frequent assessment. Consequently, the use of computers to assist in the assessment of computer programs has been important in supporting large classes at universities. The main approaches to the problem are dynamic analysis (testing student programs for expected output) and static analysis (direct analysis of the program code). The former is very sensitive to all kinds of errors in student programs, while the latter has traditionally only been used to assess quality, and not correctness. This research focusses on the application of static analysis, particularly structural similarity, to marking student programs. Existing traditional measures of similarity are limiting in that they are usually only effective on tree structures. In this regard they do not easily support dependencies in program code. Contemporary measures of structural similarity, such as similarity flooding, usually rely on an internal normalisation of scores. The effect is that the scores only have relative meaning, and cannot be interpreted in isolation, ie. they are not meaningful for assessment. The SimRank measure is shown to have the same problem, but not because of normalisation. The problem with the SimRank measure arises from the fact that its scores depend on all possible mappings between the children of vertices being compared. The main contribution of this research is a novel graph similarity measure, the Weighted Assignment Similarity measure. It is related to SimRank, but derives propagation scores from only the locally optimal mapping between child vertices. The resulting similarity scores may be regarded as the percentage of mutual coverage between graphs. The measure is proven to converge for all directed acyclic graphs, and an efficient implementation is outlined for this case. Attributes on graph vertices and edges are often used to capture domain specific information which is not structural in nature. It has been suggested that these should influence the similarity propagation, but no clear method for doing this has been reported. The second important contribution of this research is a general method for incorporating these local attribute similarities into the larger similarity propagation method. An example of attributes in program graphs are identifier names. The choice of identifiers in programs is arbitrary as they are purely symbolic. A problem facing any comparison between programs is that they are unlikely to use the same set of identifiers. This problem indicates that a mapping between the identifier sets is required. The third contribution of this research is a method for applying the structural similarity measure in a two step process to find an optimal identifier mapping. This approach is both novel and valuable as it cleverly reuses the similarity measure as an existing resource. In general, programming assignments allow a large variety of solutions. Assessing student programs through structural similarity is only feasible if the diversity in the solution space can…

Assessing program code through static structural similarity

This paper gives the results of an investigation on the use of exhaustive block matching techniques in detecting railway-line tracks in a video sequence with the view of implementing the outcome in a real-time system. The main focus of the research is to implement the exhaustive search block matching algorithm optimally and resolve any implementation problems that may arise. The algorithm is implemented on a video sequence that has undergone inverse perspective mapping. The second highlight of this research is to eliminate many video processing techniques, this is done in order to compensate for the computational expense for which the exhaustive search block matching algorithm is known for. The procedure entails traversing a very small block from the previous frame on the top and bottom ends of the current frame and marking the coordinates where the cost function is the minimum.

The effective use of the exhaustive search block matching algorithm in railway line tracking

Portable devices are ubiquitous and users are expecting more sophisticated and natural interactions, where the interface mimics and extends real world objects, such as paper. Visual communication languages are used on these media to convey a wealth of information, which can be analysed and automatically extracted. This research investigates how a digital alternative to a static medium can seamlessly and easily provide electronic support. The case study of the research, the pre-production phase of computer generated film making, was chosen as it has a comprehensive visual language on which to draw. A framework for sketch-based interaction is proposed, and a prototype system is evaluated on a number of different criteria. The main findings of the evaluation were that users found the sketch-based interaction intuitive, easy to learn and preferred it to standard interaction techniques. The main limiting factors of a sketch-based interaction technique were identified as adequately dealing with recognition errors and managing algorithm execution time so that the system remained interactive. A set of guidelines is provided to help researchers and developers in creating tools using similar techniques.

Sketch-based interfaces: drawings to data

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Kevin A. Naudé

Author Tools

Chat about Author