Medical imaging used to be primarily within the domain of radiology, but with the advent of virtual pathology slides and telemedicine, imaging technology is expanding in the healthcare enterprise. As new imaging technologies are developed, they must be evaluated to assess the impact and benefit on patient care. The authors review the hierarchical model of the efficacy of diagnostic imaging systems by Fryback and Thornbury Med. Decis. Making 11, 88‐94 1991 as a guiding principle for system evaluation. Evaluation of medical imaging systems encompasses everything from the hardware and software used to acquire, store, and transmit images to the presentation of images to the interpreting clinician. Evaluation of medical imaging systems can take many forms, from the purely technical e.g., patient dose measurement to the increasingly complex e.g., determining whether a new imaging method saves lives and benefits society. Evaluation methodologies cover a broad range, from receiver operating characteristic ROC techniques that measure diagnostic accuracy to timing studies that measure image-interpretation workflow efficiency. The authors review briefly the history of the development of evaluation methodologies and review ROC methodology as well as other types of evaluation methods. They discuss unique challenges in system evaluation that face the imaging community today and opportunities for future advances. © 2008 American Association of Physicists in Medicine. DOI: 10.1118/1.2830376

Anniversary paper: evaluation of medical imaging systems.

All decisions made under conditions of uncertainty have error rates. All meaningful decisions are made under conditions of uncertainty. Can this uncertainty be measured? Can variations in how different observers deal with this uncertainty be ascertained? The ability to measure observer performance in diagnostic imaging was one of the issues that initiated the field of medical decision analysis. This article exemplifies an approach and is worth discussing as a preamble to presenting our long-term project of measuring variations in observer performance. The paper focuses on the interpretation of chest x-ray images, although the principles and findings described can be applied to nearly every radiologic modality and interpretation task.

Measuring Observer Performance in Chest Radiology: Some Experiences

Background: There is an opportunity to improve the image quality and lesion detectability in single photon emission computed tomography (SPECT) by choosing an appropriate reconstruction method and optimal parameters for the reconstruction. Purpose: To optimize the use of the Flash 3D reconstruction algorithm in terms of equivalent iteration (EI) number (number of subsets times the number of iterations) and to compare with two recently developed reconstruction algorithms ReSPECT and orthogonal polynomial expansion on disc (OPED) for application on (123)I-metaiodobenzylguanidine (MIBG)-SPECT. Material and Methods: Eleven adult patients underwent SPECT 4 h and 14 patients 24 h after injection of approximately 200 MBq (123)I-MIBG using a Siemens Symbia T6 SPECT/CT. Images were reconstructed from raw data using the Flash 3D algorithm at eight different EI numbers. The images were ranked by three experienced nuclear medicine physicians according to their overall impression of the image quality. The obtained optimal images were then compared in one further visual comparison with images reconstructed using the ReSPECT and OPED algorithms.ResultsThe optimal EI number for Flash 3D was determined to be 32 for acquisition 4 h and 24 h after injection. The average rank order (best first) for the different reconstructions for acquisition after 4 h was: Flash 3D(32) > ReSPECT > Flash 3D(64) > OPED, and after 24 h: Flash 3D(16) > ReSPECT > Flash 3D(32) > OPED. A fair level of inter-observer agreement concerning optimal EI number and reconstruction algorithm was obtained, which may be explained by the different individual preferences of what is appropriate image quality. Conclusion: Using Siemens Symbia T6 SPECT/CT and specified acquisition parameters, Flash 3D(32) (4 h) and Flash 3D(16) (24 h), followed by ReSPECT, were assessed to be the preferable reconstruction algorithms in visual assessment of (123)I-MIBG images. (Less)

/pdf/evaluation-of-image-reconstruction-methods-for-123i-mibg-1sjcaupbkw.pdf

Evaluation of image reconstruction methods for 123I-MIBG-SPECT: a rank-order study.

Statistical Test to Assess Rank-Order Imaging Studies

The significant increase in the use of CT, alone or combined with SPECT or PET, has raised concerns about patient radiation exposure and the consequent increased risk of malignancy later in life. The overall objective of this work was to evaluate and optimise different approaches for minimising patient radiation absorbed dose and maintaining or improving image quality in CT, SPECT/CT, and PET/CT.

One way to achieve optimisation is to use automatic exposure control (AEC) systems in CT. The reduction in radiation absorbed dose ranges from 35-60% for an anthropomorphic chest phantom, depending on the system and AEC settings. The variation in image noise among images obtained along the scanning direction is lower when using the AEC systems, but the image noise generally increases.

Evaluation of an abdominal CT protocol, aimed to reduce age-specific risks, demonstrated that an increased amount of intravenous contrast medium can compensate for a reduced radiation absorbed dose and vice versa, maintaining the signal-to-noise ratio in the liver and contrast-to-noise ratio for a hypothetical hypovascular liver metastasis. Subjective image quality was affected by an increased noise level in the images, but was judged to be acceptable in all investigated patient groups except the one with the lowest radiation absorbed dose. The effective dose was reduced by 57% in the youngest patient group and the amount of intravenous contrast medium was reduced by 18% in the elderly group.

Organ and effective doses to the patient using a cone-beam O-arm system in spinal surgery were estimated using the Monte Carlo technique. The highest estimated absorbed doses were in the breast and lungs when scanning the thoracic spine, and stomach when scanning the lumbar spine. The effective dose was reduced to 1.5-2.4 mSv, which is 5 times lower than the scan settings recommended by the manufacturer for intra-operative imaging of small patients, without a negative impact on image quality in regard to the information required for spinal surgery.

A new patent-filed phantom, the MADEIRA phantom, has been developed for the investigation of spatial resolution, partial volume effect (intensity diffusion), and detectability in nuclear medicine tomography. The phantom has been tested and shows potential as a useful and important practical tool in the evaluation and optimisation work. 

New image reconstruction methods are constantly being developed. Three different reconstruction methods for SPECT were optimised and compared in a visual assessment of 123I-MIBG. Based on a rank-order study, Siemens Flash 3D using 32 (4 h post-injection) and 16 (24 h post-injection) equivalent iterations were found to be the preferable reconstruction algorithms.

Image quality optimisation and dose management in ct, spect/ct, and pet/ct

Towers JD, Holbert JM, Britton CA, Costello P, Sciulli R, Gur D. Multipoint rank-order study methodology: Observer issues. Invest Radiol 2000;35:125–130.
We performed a multipoint rank-order experiment to evaluate variability in observers’ sensitivity to small differences in image presentation and to assess observers’ performance as a function of the type and number of tasks included.
Five experienced observers were presented with four sets of chest images that had been compressed at five different levels. Each set contained six images ranging from noncompressed to approximately 60:1-compressed images. Observers were asked to review all images of each case side by side and rank-order the “quality” of each to enable determination of the presence or absence of interstitial disease and/or pneumothoraces.
Observers varied significantly in their ability to detect very small differences among the images (P < 0.001). Those who performed well did so regardless of whether they ranked a specific abnormality in a multidisease or a single-disease setting.
Selected observers can reliably detect very small differences among similar images. These readers could be used to confirm or rule out the need for objective observer-performance–type studies.

Multipoint rank-order study methodology: observer issues.

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Robert Sciulli

Author Tools

Chat about Author

Papers

Multipoint rank-order study methodology: observer issues.