Automatic vision-based inspection systems have played a key role in product quality assessment for decades through the segmentation, detection, and classification of defects. Historically, machine learning frameworks, based on hand-crafted feature extraction, selection, and validation, counted on a combined approach of parameterized image processing algorithms and explicated human knowledge. The outstanding performance of deep learning (DL) for vision systems, in automatically discovering a feature representation suitable for the corresponding task, has exponentially increased the number of scientific articles and commercial products aiming at industrial quality assessment. In such a context, this article reviews more than 220 relevant articles from the related literature published until February 2023, covering the recent consolidation and advances in the field of fully-automatic DL-based surface defects inspection systems, deployed in various industrial applications. The analyzed papers have been classified according to a bi-dimensional taxonomy, that considers both the specific defect recognition task and the employed learning paradigm. The dependency on large and high-quality labeled datasets and the different neural architectures employed to achieve an overall perception of both well-visible and subtle defects, through the supervision of fine or/and coarse data annotations have been assessed. The results of our analysis highlight a growing research interest in defect representation power enrichment, especially by transferring pre-trained layers to an optimized network and by explaining the network decisions to suggest trustworthy retention or rejection of the products being evaluated. 

/pdf/deep-learning-for-automatic-vision-based-recognition-of-3rwu0otj.pdf

Deep Learning for Automatic Vision-Based Recognition of Industrial Surface Defects: A Survey

To improve the precision of defect categorization and localization in images, this paper proposes an approach for detecting surface defects in hot-rolled steel strips. The approach uses an improved YOLOv5 network model to overcome the issues of inadequate feature extraction capacity and suboptimal feature integration when identifying surface defects on steel strips. The proposed method achieves higher detection accuracy and localization precision, making it more competitive and applicable in real production. Firstly, the multi-scale feature fusion (MSF) strategy is utilized to fuse shallow and deep features effectively and enrich detailed information relevant to target defects. Secondly, the CSPLayer Res2Attention block (CRA block) residual module is introduced to reduce the loss of defect information during hierarchical transmission, thereby enhancing the extraction of fine-grained features and improving the perception of details and global features. Finally, the experimental results indicate that the mAP on the NEU-DET and GC10-DET datasets approaches 78.5% and 67.3%, respectively, which is 4.9% and 2.1% higher than that of the baseline. Meanwhile, it has higher precision and more precise localization capabilities than other methods. Furthermore, it also achieves 59.2% mAP on the APDDD dataset, indicating its potential for growth in further domains. 

Surface defect detection of hot rolled steel based on multi-scale feature fusion and attention mechanism residual block

In recent years, visual automatic non-destructive testing using machine vision algorithms has been widely used in industry. This approach for detecting, classifying, and segmenting defects in building materials and structures can be effectively implemented using convolutional neural networks. Using intelligent systems in the initial stages of manufacturing can eliminate defective building materials, prevent the spread of defective products, and detect the cause of specific damage. In this article, the solution to the problem of building elements flaw detection using the computer vision method was considered. Using the YOLOv5s convolutional neural network for the detection and classification of various defects of the structure, the appearance of finished products of facing bricks that take place at the production stage is shown during technological processing, packaging, transportation, or storage. The algorithm allows for the detection of foreign inclusions, broken corners, cracks, and color unevenness, including the presence of rust spots. To train the detector, our own empirical database of images of facing brick samples was obtained. The set of training data for the neural network algorithm for discovering defects and classifying images was expanded by using our own augmentation algorithm. The results show that the developed YOLOv5s model has a high accuracy in solving the problems of defect detection: mAP0.50 = 87% and mAP0.50:0.95 = 72%. It should be noted that the use of synthetic data obtained by augmentation makes it possible to achieve a good generalizing ability from the algorithm, it has the potential to expand visual variability and practical applicability in various shooting conditions.

https://www.mdpi.com/2076-3417/13/9/5413/pdf?version=1682521329

Discovery and Classification of Defects on Facing Brick Specimens Using a Convolutional Neural Network

Time of flight (ToF) light detection and ranging (Lidar) range finder is widely used in various applications, such as automotive driving, marine environment surveying, atmospheric observing, and physical distancing monitoring. The ToF Lidar offers a high-resolution range, which can be achieved by increasing the modulation frequency. However, when the modulation frequency is increased, the maximum unambiguous range tends to decrease. To overcome such a problem, we develop a novel ToF Lidar range finder based on dual-modulation frequencies switching, capable of providing an extended range while maintaining the high resolution. In the design, the modulation frequency is alternately changed between 2 and 10 MHz to modulate the intensity of the laser diode as a transmitter. We operate an avalanche photodetector as a receiver to obtain a reflection of modulated light from a target. By switching between dual-modulation frequencies, the photodetector detects two-phase shifts, which are calculated by our phase shift algorithm. It is shown that the proposed algorithm can solve the ambiguous range problem. As proof of the concept, we experimentally demonstrate a range measurement of a target up to 55-m distance, where the ambiguous range is experienced four times if only a single modulation frequency of 10 MHz is used. Interestingly, the range ambiguity no longer appears after applying the proposed design and algorithm. 

Time of Flight Lidar Employing Dual-Modulation Frequencies Switching for Optimizing Unambiguous Range Extension and High Resolution

Photovoltaic (PV) wafers are a basic material for solar cells, and surface defect detection for solar PV wafers is necessary to improve the quality of solar PV. However, the complexity of the wafer manufacturing environment causes the surface defects of PV wafers to appear diverse, and it is still challenging in practical applications. In this research, a deep-learning (DL) network based on global information branches and dual-branch feature fusion (DBFF) is proposed for surface defect detection (SDD), named GFYolov7. First, for the issue of the low contrast on the wafer surface samples, the global information branching (GIB) is generated using the multiscale features of the backbone, which improves the information utilization of the backbone. Second, a DBFF module is designed for the GIB module, and a lightweight attention mechanism is used for the fused multiscale features, which serve to highlight defective regions and suppress irrelevant regions. The experimental results of GFYolov7 on NEU-DET and the GC10-DET reach 79.5 and 72.1 mAP and that on the real-world PV wafer dataset is 72.7 mAP. By comparing with other DL methods, GFYolov7 has been demonstrated that it has good detection ability and its detection speed can reach 61 FPS.

A New Dual-Branch Network With Global Information for the Surface Defect Detection on Solar PV Wafer

3D time-of-flight (ToF) cameras have recently attracted widespread attention because of their potential in wide applications. However, the lack of versatile 3D image signal processor (ISP) is still an obstacle that hinders their popularity. The main challenges lie in the management of high-throughput raw data and the complexity of 3D image processing in hardware. To address these challenges, this paper presents a top-down 3D ISP with high real-time performance and high energy efficiency for indirect ToF (IToF) imaging. First, we propose configurable cache strategies to improve the efficiency of non-streaming memory accesses and support different IToF sensors. Then, to resolve the phase ambiguity problem commonly existing in IToF sensors, we develop a hardware-optimized cascaded multi-frequency fusion method, which not only supports up to six modulation frequencies but also facilitates the pixel-wise detection of multipath interference. Moreover, we propose a scalable calculation engine for depth calculation and calibration, 3D image enhancement, and post-processing. The pipeline is prototyped on a Xilinx ZCU102 FPGA board. Results show that the 3D ISP has negligible precision loss and low latency, achieving <inline-formula> <tex-math notation="LaTeX">$53.35\times $ </tex-math></inline-formula> energy efficiency improvement compared with the implementation on Intel CPU and <inline-formula> <tex-math notation="LaTeX">$55.45\times $ </tex-math></inline-formula> improvement over the ARM CPU implementation. When running at 75 MHz, the 3D ISP can support <inline-formula> <tex-math notation="LaTeX">$1280\times 960$ </tex-math></inline-formula> depth, intensity, and point cloud output at a frame rate of 30 f/s.

A Configurable and Real-Time Multi-Frequency 3D Image Signal Processor for Indirect Time-of-Flight Sensors

Metallic surface defect detection is critical to ensure the quality of industrial products. Recently, human-advanced surface defect detection algorithms have been proposed. Most of these algorithms rely on convolutional neural networks (CNN) and an anchoring scheme. However, a convolution unit only samples the input feature maps at fixed shapes and locations. Similarly, a set of anchors are uniformly predefined with fixed scales and shapes, which increases the difficulties of bounding box regression. Therefore, we propose an adaptive convolution and anchor network for metallic surface defect detection, named ACA-Net. Specifically, an adaptive convolution and anchor (ACA) module is proposed, which mainly consists of adaptive convolution and an adaptive anchor. Firstly, an adaptive convolution module (ACM) is designed, which adaptively determines the location and shape of each convolution unit. In addition, a multi-scale feature adaptive fusion (MFAF) is proposed, which is used in ACM to extract and integrate multi-scale features. Then, an adaptive anchor module (AAM) is proposed to yield more suitable anchor boxes by adaptively adjusting shapes. Extensive experiments on NEU-DET dataset and GC10 dataset validate the performance of the proposed approach. ACA-Net achieves 1.8% on NEU-DET dataset higher Average Precision (AP) than GA-RetinaNet. Furthermore, the proposed ACA module is also adopted in GA-Faster R-CNN, improving the AP by 1.2% on NEU-DET dataset.

/pdf/aca-net-an-adaptive-convolution-and-anchor-network-for-2rtt5in3.pdf

ACA-Net: An Adaptive Convolution and Anchor Network for Metallic Surface Defect Detection

Spiking neural networks (SNNs) have attracted considerable attention as third-generation artificial neural networks, known for their powerful, intelligent features and energy-efficiency advantages. These characteristics render them ideally suited for edge computing scenarios. Nevertheless, the current mapping schemes for deploying SNNs onto neuromorphic hardware face limitations such as extended execution times, low throughput, and insufficient consideration of energy consumption and connectivity, which undermine their suitability for edge computing applications. To address these challenges, we introduce EdgeMap, an optimized mapping toolchain specifically designed for deploying SNNs onto edge devices without compromising performance. EdgeMap consists of two main stages. The first stage involves partitioning the SNN graph into small neuron clusters based on the streaming graph partition algorithm, with the sizes of neuron clusters limited by the physical neuron cores. In the subsequent mapping stage, we adopt a multi-objective optimization algorithm specifically geared towards mitigating energy costs and communication costs for efficient deployment. EdgeMap—evaluated across four typical SNN applications—substantially outperforms other state-of-the-art mapping schemes. The performance improvements include a reduction in average latency by up to 19.8%, energy consumption by 57%, and communication cost by 58%. Moreover, EdgeMap exhibits an impressive enhancement in execution time by a factor of 1225.44×, alongside a throughput increase of up to 4.02×. These results highlight EdgeMap’s efficiency and effectiveness, emphasizing its utility for deploying SNN applications in edge computing scenarios.

EdgeMap: An Optimized Mapping Toolchain for Spiking Neural Network in Edge Computing

As Light Detection And Ranging (LiDAR) increasingly becomes an essential component in robotic navigation and autonomous driving, the processing of high throughput 3D point clouds in real time is widely required. This work considers the point cloud k-Nearest Neighbor (kNN) search, which is an important 3D processing kernel. Although applying fine-grained parallelism optimization on internal processing, e.g., using multiple workers, has demonstrated high efficiency, previous accelerators with DDR external memory are fundamentally limited by the external bandwidth bottleneck. To break this bottleneck, this work proposes a highly parallel architecture, namely ParallelNN, for highly efficient kNN search processing of high throughput point clouds. First, we optimize the multichannel cache based on High Bandwidth Memory (HBM) and on-chip memory to provide large external bandwidth. Then, a novel parallel depth-first octree construction algorithm is proposed and mapped onto multiple construction branches with trace-coded construction queues, which can regularize random accesses and perform multi-branch octree construction efficiently. Furthermore, in the search stage, we present algorithm-architecture co-optimization, including parallel keyframe-based scheduling and multi-branch flexible search engines, to provide conflict-free access and maximum reuse opportunities for reference points, which achieves more than 27.0× speedup compared with baseline architectures. We prototype ParallelNN on Virtex HBM FPGA and perform extensive benchmarking on the KITTI dataset. The results demonstrate that ParallelNN achieves up to 107.7× and 12.1× speedup over CPU and GPU implementations, while being more energy efficient, e.g., outperforming CPU and GPU implementations by 73.6× and 31.1×, respectively. Besides, with the proposed algorithm-architecture co-optimization, ParallelNN achieves 11.4× speedup over state-of-the-art architecture. Moreover, ParallelNN is configurable and can be easily generalized to similar octree-based applications.

ParallelNN: A Parallel Octree-based Nearest Neighbor Search Accelerator for 3D Point Clouds

Multi-core neuromorphic hardware for spiking neural networks (SNNs) has garnered considerable attention due to its biological plausibility and energy efficiency. However, the performance of SNN applications on such hardware is constrained by the rigid architecture and interconnection among neuron cores. To enable early-stage evaluation of SNN performance on multi-core neuromorphic hardware, we introduce an accurate and scalable simulator, SpikeNC. We present the entire workflow, ranging from SNN model training to simulation, providing comprehensive insights into both model and Network-on-Chip (NoC) related statistics. Moreover, we identify a considerable amount of time wastage in the widely adopted tick-based synchronous scheme. A three-stage agent-based asynchronous scheme is proposed for fast simulation. We evaluate the performance of deep spiking neural networks (DSNNs) with various scales trained on spike-converted datasets using SpikeNC. The results demonstrate that SpikeNC achieves precise and scalable simulation for SNNs on multi-core neuromorphic hardware. Additionally, the proposed asynchronous scheme significantly reduces the simulation cycles and absolute simulation time by approximately 63 % and 56 % respectively, compared to the synchronous scheme. We also delve into the trade-offs between different design parameters and explore the influence of mapping schemes utilizing SpikeNC.

SpikeNC: An Accurate and Scalable Simulator for Spiking Neural Network on Multi-Core Neuromorphic Hardware

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Faquan Chen

Author Tools

Chat about Author