General Matrix Multiplication (GEMM) is a computationally expensive operation that is used in many applications such as machine-learning. Hardware accelerators are increasingly popular for speeding up GEMM computation, with Tiled Matrix Multiplication (TMUL) in recent Intel processors being an example. Unfortunately, the TMUL hardware is susceptible to errors necessitating online error detection. Algorithm-based Error Detection techniques (ABED) is a powerful technique to detect errors in matrix multiplications. In this paper, we consider implementation of ABED that integrates seamlessly with the TMUL hardware to minimize performance overhead. Unfortunately, rounding errors introduced by floating-point operations do not allow a straightforward implementation of ABED in TMUL. Previously an error bound was considered for addressing rounding errors in ABED. If the error detection threshold is set too low, it will trigger false alarm while a loose bound will allow errors to escape detection. In this paper, we propose an adaptive error threshold that takes into account the TMUL input values to address the problem of false triggers and error escapes, and provide a taxonomy of various error classes. This threshold is obtained from theoretical error analysis but is not easy to implement in hardware. Consequently, we relax the threshold such that it can be easily computed in hardware. While ABED ensures error free computation it does not guarantee full coverage of all hardware faults. To address this problem, we propose an algorithmic pattern-generation technique to ensure full coverage for all hardware faults. To evaluate the benefits of our proposed solution, we conducted fault injection experiments and show that our approach does not produce any false alarms or detection escapes for observable errors. We conducted additional fault injection experiments on a Deep Neural Network (DNN) model and find that if a fault is not detected, it does not cause any misclassification.

Highly Efficient Self-Checking Matrix Multiplication on Tiled AMX Accelerators

General Matrix Multiplications (GEMMs) are widely used in scientific and machine learning applications. Convolutional Neural Networks (CNNs) rely primarily on convolution operation, which is often expressed as a GEMM problem. To speed up GEMM operations, modern CPUs include hardware accelerators specifically designed for this purpose. One such hard-ware accelerator is the Tile Multiplication unit (TMUL), which is supported by the Advanced Matrix Multiplication (AMX®) instruction set in the Intel Sapphire Rapids microarchitecture. However, like other frequently used hardware components, the TMUL is susceptible to permanent or persistent faults due to aging, which can lead to Silent Data Corruption (SDC). Google and Meta have recently reported recurring SDCs. Algorithm-based error detection (ABED) is a highly effective method for detecting persistent faults in GEMM operations. Our previous work involved a novel hardware implementation of ABED in TMUL, which could locate faults but only identify the of Fending matrix column, not the exact cell. In this paper, we present a self-repair technique for persistent faults in GEMM based on the aforementioned ABED approach. Our solution hinges on software-based column avoidance. When the hardware-based ABED detects an error caused by a faulty column in the TMUL, our software redirects computations to circumvent the faulty column. This eliminates the need for additional hardware modifications to ABED. We demonstrate that our proposed approach can perform self-repair for single faults for all matrix sizes, from those smaller than the TMUL dimension to larger matrices. Furthermore, we show that our solution can even correct multiple faults with low-performance overhead, and we report the performance overhead for multiple fault repair using a clustered fault model. Unlike prior solutions, our approach performs online detection and repair, enabling real-time correction of both persistent and intermittent faults.

A Novel Self-Repair Mechanism for Tiled Matrix Multiplication Unit

General Matrix-Matrix Multiplication (GEMM) is a commonly used kernel in machine learning, scientific computing and many other applications. Designing a customized GEMM accelerator can bring obvious performance and power consumption benefits. In this paper, we first perform a detailed workload characterization for different sizes of GEMM kernels. Then, we make a comprehensive design space exploration to find the Pareto optimal architecture configurations. Lastly, we compare two versions of multiple GEMM accelerator systems with main-stream BLAS libraries (e.g., OpenBLAS, MKL and cuBLAS). The proposed GEMM acceleration hardware system shows higher energy efficiency than existing software implementations.

Exploring the Architecture of Multiple GEMM Accelerators in Heterogeneous Systems

Editor’s notes: Hardware realization of artificial intelligence (AI) requires new design styles and even underlying technologies than those used in traditional digital processors or logic circuits. Therefore, their dependability and testability requirements, challenges, and solutions are fundamentally different and unique. This survey article covers this very important topic and provides insight to such issues.—Mehdi Tahoori, Karlsruhe Institute of Technology

Testability and Dependability of AI Hardware: Survey, Trends, Challenges, and Perspectives

General Matrix Multiplication (GEMM) is instrumental in myriads of scientific, high-performance computing, and machine learning applications such as computer vision, recommendation models, and weather forecasts. It is vital to make them fail-safe in safety-critical and high-precision applications. Companies like Meta and Google have recently reported sporadic silent errors in GEMM computations traced to hardware sources. Silent errors are hard to detect, requiring specialized solutions to detect them. Hardware redundancy approaches such as double or triple modular redundancy effectively detect or correct such errors, but they have a large area and power overhead. Algorithm-based Fault Tolerance (ABFT) has been shown to offer an effective alternative at a far lower overhead. Modern CPUs feature advanced vector extensions (AVX) capable of executing SIMD instructions. This paper describes a new ABFT approach designed to take advantage of the AVX feature. Our core algorithm relies on the classical tile-based outer-product approach but enhances standard check-sum calculation using a tile vector. The implementation parameters are fine-tuned to fit the available number of AVX registers. Our results indicate that we can achieve 100% error detection in GEMM at an overhead of just 0.21% for the integer data type. Unfortunately, due to rounding errors, addition of floating-point numbers is not an associative operation, creating difficulties for ABFT. To mitigate the impact of rounding errors, we introduce the concept of relative error checking and perform error analysis for various error classes to show that the proposed approach totally eliminates false positive errors.

A Highly-Efficient Error Detection Technique for General Matrix Multiplication using Tiled Processing on SIMD Architecture

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Brunno F. Goldstein

Author Tools

Chat about Author

Papers

A Highly-Efficient Error Detection Technique for General Matrix Multiplication using Tiled Processing on SIMD Architecture