Vision transformers (ViTs) have been trending in image classification tasks due to their promising performance when compared to convolutional neural networks (CNNs). As a result, many researchers have tried to incorporate ViTs in hyperspectral image (HSI) classification tasks. To achieve satisfactory performance, close to that of CNNs, transformers need fewer parameters. ViTs and other similar transformers use an external classification (CLS) token which is randomly initialized and often fails to generalize well, whereas other sources of multimodal datasets, such as light detection and ranging (LiDAR) offer the potential to improve these models by means of a CLS. In this paper, we introduce a new multimodal fusion transformer (MFT) network which comprises a multihead cross patch attention (mCrossPA) for HSI land-cover classification. Our mCrossPA utilizes other sources of complementary information in addition to the HSI in the transformer encoder to achieve better generalization. The concept of tokenization is used to generate CLS and HSI patch tokens, helping to learn a distinctive representation in a reduced and hierarchical feature space. Extensive experiments are carried out on widely used benchmark datasets i.e., the University of Houston, Trento, University of Southern Mississippi Gulfpark (MUUFL), and Augsburg. We compare the results of the proposed MFT model with other state-of-the-art transformers, classical CNNs, and conventional classifiers models. The superior performance achieved by the proposed model is due to the use of multihead cross patch attention. The source code will be made available publicly at https://github.com/AnkurDeria/MFT. 

/pdf/multimodal-fusion-transformer-for-remote-sensing-image-9of61lgg.pdf

Multimodal Fusion Transformer for Remote Sensing Image Classification

Vision transformers (ViTs) have been trending in image classification tasks due to their promising performance when compared with convolutional neural networks (CNNs). As a result, many researchers have tried to incorporate ViTs in hyperspectral image (HSI) classification tasks. To achieve satisfactory performance, close to that of CNNs, transformers need fewer parameters. ViTs and other similar transformers use an external classification (CLS) token, which is randomly initialized and often fails to generalize well, whereas other sources of multimodal datasets, such as light detection and ranging (LiDAR), offer the potential to improve these models by means of a CLS. In this article, we introduce a new multimodal fusion transformer (MFT) network, which comprises a multihead cross-patch attention (mCrossPA) for HSI land-cover classification. Our mCrossPA utilizes other sources of complementary information in addition to the HSI in the transformer encoder to achieve better generalization. The concept of tokenization is used to generate CLS and HSI patch tokens, helping to learn a distinctive representation in a reduced and hierarchical feature space. Extensive experiments are carried out on widely used benchmark datasets, i.e., the University of Houston (UH), Trento, University of Southern Mississippi Gulfpark (MUUFL), and Augsburg. We compare the results of the proposed MFT model with other state-of-the-art transformers, classical CNNs, and conventional classifiers models. The superior performance achieved by the proposed model is due to the use of mCrossPA. The source code will be made available publicly at https://github.com/AnkurDeria/MFT.

https://export.arxiv.org/pdf/2203.16952v2.pdf

Microorganisms such as protozoa and bacteria play very important roles in many practical domains, like agriculture, industry and medicine. To explore functions of different categories of microorganisms is a fundamental work in biological studies, which can assist biologists and related scientists to get to know more properties, habits and characteristics of these tiny but obbligato living beings. However, taxonomy of microorganisms (microorganism classification) is traditionally investigated through morphological, chemical or physical analysis, which is time and money consuming. In order to overcome this, since the 1970s innovative content-based microscopic image analysis (CBMIA) approaches are introduced to microbiological fields. CBMIA methods classify microorganisms into different categories using multiple artificial intelligence approaches, such as machine vision, pattern recognition and machine learning algorithms. Furthermore, because CBMIA approaches are semi- or full-automatic computer-based methods, they are very efficient and labour cost saving, supporting a technical feasibility for microorganism classification in our current big data age. In this article, we review the development history of microorganism classification using CBMIA approaches with two crossed pipelines. In the first pipeline, all related works are grouped by their corresponding microorganism application domains. By this pipeline, it is easy for microbiologists to have an insight into each special application domain and find their interested applied CBMIA techniques. In the second pipeline, the related works in each application domain are reviewed by time periods. Using this pipeline, computer scientists can see the dynamic of technological development clearly and keep up with the future development trend in this interdisciplinary field. In addition, the frequently-used CBMIA methods are further analysed to find technological common points and potential reasons.

A survey for the applications of content-based microscopic image analysis in microorganism classification domains

Semantic segmentation of remote sensing images plays a crucial role in a wide variety of practical applications, including land cover mapping, environmental protection, and economic assessment. In the last decade, convolutional neural network (CNN) is the mainstream deep learning-based method of semantic segmentation. Compared with conventional methods, CNN-based methods learn semantic features automatically, thereby achieving strong representation capability. However, the local receptive field of the convolution operation limits CNN-based methods from capturing long-range dependencies. In contrast, Vision Transformer (ViT) demonstrates its great potential in modeling long-range dependencies and obtains superior results in semantic segmentation. Inspired by this, in this letter, we propose a class-guided Swin Transformer (CG-Swin) for semantic segmentation of remote sensing images. Specifically, we adopt a Transformer-based encoder–decoder structure, which introduces the Swin Transformer backbone as the encoder and designs a class-guided Transformer block to construct the decoder. The experimental results on ISPRS Vaihingen and Potsdam datasets demonstrate the significant breakthrough of the proposed method over ten benchmarks, outperforming both advanced CNN-based and recent Transformer-based approaches. 

https://eprints.lancs.ac.uk/id/eprint/177531/1/GRSL_01132_2022.pdf

Class-Guided Swin Transformer for Semantic Segmentation of Remote Sensing Imagery

Pavement crack detection with hybrid-window attentive vision transformers

RoadFormer: Pyramidal deformable vision transformers for road network extraction with remote sensing images

CapViT: Cross-context capsule vision transformers for land cover classification with airborne multispectral LiDAR data

Traffic signs are indispensable fixtures in modern transportation activities, which are installed along the roadside or over the pathway to provide important prompt messages. The timely emergences, complete visibilities, and definite signals of traffic signs matter significantly to direct the driving behaviors and ensure the convenience and security of the transportation activities. Moreover, the efficient and accurate recognition of traffic signs can also provide decisive evidences to a variety of intelligent transportation systems. Nevertheless, the unignorable presence issues of traffic signs caused by vision factors, such as small sizes, dimness, deformations, and occlusions, impact essentially on the high-quality recognition of traffic signs. This paper develops a novel attentive semi-anchoring guided high-resolution network, named SignHRNet, for street-level traffic signs recognition purpose. First, stacked with a high-resolution network boosted by a dual-attention module as the feature extractor, the SignHRNet can exploit informative channel features and task-oriented spatial features to generate multiscale strong feature semantics for instance-level predictions. Second, designed with a semi-anchoring guided detection strategy assisted by instance-aware feature alignment, the SignHRNet can achieve highly-efficient sign type categorization and highly-accurate sign location determination. The proposed SignHRNet is intensively evaluated on three large-size datasets towards traffic signs recognition. Quantitative verifications demonstrate a competitive accuracy with an average mAP, mAP50, and mAP75 of 72.85%, 96.48%, and 85.31%, respectively, in processing traffic signs of varying self-conditions under diverse scenarios. Ablative and comparative analyses also confirmed the practical reliability and performance superiority of the SignHRNet in traffic signs recognition applications. 

SignHRNet: Street-level traffic signs recognition with an attentive semi-anchoring guided high-resolution network

Ausgehend von den Facettenaugen der Insekten haben Wissenschaftler seit 10 Jahren viele kunstliche Facettenaugensysteme erstellt, die auf der Multi-Apertur-Optik basieren Im Vergleich zu den auf Single-Apertur-Optik basierenden Systemen sind diese Systeme kleiner und leichter Auserdem haben solche Systeme ein groses Sichtfeld und eine hohe Empfindlichkeit Das eCley (Electronic cluster eye) ist ein neues kunstliches Facettenaugensystem, das Bilder mit Super-Pixel-Auflosung erstellen kann, welches vom Sehsystem der parasitaren Wespe „Xenos Peckii“ inspiriert ist Wegen seiner ausgezeichneten Fahigkeiten sind eCley-Systeme in den Bereichen arztliche Untersuchung, Identitatsauthentifizierung, Roboternavigation und Flugkorperlenkung angewendet worden Aber solche Anwendungen basieren nur auf der Datenverarbeitung im 2D-Bereich Wenn jedoch mit einem eCley-System raumliche 3D-Daten erzeugt werden konnen, kann man nur mit eCley 3D-Rekonstruktion, Lokalisierung und Entfernungsmessung erledigen, die man vorher mit anderen Geraten durchfuhren musste
Zwar konnen je zwei horizontal benachbarte Mikrokameras im eCley als ein Stereo-Sehsystem genutzt werden, aber es ist nicht leicht, die raumlichen Informationen durch so kleine Kameras zu erhalten Die von der Mikrokamera gemachten Fotos haben nur eine ziemlich niedrige Auflosung Auserdem ist die Tiefenveranderung der Szene kleiner als 1 Pixel, wenn die Entfernung groser als 86mm ist, dh dass viele verbreitete Algorithmen zum Stereosehen mit eCley nicht gut funktionieren konnen
Um die verbreiteten Stereosehalgorithmen mit dem eCley besser anwenden zu konnen, wurde eine neue Methode dafur im Bereich des Subpixel-Stereosehen erstellt Diese Methode basiert auf der positiven Eigenschaft des eCleys, dass die Kanten des Ziels im eCley sehr gut behalten werden konnen Im Ubergang zwischen Bilder benachbarter Mikrokameras gibt es zahlreiche Tiefeninformationen Mit diesen Tiefeninformationen kann der entsprechende Subpixelabstand ausgerechnet werden Danach kann die Entfernung des Ziels mit dem Subpixelabstand berechnet werden Aufgrund der Struktur des eCleys haben wir in dieser Doktorarbeit ein mathematisches Modell des Stereosehens fur eCley abgeleitet Dazu werden die optische Ausrichtung und die geometrische Korrektur, die die Voraussetzungen zur prazisen Messung sind, diskutiert Zum Schluss haben wir die Subpixel-Baseline-Methode, die auf der Helligkeit und den Gradienten basiert, und die Echtzeit-Messung fur den Subpixelabstand, die auf der Eigenschaft der Kanten basiert, entwickelt 
Um unsere Methode zu uberprufen, haben wir viele kunstliche und reale Szenenbilder angewendet Das Ergebnis zeigt, dass unsere Methode die Messung zum Subpixelabstand fur Stereopixelpaare ausgezeichnet realisiert hat Auserdem funktioniert diese Methode in vielen komplexen Umgebungen robust Das bedeutet, dass die Methode die Fahigkeit des eCleys verbessert hat, die 3D-Umgebung zu erkennen Das eCley kann daher in verschiedenen 3D-Anwendungsbereichen eingesetzt werden 
In the last decade, scientists have put forth many artificial compound eye systems, inspired by the compound eyes of all kinds of insects These systems, employing multi-aperture optical systems instead of single-aperture optical systems, provide many specific characteristics, such as small volume, light weight, large view, and high sensitivity Electronic cluster eye (eCley) is a state-of-the-art artificial superposition compound eye with super resolution, which is inspired by a wasp parasite called the Xenos Peckii Thanks to the inherent characteristics of eCley, it has successfully been applied to aspects of medical inspection, personal identification, bank safety, robot navigation, and missile guidance However, all these applications only involve a two-dimensional image space, ie, no three-dimensional (3D) information is provided Conceiving of the ability of detecting 3D space information using eCley, the performances of 3D reconstruction, object position, and distance measurement will be obtained easily from the single eCley rather than requiring extra depth information devices

In practice, there is a big challenge to implementing 3D space information detection in the minimized eCley, although structures similar to stereo vision exist in each pair of adjacent channels In the case of an imaging channel with short focal length and low resolution, the determination of the depth information not only is an ill-posed problem but also varies in the range of one pixel from quite near distance (≥86 mm), which restricts the applicability of popular stereo matching algorithms to eCley

Taking aim at this limitation, and with the goal of satisfying the real demands of applications in eCley, this thesis mainly studies a novel method of subpixel stereo vision for eCley This method utilizes the significant property of object edges still retained in eCley, ie, the transitional areas of edges contain rich information including the depths or distances of objects, to determine subpixel distances of the corresponding pixel pairs in the adjacent channels, to further obtain the objects' depth information by employing the triangle relationship
In the whole thesis, I mainly deduce the mathematical model of stereo vision in eCley theoretically based on its special structure, discuss the optical correction and geometric calibration that are essential to high precision measurement, study the implementation of methods of the subpixel baselines for each pixel pair based on intensity information and gradient information in transitional areas, and eventually implement real-time subpixel distance measurement for objects through these edge features

To verify the various methods adopted, and to analyze the precision of these methods, I employ an artificial synthetical stereo channel image and a large number of real images captured in diverse scenes in my experiments The results from either a process or the whole method prove that the proposed methods efficiently implement stereo vision in eCley and the measurement of the subpixel distance of stereo pixel pairs Through a sensitivity analysis with respect to illumination, object distances, and pixel positions, I verify that the proposed method also performs robustly in many scenes This stereo vision method extends the ability of perceiving 3D information in eCley, and makes it applicable to more comprehensive fields such as 3D object position, distance measurement, and 3D reconstruction

Stereo Vision for Facet Type Cameras

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Tao Jiang

Author Tools

Chat about Author

Papers

RoadFormer: Pyramidal deformable vision transformers for road network extraction with remote sensing images

CapViT: Cross-context capsule vision transformers for land cover classification with airborne multispectral LiDAR data

SignHRNet: Street-level traffic signs recognition with an attentive semi-anchoring guided high-resolution network

Stereo Vision for Facet Type Cameras