Single-view height estimation and semantic segmentation have received increasing attention in recent years and play an important role in the photogrammetry and remote sensing communities. The height information and semantic information of images are correlated, and some recent works have shown that multi-task learning methods can achieve complementation of task-related features and improve the prediction results of the multiple tasks. Although much progress has been made in recent works, how to effectively extract and fuse height features and semantic features is still an open issue. In this paper, a self- and cross-enhancement network (SCE-Net) is proposed to jointly perform height estimation and semantic segmentation on single aerial images. A feature separation–fusion module is constructed to effectively separate and fuse height features and semantic features based on an attention mechanism for feature representation enhancement across tasks. In addition, a height-guided feature distance loss and a semantic-guided feature distance loss are designed based on deep metric learning to achieve task-aware feature representation enhancement. Extensive experiments are conducted on the Vaihingen dataset and the Potsdam dataset to verify the effectiveness of the proposed method. The experimental results demonstrate that the proposed SCE-Net could outperform the state-of-the-art methods and achieve better performance in both height estimation and semantic segmentation.

/pdf/sce-net-self-and-cross-enhancement-network-for-single-view-3th9c63t.pdf

SCE-Net: Self- and Cross-Enhancement Network for Single-View Height Estimation and Semantic Segmentation

Height estimation from single images, strictly speaking, is an ill-posed problem. However, recently, it is shown that it is both possible and feasible to learn a mapping from image statistics to height information. In spite of recent efforts in this field, how to learn fine-shape preserving features, such as object boundaries and contours, is still an open issue. In this work, we propose a progressive learning network to estimate height information from single aerial images in a coarse-to-fine manner. In particular, a gated feature aggregation module is introduced to effectively combine low-level and high-level features. The proposed method is validated on three public datasets, including the Vaihingen dataset, the Potsdam dataset, and the DFC2019 dataset. Both quantitative and qualitative experimental results demonstrate that the proposed method can achieve more accurate height estimation from single aerial images, especially with better object boundary and contour preserving capability, than four related height estimation methods. 

Gated Feature Aggregation for Height Estimation From Single Aerial Images

Estimating depth from RGB images can facilitate many computer vision tasks, such as indoor localization, height estimation, and simultaneous localization and mapping (SLAM). Recently, monocular depth estimation has obtained great progress owing to the rapid development of deep learning techniques. They surpass traditional machine learning-based methods by a large margin in terms of accuracy and speed. Despite the rapid progress in this topic, there are lacking of a comprehensive review, which is needed to summarize the current progress and provide the future directions. In this survey, we first introduce the datasets for depth estimation, and then give a comprehensive introduction of the methods from three perspectives: supervised learning-based methods, unsupervised learning-based methods, and sparse samples guidance-based methods. In addition, downstream applications that benefit from the progress have also been illustrated. Finally, we point out the future directions and conclude the paper.

Deep Learning based Monocular Depth Prediction: Datasets, Methods and Applications.

Semantic segmentation and height estimation are two critical tasks in remote sensing scene understanding that are highly correlated with each other. To address both tasks simultaneously, it is natural to consider designing a unified deep learning model that aims to improve performance by jointly learning complementary information among the associated tasks. In this paper, we learn the two tasks jointly under a deep multi-task learning framework and propose two novel objective functions, called cross-task contrastive loss and cross-pixel contrastive loss, respectively, to enhance multi-task learning performance through contrastive learning. Specifically, the cross-task contrastive loss is designed to maximize the mutual information of different task features and enforce the model to learn the consistency between semantic segmentation and height estimation. In addition, our method goes beyond previous approaches that only apply contrastive learning at the instance level. Instead, we design a pixel-wise contrastive loss function that pulls together pixel embeddings belonging to the same semantic class, while pushing apart pixel embeddings from different semantic classes. Furthermore, we find that this semantic-guided contrastive loss simultaneously improves the performance of the height estimation task. Our proposed approach is simple and effective and does not introduce any additional overhead to the model during the testing phase. We extensively evaluate our method on the Vaihingen and Potsdam datasets, and the experimental results demonstrate that our approach significantly outperforms the state-of-the-art methods in both height estimation and semantic segmentation. 

Joint Learning of Semantic Segmentation and Height Estimation for Remote Sensing Image Leveraging Contrastive Learning

Semantic segmentation of remote sensing images is an extensively employed and demanding task. Although deep convolutional neural networks have significantly increased the accuracy of semantic segmentation, the problems of losing detailed features in segmentation and ignoring rich contextual information of images still exist. To solve these challenges, we propose a multilateral semantic with dual relation network (MSDRNet) for remote sensing images segmentation. The proposed MSDRNet consists of two parallel modules, the detail semantic module (DSM) and the global semantic module (GSM), for extracting image detail and global features, respectively. Subsequently, improved spatial relation block (SRB) and channel relation block (CRB) are introduced in two separate parallel modules to further enhance the contextual connection of the images. Finally, a feature refinement module (FRM) is added to balance the multilateral features between the features extracted from the two branches. We display the robustness and effectiveness of the proposed MSDRNet on the publicly available ISPRS Potsdam and Vaihingen datasets. We further experimented with the Gaofen Image Dataset (GID), which contains information on larger scale features, to demonstrate the validity of our model. The results of extensive experiments conducted on the above three datasets show that the proposed approach outperforms several state-of-the-art semantic segmentation methods. 

Multilateral Semantic with Dual Relation Network for Remote Sensing Images Segmentation

Extracting elevation information from single aerial images is an important and challenging task in photogrammetry and remote sensing. The difficulty lies in the ambiguity and ill-condition of the 2-D–3-D projection. As neural networks show advantages in solving ill-posed problems, fully convolutional neural networks (FCNs) have been applied to this task and have achieved worthwhile results. However, the previous works simply focus on the application of the neural network structures originally applied to monocular depth estimations or semantic segmentations to single-image elevation estimations. This procedure ignores the task-specific weak correlation problem between the elevation systems of training labels and outputs and the one between heights of adjacent pixels. To address these problems, we use the FCN to generate height differences between adjacent pixels in specific orientations (i.e., oriented elevation gradients), instead of directly generating the elevation values. Subsequently, the elevation gradients in the two orients of XY are combined to form a huge relationship chain, and the relative elevation of each pixel is calculated using the maximum likelihood method. Finally, a new surface-based soft alignment method is used to establish the correlation between the elevation systems of training labels and predicted outputs robustly. The quantitative evaluation on the International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam data set reveals a significant improvement of the bare FCN. The visual contrast evaluation indicates that the soft-aligned gradient-chaining network can retain more surface features, and its outputs are more reasonable and visually pleasing than the bare FCN.

Soft-Aligned Gradient-Chaining Network for Height Estimation From Single Aerial Images

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Ru Lu

Author Tools

Chat about Author

Papers

Soft-Aligned Gradient-Chaining Network for Height Estimation From Single Aerial Images