Cross-city matters: A multimodal remote sensing benchmark dataset for cross-city semantic segmentation using high-resolution domain adaptation networks

Artificial intelligence (AI) approaches nowadays have gained remarkable success in single-modality-dominated remote sensing (RS) applications, especially with an emphasis on individual urban environments (e.g., single cities or regions). Yet these AI models tend to meet the performance bottleneck in the case studies across cities or regions, due to the lack of diverse RS information and cutting-edge solutions with high generalization ability. To this end, we build a new set of multimodal remote sensing benchmark datasets (including hyperspectral, multispectral, SAR) for the study purpose of the cross-city semantic segmentation task (called C2Seg dataset), which consists of two cross-city scenes, i.e., Berlin-Augsburg (in Germany) and Beijing-Wuhan (in China). Beyond the single city, we propose a high-resolution domain adaptation network, HighDAN for short, to promote the AI model's generalization ability from the multi-city environments. HighDAN is capable of retaining the spatially topological structure of the studied urban scene well in a parallel high-to-low resolution fusion fashion but also closing the gap derived from enormous differences of RS image representations between different cities by means of adversarial learning. In addition, the Dice loss is considered in HighDAN to alleviate the class imbalance issue caused by factors across cities. Extensive experiments conducted on the C2Seg dataset show the superiority of our HighDAN in terms of segmentation performance and generalization ability, compared to state-of-the-art competitors. The C2Seg dataset and the semantic segmentation toolbox (involving the proposed HighDAN) will be available publicly at https://github.com/danfenghong.

pdf/cross-city-matters-a-multimodal-remote-sensing-benchmark-56l4xkmgqt.pdf

Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networks


 Satellite remote sensing is transforming coastal science from a “data-poor” field into a “data-rich” field. Sandy beaches are dynamic landscapes that change in response to long-term pressures, short-term pulses, and anthropogenic interventions. Until recently, the rate and breadth of beach change have outpaced our ability to monitor those changes, due to the spatiotemporal limitations of our observational capacity. Over the past several decades, only a handful of beaches worldwide have been regularly monitored with accurate yet expensive in situ surveys. The long-term coastal-change data of these few well-monitored beaches have led to in-depth understanding of many site-specific coastal processes. However, because the best-monitored beaches are not representative of all beaches, much remains unknown about the processes and fate of the other >99% of unmonitored beaches worldwide. The fleet of Earth-observing satellites has enabled multiscale monitoring of beaches, for the very first time, by providing imagery with global coverage and up to daily frequency. The long-standing and ever-expanding archive of satellite imagery will enable coastal scientists to investigate coastal change at sites vulnerable to future sea-level rise, that is, (almost) everywhere. In the past decade, our capability to observe coastal change from space has grown substantially with computing and algorithmic power. Yet, further advances are needed in automating monitoring using machine learning, deep learning, and computer vision to fully leverage this massive treasure trove of data. Extensive monitoring and investigation of the causes and effects of coastal change at the requisite spatiotemporal scales will provide coastal managers with additional, valuable information to evaluate problems and solutions, addressing the potential for widespread beach loss due to accelerated sea-level rise, development, and reduced sediment supply. Monitoring from Earth-observing satellites is currently the only means of providing seamless data with high spatiotemporal resolution at the global scale of the impending impacts of climate change on coastal systems.

/pdf/the-future-of-coastal-monitoring-through-satellite-remote-1i5pho6a.pdf

The future of coastal monitoring through satellite remote sensing

Cross-sensor domain adaptation for high spatial resolution urban land-cover mapping: From airborne to spaceborne imagery

Change Detection (CD) provides a research basis for environmental monitoring, urban expansion and reconstruction as well as disaster assessment, by identifying the changes of ground objects in different time periods. Traditional CD focused on the Binary Change Detection (BCD), focusing solely on the change and no-change regions. Due to the dynamic progress of earth observation satellite techniques, the spatial resolution of remote sensing images continues to increase, Multi-class Change Detection (MCD) which can reflect more detailed land change has become a hot research direction in the field of CD. Although many scholars have reviewed change detection at present, most of the work still focuses on BCD. This paper focuses on the recent progress in MCD, which includes five major aspects: challenges, datasets, methods, applications and future research direction. Specifically, the background of MCD is first introduced. Then, the major difficulties and challenges in MCD are discussed and delineated. The benchmark datasets for MCD are described, and the available open datasets are listed. Moreover, MCD is further divided into three categories and the specific techniques are described, respectively. Subsequently, the common applications of MCD are described. Finally, the relevant literature in the main journals of remote sensing in the past five years are analyzed and the development and future research direction of MCD are discussed. This review will help researchers understand this field and provide a reference for the subsequent development of MCD. Our collections of MCD benchmark datasets are available at: https://zenodo.org/record/6809804#.YsfvxXZByUk 

A review of multi-class change detection for satellite remote sensing imagery

Earth observation (EO) technologies, such as optical imaging and synthetic aperture radar (SAR), provide excellent means to continuously monitor ever-growing urban environments. Notably, in the case of large-scale disasters (e.g., tsunamis and earthquakes), in which a response is highly time-critical, images from both data modalities can complement each other to accurately convey the full damage condition in the disaster aftermath. However, due to several factors, such as weather and satellite coverage, which data modality will be the first available for rapid disaster response efforts is often uncertain. Hence, novel methodologies that can utilize all accessible EO datasets are essential for disaster management. In this study, we developed a global multimodal and multitemporal dataset for building damage mapping. We included building damage characteristics from three disaster types, namely, earthquakes, tsunamis, and typhoons, and considered three building damage categories. The global dataset contains high-resolution (HR) optical imagery and high-to-moderate-resolution SAR data acquired before and after each disaster. Using this comprehensive dataset, we analyzed five data modality scenarios for damage mapping: single-mode (optical and SAR datasets), cross-modal (pre-disaster optical and post-disaster SAR datasets), and mode fusion scenarios. We defined a damage mapping framework for semantic segmentation of damaged buildings based on a deep convolutional neural network (CNN) algorithm. We also compared our approach to another state-of-the-art model for damage mapping. The results indicated that our dataset, together with a deep learning network, enabled acceptable predictions for all the data modality scenarios. We also found that the results from cross-modal mapping were comparable to the results obtained from a fusion sensor and optical mode analysis.

Learning from multimodal and multitemporal earth observation data for building damage mapping

Building damage such as to side-walls or mid-story collapse is often overlooked in vertical optical images. Hence, in
order to observe such building damage modes, high-resolution SAR images are introduced considering the side-looking
nature of SAR. In the 2011 Tohoku, Japan, earthquake, a large number of buildings were collapsed or severely damaged
due to repeated tsunamis. One of the important tsunami effects on buildings is that the damage is concentrated to their
side-walls and lower stories. Thus this paper proposes the method to detect this kind damage from the change in layover
areas in SAR intensity images. Multi-temporal TerraSAR-X images covering the Sendai-Shiogama Port were employed
to detect building damage due to the tsunamis caused by the earthquake. The backscattering coefficients in layover areas
of individual buildings were extracted and then, the average value in each layover area was calculated. The average value
was seen to decrease in the post-event image due to the reduced backscatter from building side-walls. This example
demonstrated the usefulness of high-resolution SAR intensity images to detect severe damage to building side-walls
based on the changes of the backscattering coefficient in the layover areas.

Detection of damage to building side-walls in the 2011 Tohoku, Japan earthquake using high-resolution TerraSAR-X images

A method for capturing the two-dimensional (2D) surface movements from two temporal TerraSAR-X (TSX) intensity images has been proposed by the authors in previous research. However, it is impossible to detect the three-dimensional (3D) actual displacement from one pair of TSX images. Hence, three pairs of TSX images taken in ascending and descending paths were used to estimate 3D crustal movements in this study. First, the 2D crustal movements due to the 2011 Tohoku earthquake were detected from the three sets respectively. The relationship between the 3D actual displacement and 2D converted movement in SAR images was derived according to the observation model and shooting condition of the SAR sensor. Then the absolute 3D movements were estimated by the combination of the detected 2D movements that occurred within a short time interval. The results were verified by the GEONET observation records.

Estimation of three-dimensional crustal movements from mutli-temporal TerraSAR-X intensity images

A method for capturing the two-dimensional (2D) surface movements from two temporal TerraSAR-X (TSX) intensity images has been proposed by the authors in the previous research. In this study, three pairs of TSX images taken in ascending and descending paths were used to estimate 3D crustal movements by the proposed method. First, the 2D crustal movements were detected from the three sets respectively. Then the absolute 3D movements were estimated by the combination of the detected 2D movements. The results were verified by the GEONET observation records.

Detection of three-dimensional crustal movements due to the 2011 tohoku earthquake from sar images

Significant crustal movements were caused by the 2 011 Tohoku, Japan earthquake. A method for capturing the surface movements from T erraSAR-X intensity images is proposed in this study. First, the buildings were extracted from the pre- and post-event SAR images using a segmentation approach. Then, the unchanged buildings were detected by matching the buildings in the pre- and post-event images at similar locations. Finally, the shifts were calculated and considered as crustal movements. The method was tested and its result was compared with GPS observation records. The proposed method was found to be able to detect crustal movement at a subpixel level.

Detection of crustal movements for the 2011 tohoku, japan earthquake from multi-temporal high- resolution sar intensity images

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Wen Liu

Author Tools

Chat about Author