Geoinformatics Unit Team (Talk by Xiaoyan Lu).

Speaker: Xiaoyan Lu

Title
Mapping global-scale roads with geospatial big data and deep learning networks

Abstract
Rapid road expansion is currently underway and is expected to continue this century, which is promoted by a multitude of factors, including urbanization and increased demands for travel and transportation. Accurate and up-to-date mapping and extraction of road networks are critical to maintaining the function of urbanicity and socioeconomic development. Although there are some available datasets like OpenStreetMap (OSM) mapping roads globally, they are typically outdated, spatially incomplete, and uneven in accuracy. Moreover, the efficiency and accuracy of crowdsourcing road-mapping efforts such as OSM cannot meet the requirement of real-world urban applications. In recent years, with the successful launch of a series of high-resolution remote sensing satellites at home and abroad, such as IKONOS, QuickBird, Worldview, GJ-1, GF-7, etc., the ability to acquire very high resolution (VHR) remote sensing data has been greatly enhanced, enabling us to obtain massive VHR remote sensing imagery. High resolution remote sensing images contain abundant spatial details and can accurately describe the geometric properties of ground objects, which have become the main data source for road extraction by virtue of its mature acquisition means, high spatial resolution and wide coverage.
To deal with massive remote sensing imagery, the most widely used method is based on deep learning. The emergence of deep learning has made self-learning features a reality, and opened the paradigm of data-driven "representation learning", and avoided the problems of complex feature design and manual intervention in traditional methods. However, deep learning methods rely on massive data to train models, and VHR remote sensing images, due to their high spatial resolution and highly detailed spatial information, bring a lot of interference to road extraction. For example: 1) At present, deep learning methods cannot effectively achieve the unified expression of road surface segmentation, road centerline extraction and road edge detection; 2) Buildings, trees and other ground objects on complex VHR remote sensing images bring occlusion and shadow to road recognition, resulting in discontinuous road results; 3) In global-scale applications, the image radiation varies greatly in different regions, leading to the serious road missing in model outputs.
To solve the problems of deep learning model for road extraction from very high- resolution remote sensing imagery, based on the powerful self-learning ability and feature expression ability of deep learning, this study develops the road extraction methods to address the three problems: multi-task road extraction, road discontinuity, and serious road missing. The main research contents are summarized as follows:
(1) In the aspect of multi-task expression, the three tasks of road extraction lack a unified expression framework. Based on the symbiotic relationship between the road surface, the road centerline and the road edge, this study proposes a cascaded multi-task road extraction framework, which can not only complete road surface segmentation, centerline extraction and edge detection simultaneously, but also the three tasks can constrain and promote each other. In this framework, the road surface segmentation network is the backbone network, and the centerline extraction branch and edge detection branch learn from the road surface segmentation result and low-level features of the backbone network. At the same
time, complex VHR remote sensing images bring many disturbances to the road, making road recognition difficult, the multi-task framework uses the context-aware module to capture the long-distance spatial context information, and alleviate the local interference problem, so as to obtain topologically correct roads. In terms of model optimization, the framework uses the hard example mining loss to improve the road recognition rate by reducing the false negatives and focusing more on hard samples.
(2) In the aspect of road discontinuities, the convolution operations in convolutional neural networks process one local neighborhood at a time, which leads to the problem of discontinuous road extraction results when there is local interference. To address this problem, this thesis proposes a globally aware road detection network to capture the long-range dependencies in the feature extraction process, to mitigate the incomplete road extraction phenomenon. The network applies multi-scale residual learning to obtain multi-scale features and expand the receptive field, and uses the global awareness operation to capture the spatial context dependencies and inter-channel dependencies. Through capturing useful information over long distances, the proposed network can significantly improve the road connectivity.
(3) In the aspect of serious road missing, the model performance of road extraction decreases sharply and the road is missing seriously in the global-scale application. Based on the idea of adversarial learning, this study uses the domain adaptation technique to make the model trained in the source domain be better generalized to the target domain. This study proposes a global-local adversarial learning framework for cross-domain road extraction. The framework enhances the model generalization ability from two aspects: a) considering the spatial information similarities between the source and target domains, feature space driven adversarial learning is applied to explore the shared features across domains; b) the complex background of remote sensing images makes some roads easy to recognize, while others are much more difficult. The framework uses a local alignment operation to adaptively adjusts the weight of the adversarial loss according to the road recognition difficulty.
In addition, considering there is already a lot of historical geospatial big data: OSM road centerline available, which can be used as auxiliary supervision information for the target domain. This study proposes an open-source data-driven domain-specific representation framework. The framework improves the cross-domain road extraction performance from two aspects: a) considering both the similarity of the structural information and the difference of the texture information between the two domains, the framework is introduced to learn the domain-invariant structure features and domain-specific texture information; b) crowdsourced OSM data are freely available, and OSM data have road centerlines covering almost all regions in the world. Therefore, OSM road centerlines with fixed buffers are incorporated in the network training process to enable the framework to better adapt to the target domain.
Besides, this study proposes a robust global-scale road benchmark dataset—GlobalRoadSet (GRSet)—and a pseudo-label guided global-scale road extraction network—GlobalRoadNet (GRNet). This method improves the global-scale road extraction performance from two aspects: a) a robust global-scale road benchmark
dataset—GRSet—was built with open-source data without any manual annotation. The GRSet dataset has a large data volume and high data diversity, covering the six continents of Europe, Africa, Asia, South America, Oceania, and North America, with an area of 49,503 km2; b) a pseudo-label guided global-scale road extraction method—GRNet—is proposed. GRNet is first pre-trained on the GRSet dataset, to provide initialization parameters for the road extraction network. The pre-trained GRNet then produces pseudo-labels for the unlabeled images of the target region, along with the few labeled samples, to retrain the network, to adapt the model to the target region.
This study conducts the research of deep learning methods to address the three problems: multi-task road extraction, road discontinuity, and serious road missing. This is able to significantly improve the model performance of global-scale road extraction, and further promotes the global-scale practical application, therefor have important scientific and social values for many fields, such as maintaining the function of urbanicity and socioeconomic development.
Key words: road extraction, high-resolution remote sensing imagery, deep learning, multi-task, global-scale, cross-domain