Deep Learning Approaches to the Analysis of Remote Sensing Image Time Series

In recent years, advancements in Remote Sensing (RS) technology have led to the development of new satellite constellations and sensors capable of capturing data with higher temporal and spatial details. Multispectral (MS) data have become standard for Earth observation (EO), providing rich time series essential for understanding changes over time. Analysing satellite image time series (SITS) is increasingly important due to the continuous acquisition of data, offering opportunities to detect and understand large-scale phenomena. However, this requires robust methods capable of incorporating temporal information and performing accurately with limited training data, which is common in remote sensing applications. In this context, this thesis contributes to four main areas: (i) design of a SITS classification system for multi-year local-scale crop-type mapping, (ii) extension of the system to multi-year large-scale crop-type mapping and its application to the estimation of water irrigation demand, (iii) definition of robust classification architectures capable to be accurate with limited labelled data, and (iv) proposal of novel classification techniques capable to exploit the temporal consistency of data acquired over large time intervals. The first contribution focuses on the design of a system for the classification of high resolution SITS at local-scale for a single-year and its adaptation to multi-year SITS with limited labelled data, exploiting a hybrid approach based on self-paced learning and fine-tuning. The system has been validated on data acquired in Austria, by considering spatially independent tiles acquired over the full country. The quantitative results pointed out the importance of harmonising time series from different areas, showing an Overall Accuracy (OA) of 84.54% on the test set against the 83.23% of a single non-harmonised tile. In the multi-year scenario, the self-paced approach combined with fine-tuning optimised on only a hundred samples notably improved the performance compared to the direct application of the pre-trained architecture to a different agronomic year. Indeed, quantitative results showed mF-Score (F1) values of 76.62% and 70.13% on 2019 and 2020, respectively, against a 64,24% and 62.88% of the pretrained model. The second contribution addresses the design of a system for the generation of crop-type maps, which are then exploited by an agrohydrological model to produce water irrigation demand maps. Unlike most of the literature, which focuses on small test-case scenarios, the method leverages proper pre-processing chain and Long Short-Term Memory Neural Networks (LSTMs) to estimate the crop-type maps on three different years at country/sub-continental scale. As a side but important outcome of this work, a very large agricultural dataset has been developed and published open-source, to further evaluate the methodologies. The methodologies have been trained in the full Austrian country, and validated on three agronomic years (i.e., 2018-2019-2020) in the Danube region, spanning approximately ∼357.752 km2. Quantitative results highlight the generalisation capabilities of the system, showing similar performances on the training country and the other analysed regions. Indeed, the proposed system achieved and OA of 89.10% in the training region, and 84.38% in the testing areas. The third contribution focuses on the definition of hierarchical modular methodologies that can adapt to different scenarios without significant loss in generalisation, while reducing the number of samples needed to train deep learning (DL) models from scratch. To validate the methodology, experiments were conducted considering spatially independent patches for training and fine-tuning, considering both available generated labelled maps and photointerpreted ground truth samples. In the quantitative analysis, the proposed methodologies showed a mF1 of 90.52% against a 89.00% of the baseline. The fine-tuning experiments to assess the generalisation capabilities of the method confirmed the capability of the network to be adapted with small amount of labelled samples, achieving a mF1 of 64.39% against a 63.55% of the baseline, while a new architecture trained from scratch fails to model the problem. The fourth contribution presents a methodology based on Hidden Markov Models (HMM) to exploit inter-annual land-cover transitions applied to crop-type mapping. It models the crop rotation patterns in a Bayesian-fashion, leveraging the entire temporal correlation to perform the decision process. The methodology has been validated on a Sentinel-2 tile in Austria, considering a stratified random sampling approach based on the unique crop-sequences in six agronomic years. The effectiveness of the proposed approach is quantitatively demonstrated on 47 crop type classes, where the Cascade HMM approach obtained a mF1 of 73.59% against the 70.88% of the standard Transformer. The proposed methodologies have been tested on Sentinel-2 multitemporal time series. Quantitative and qualitative experimental results confirm the effectiveness of the proposed approaches, and their applicability on large-scale scenarios and real use-cases, while also showing improvement compared to existing methodologies.