Temporal Vegetation Modelling with Recurrent Networks

Land Cover Classification (LCC) approaches traditionally concentrate on spectral and textural features. However, some classes (e.g. crops) distinguish themselves with characteristic spectro-temporal behaviour, which can be utilized for classificaton. Common multi-temporal approaches concentrate either on temporal statistics, hidden Markov models and to some degree kernel based techniques. These approaches, however, require complex preprocesssing and feature extraction to achieve good results. Deep Learning techniques on the other hand are trained via end-to-end learning regime, thus having potential to simplify training and classification pipelines and to be applied for a variety of applications.

In this project we employ Long Short-Term Memory (LSTM) neural networks, which are used e.g. in speech recognition, image captioning, or text generation, for multi-temporal crop classification. By utilizing information from the previous observation (short term) and reading and writing to memory vector (long term) at every observation, LSTMs can remember previous states for a theoretically unlimited amount of timesteps. We train a LSTM network with a series of 26 Sentinel-2A images and information of 137 000 fields in the growth season 2016. Geometric and semantic information of these fields is provided by the Bavarian Ministry of Agriculture. We show that our LSTM network utilizes phenological changes in crop development to classify 19 crops classes with state-of-the-art accuracies.

Code:

https://github.com/TUM-LMF/fieldRNN

Publications

  • Rußwurm, Marc; Lefèvre, Sébastien; Courty, Nicolas; Emonet, Rémi; Körner, Marco; Tavenard, Romain: End-to-end Learning for Early Classification of Time Series. arXiv preprint arXiv:1901.10681, 2019 mehr…
  • Rußwurm, Marc; Körner, Marco: Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders. ISPRS International Journal of Geo-Information 7 (4), 2018, 129 mehr…
  • Rußwurm, Marc; Körner, Marco: Convolutional LSTMs for Cloud-Robust Segmentation of Remote Sensing Imagery. Proceedings of the Conference on Neural Information Processing Systems Workshops (NeurIPSW), 2018 mehr…
  • Rußwurm, Marc; Körner, Marco: Multitemporal Crop Identification from Medium-Resolution Multi-Spectral Satellite Images based on Long Short-Term Memory Neural Networks. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (ISPRS) XLII-1/W1, 2017, 551--558 mehr…
  • Rußwurm, Marc; Körner, Marco: Temporal Vegetation Modelling using Long Short-Term Memory Networks for Crop Identification from Medium-Resolution Multi-Spectral Satellite Images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2017 mehr…
  • Rußwurm, Marc; Körner, Marco: Temporal Vegetation Modelling using Long Short-Term Memory Networks for Crop Identification from Medium-Resolution Multi-Spectral Satellite Images. (anderer Eintrag) mehr…