Canopy height mapping from optical remote sensing data: reassessing deep learning methods
This page documents the progress of the research on the operating mechanisms of the global canopy height model by Lang et al. (2023a). For the full data and scripts please visit the corresponding GitHub repository and for other projects and more information about me visit my Main Site.
1 Abstract
Accurate canopy height data is crucial for understanding ecosystem structure and function, assessing biodiversity or calculating carbon stocks. Traditional in-situ collection of canopy height data is complex, labor-intensive and spatially limited. While Airborne Laser Scanning enables analysis on a regional or country-wide scale, a global application is not feasible due to high costs. Fusing deep learning, space borne LIDAR samples and optical remote sensing data, recent developments of canopy height models have produced remarkable results. Although spectral data is widely used in ecological analysis, a link to canopy height is unknown. The driving mechanisms of canopy height estimation on a global scale were investigated, using the approach presented by Lang et al. (2023a). To isolate the relevance of optical properties in prediction, controlled spectral manipulation of the input data was performed, and the model reaction analyzed. The observed patterns do not suggest a direct link of spectral values to canopy height. The results demonstrate that its prediction from optical remote sensing data is primarily influenced by factors beyond spectral signatures. Thorough understanding of the mechanisms driving prediction and clear communication of limitations in applicability and explanatory value is crucial in the development of ecological models.
Lang, N., Jetz, W., Schindler, K., & Wegner, J. D. (2023a). A high-resolution canopy height model of the earth. Nature Ecology & Evolution, 7(11), 1778–1789. https://doi.org/10.1038/s41559-023-02206-6
2 Research question
The goal of this research is to understand the role of spectral data in the creation of deep learning approaches in order to predict canopy height, and generate insights to improve future approaches. The lack of a known ecological connection between spectral data and canopy height raises questions on the mechanisms driving prediction. This thesis aims to address the following question:
Are spectral reflectance values the driving force in recent deep learning models predicting canopy height?
It is hypothesized that spectral data does not contain information about canopy height, and that other factors, like coordinates, are more relevant for prediction.
3 Methods
This research builds on an existing model architecture promising global canopy height prediction, delivering high spatial resolution, easy data accessibility and reproducibility (Lang et al., 2023b). The authors claim that the model has learned connections to spectral values as well as texture and present a remarkably low overall Root-Mean-Square-Error (RMSE) of 6.0 m. It was therefore selected to be used as an experimental subject in this research.
ESA. (2020). Copernicus sentinel-2. European Space Agency (ESA). https://browser.dataspace.copernicus.eu
Zanaga, D., Van De Kerchove, R., De Keersmaecker, W., Souverijns, N., Brockmann, C., Quast, R., Wevers, J., Grosu, A., Paccini, A., Vergnaud, S., Cartus, O., Santoro, M., Fritz, S., Georgieva, I., Lesiv, M., Carter, S., Herold, M., Li, L., Tsendbazar, N.-E., … Arino, O. (2020). ESA WorldCover 10 m 2020 v100. Zenodo. https://doi.org/10.5281/zenodo.5571936
As we cannot retrace the reasoning of the model due to the nature of deep learning approaches and their size, the effect of spectral values has to be isolated. To address this limitation, we approach this question by deploying the selected model on systematically manipulated spectral input data to observe the resulting change in canopy height estimates. Assuming the model has learned connections to spectral values, such perturbations would be expected to change proportionally, resulting in an approximately linear explanatory relationship between the magnitude of the spectral manipulation and the corresponding change in the predicted canopy height. Consistent with the original paper, Sentinel-2 imagery (ESA, 2020) and ESA-WordCover data (Zanaga et al., 2020) from 2020 was used.
To systematically analyze the role of spectral data in deep learning models for estimating canopy height, an automated workflow was developed (See Figure 1). This workflow encompasses data preparation, variable creation, file management, model deployment, and the export of results. Figure 1 illustrates the key elements of the process and the sequence of operations. The spectral manipulation function is integrated into the model deployment scripts. The original code of the model remained unchanged.
4 Sample Tiles
Sample tiles at eleven different locations were selected to ensure broad geographic representation across all continents, covering a wide range of latitudes, elevations, and biomes. I used the three sample tiles presented in the paper as well as the tile provided as demo data (Lang et al., 2023b). This set was extended by three European forests under different management regimes, of which two are located outside the GEDI reference range of 51.6° N-S (Dubayah et al., 2020). Four additional tiles from various continents, considered to be representative of their respective regions, were included. Care was taken to ensure sufficient vegetation cover, within each individual tile.
Lang, N., Jetz, W., Schindler, K., & Wegner, J. D. (2023b). A high-resolution canopy height model of the Earth [Source Code]. https://github.com/langnico/global-canopy-height-model
Dubayah, R., Blair, J. B., Goetz, S., Fatoyinbo, L., Hansen, M., Healey, S., Hofton, M., Hurtt, G., Kellner, J., Luthcke, S., Armston, J., Tang, H., Duncanson, L., Hancock, S., Jantz, P., Marselis, S., Patterson, P. L., Qi, W., & Silva, C. (2020). The Global Ecosystem Dynamics Investigation: High-resolution laser ranging of the Earth’s forests and topography. Science of Remote Sensing, 1. https://doi.org/10.1016/j.srs.2020.100002
5 Results
5.1 Influece of spectral properties on prediction
The results do not show any linear relationship, but rather a symmetric behavior around 0 along the x-axis. The direction of manipulation does not influence the direction and magnitude of differences. Figure 2 illustrates the average absolute difference by absolute manipulation degree for each band, across all tiles.
5.2 Differences in response between sample tiles
The impact of spectral manipulation on canopy height prediction was assessed across eleven globally distributed sample tiles. The selection of these tiles tried to cover a wide range of biomes, continents, management regimes and latitudes, both inside and outside the range of the reference data. They differ in landscape structure and species composition, resulting in different ranges of canopy heights. The distribution of unmanipulated canopy height estimations per tile is illustrated in Figure 3, sorted by mean canopy height from top to bottom. The tiles Finland and Mongolia consist of very low vegetation with an average lower than 5 m. Malaysia has the highest predicted average canopy height with 43.5 m. Switzerland and Cameroon show a very wide spectrum of canopy heights with a very strong distinction between high and low vegetation in Cameroon.
6 Discussion
7 Conclusion
This research demonstrates that spectral values are not a primary driver in the operating mechanism of optical data based deep learning models for canopy height prediction. While sensitivity to the spectral input was observed, no systematic pattern emerged linking spectral values directly to canopy height. Instead, my results indicate a substantial influence of tiles specific properties, including the canopy height it is deployed on, representation of biomes in the training data, local unique characteristics and geographical coordinates. Furthermore, substantial variability within the individual models of the ensemble prediction influences prediction accuracy.
These findings contribute to the development of deep learning models in ecological modeling by highlighting the importance of spatial context and model variability. Future research should explore the influence of spatial patterns and texture, to further refine our understanding of the driving mechanisms of these models.
Deep learning, ensemble prediction and the use of publicly accessible remote sensing data have the potential to significantly expand our knowledge of ecosystems, improve modeling approaches and facilitate large scale analysis. However, understanding the spatial and ecological limits of model application is crucial. It needs to be our goal to understand the products we produce and communicate its limitations. Model metrics do not tell the full story, ignoring its applicability and prediction mechanisms. What a model can’t do and where it is applicable is even more important than its performance.
As creators of deep learning models, we need to clearly understand and communicate their operating principles, to avoid misunderstandings and build trust. This ensures a high quality of derivative products and information, as well as a strong explanatory value. Establishing common standards for evaluation, contextual calibration and unambiguous communication of application limits are the keystones for future development. Measures of applicability are vital for model assessment. This way we ensure high quality products using the full potential of deep learning in remote sensing.
8 Resources
Sildes were created for the ILÖK Graduate Conference, at an intermediate state of the Project. Newer slides will be provided soon, in preperation for the GFÖ Conference.