Summary

The Product Validation Report (PVR) for the product H16/SM ASCAT-B O12.5 NRT, METOP-B ASCAT soil moisture 12.5 km sampling NRT (hereafter called H16), shows the performance of H16 products over Europe through a detailed comparison with both in situ and modelled soil moisture data in the 20 month period from 2013-05-01 to 2014-12-31. The analysis will be carried out following the structure already tested for the validation of the product H08/SM ASCAT DIS NRT, ASCAT disaggregated soil moisture at 1 km NRT. Similarly, the comparison with modelled soil moisture data in Italy has been done by considering the soil moisture based on both Metop-A (H101/SM ASCAT-A O12.5 NRT) and Metop-B (H16/SM ASCAT-B O12.5 NRT) for determining clearly the consistency between the two soil moisture products.

In situ observations

In situ soil moisture observations for 242 stations/sensors in Austria (9 stations/sensors), Bulgaria (27), Denmark (94), France (21), Germany (40), Italy (26), Spain (20), and UK (5) are selected for the validation of the H16 soil moisture product by considering the period May 2013 - December 2014. It is worth noting that most of the data are obtained from the International Soil Moisture Network hosting facility ISMN. Figure 1 shows the location of the employed soil moisture stations while Table A1 in the Appendix summarizes their main characteristics. Note that, for some of the stations, soil moisture measurements are taken at several depths (5, 10, 15, 20, ... cm). Specifically, the analysis is carried out also separately for sensors located at depths lower (136), and greater (106), than 10 cm.

Figure 1: Location of 242 in situ stations used for the validation of the H16 soil moisture product

Figure 1: Location of 242 in situ stations used for the validation of the H16 soil moisture product

Modelled data

A further validation of the H16 product is carried out through a comparison with modelled data deriving from hydrological models. In particular, two models have been applied in two different countries are employed: the hydrological models SCHEME [1] in Belgium, and the soil water balance model SWBM-GA [2], [3] in Italy. Specifically, the SCHEME model was applied to the Demer catchment (Scheldt River Basin). The SWBM-GA model was applied to a regular grid of 1806 points (spacing of 12.5 km) covering the whole Italian country. In both cases, the two hydrological models were forced with high-quality ground-based meteorological observations (i.e., rainfall and air temperature) thus ensuing the good performance of the selected models. It should be underlined that due to their uncertainties, in situ measurements and modelled data should be integrated to achieve a more comprehensive and efficient assessment of the reliability of satellite soil moisture products [4]. Indeed, the use of modelled data is highly important to reduce the spatial mismatch between in situ observations (representative of ~1 cm²) and satellite observations (~500 km²). Hydrological models are forced with meteorological observations averaged over the satellite pixel size, or at basin scale. Therefore, the spatial scale of modelled and satellite data is nearly the same, differently from in situ observations.

Satellite data pre-processing

Currently, the H16 product is archived in the EUMETcast dissemination system and can be downloaded through the EUMETSAT Data Centre Online Ordering application. H16 data are stored in orbit files and can be downloaded in different format (NetCDF, HDF5, EPS, BUFR). For this analysis, the orbit data passing over Europe from METOP-A and METOP-B satellites have been downloaded in NetCDF format from 2013-05-01 to 2015-01-01. The data are processed for passing from orbit files to a regular grid with spacing 12.5 km, the same grid used for the H25 (METOP ASCAT Soil Moisture Time Series) product. The interpolation is simply carried out with the nearest neighbourhood method.

Method of comparison

The performance metrics used for the evaluation of the accuracy of the H16 product are the Pearson correlation coefficient (CC), the Root Mean Square Difference (RMSD), and the bias or mean error (i.e. difference between mean of satellite and in situ). The H16 soil moisture product is an index between 0 and 100% while in situ measurements of soil moisture, and also modelled data, are usually expressed in m³m⁻³. Therefore, accordingly to the product validation reports, the in situ observations are converted between 0 and 1 by linearly rescaling for the maximum and minimum values. Then, the comparison with the H16 product is carried out and the dimensionless scores (RMSD and BIAS) are computed. Finally, the RMSD in m³m⁻³ is computed by multiplying the resulting dimensionless RMSD for the dynamical range of the in situ data (maximum - minimum).

Results for in situ observations

The validation with in situ observations is subdivided by sensor depth as the direct comparison between satellite and in situ observations makes sense only for sensors located close to the surface (≤ 10 cm), as the layer depth sensed by the satellite sensor is equal to 0 - 2.5 cm. However, as measurements are also available for deeper layers, the comparison also with these data sets is reported for sake of completeness. All the performance scores, also subdivided for the different seasons, are reported in Table A2 in the Appendix. In Figure 2 the Pearson Correlation Coefficient (CC), between in situ and satellite data for the whole period and subdivided by seasons is reported by grouping the data by country and by taking into account only in situ sensors located at depth ≤ 10 cm. Figure 3 shows the same comparison for the remaining sensors. In the figures, the product requirements are also reported, i.e., threshold accuracy = 0.50, target accuracy = 0.65, and optimal accuracy = 0.80.

Figure 2: Box plot of the Pearson correlation coefficient (CC) between H16 and in situ observations for the sensors at depth ≤ 10 cm subdivided by country and by season (JJA: June-July-Aug, SON: September-October-November, DJF: December-January-February, MAM: March-April-May, ALL: whole period/all data). The thresholds for CC are also shown.

Figure 2: Box plot of the Pearson correlation coefficient (CC) between H16 and in situ observations for the sensors at depth ≤ 10 cm subdivided by country and by season (JJA: June-July-Aug, SON: September-October-November, DJF: December-January-February, MAM: March-April-May, ALL: whole period/all data). The thresholds for CC are also shown.

Figure 3: As in Figure 2 for the sensors at depth > 10 cm

Figure 3: As in Figure 2 for the sensors at depth > 10 cm

By comparing the prescribed requirements with the obtained results for the surface in situ sensors (depth ≤ 10 cm), it is found that 66%, 28% and 1% of stations/sensors overcomes the threshold, target and optimal requirements, respectively. Similarly, for the remaining sensors (depth > 10 cm), 42%, 7% and 0% of data overcomes the threshold, target and optimal requirements, respectively. It should be noted that the product accuracy is varying by season and by country. Specifically, by considering the surface sensors, the better performance are obtained in the SON (September-October-November) and DJF (December-January-February) periods with median value equal to 0.62 very close to the target requirement. Moreover, much better performance are obtained for stations located in Italy, Spain and UK. In the DJF period for Italy, the median correlation reach a value equal to 0.83. To summarize the results of the comparison of satellite and in situ observations, Table 1 reports the median CC for the in situ sensors by considering the whole period and the different seasons. Therefore, the median CC for the in situ stations at depth ≤ 10 cm is equal to 0.57, with a value significantly higher (better) than the threshold requirement. By excluding the in situ observations from HOBE network in Denmark, that are affected by snow and freezing conditions, the median correlation reach a value of 0.60, quite close to the target requirement.

Table 1: Median Pearson Correlation Coefficient (CC) for the comparison between the H16 product and in situ observations (JJA: June-July-Aug, SON: September-October-November, DJF: December-January-February, MAM: March-April-May, ALL: whole period).
Period 136 Sites (Depth ≤10cm) 106 Sites (Depth >10cm) All 215 sites
JJA 0.17 0.06 0.11
SON 0.62 0.53 0.59
DJF 0.62 0.46 0.55
MAM 0.25 0.15 0.19
ALL 0.57 0.47 0.53

Results for modelled data

The analysis carried out with in situ observations is also performed by using modelled data and the summary results are reported in Table 2. All the performance scores, also subdivided for the different seasons, are reported in the Table A3 in the Appendix. For the case study in Belgium, very good performance are obtained with a median correlation equal to 0.84 that is better than the optimal requirement.

Table 2: Median Correlation Coefficient (CC) for the comparison between the H08 product and modelled data. (* median values of 1806 grid points)
Period Belgium Italy*
JJA 0.59 0.34
SON 0.83 0.67
DJF 0.55 0.63
MAM 0.51 0.50
ALL 0.84 0.62

For the case study in Italy, a more detailed analysis is done as the comparison assess the H16 performance over a large number of points (1806) through a consistent approach and the performance of Metop-A and Metop-B satellites are contrasted. Figure 4 shows the correlation maps between modelled and H16 product obtained through the two satellites. It is evident that the two satellite performs basically the same with median correlation values equal to 0.627 and 0.624 for Metop-A and Metop-B satellites, respectively. Moreover, the spatial pattern of correlation is the same with lower performance in correspondence of mountainous regions and of the main urban areas (e.g. Rome, Milan).

Figure 4: Maps of the temporal correlation between H16 and modelled soil moisture data in Italy for the period May 2013 – December 2014 by using Metop-A (left) and Metop-B (right) satellite data.

Figure 4: Maps of the temporal correlation between H16 and modelled soil moisture data in Italy for the period May 2013 – December 2014 by using Metop-A (left) and Metop-B (right) satellite data.

Summarizing, for H16 product (SM ASCAT-B O12.5 NRT) it is found that 88%, 38% and 1% of pixels overcomes the threshold, target and optimal requirements, respectively. Similarly to in situ data, the analysis by season is also performed and the results are shown in Figure 5. In good agreement with the previous analyses, also the seasonal performance of the two satellites are the same, and also well agrees with those obtained in the comparison with in situ observations. Specifically, the best performance for the H16 product are obtained in SON and DJF periods with median correlation values equal to 0.67 and 0.63, respectively.

Figure 5: Box plot of the correlation coefficient, CC, between H16 and modelled data in Italy (1806 points) by using Metop-A (left) and Metop-B (right) satellite data and subdivided by season (JJA: June-July-Aug, SON: September-October-November, DJF: December-January-February, MAM: March-April-May, ALL: whole period). The thresholds for CC are also shown.

Figure 5: Box plot of the correlation coefficient, CC, between H16 and modelled data in Italy (1806 points) by using Metop-A (left) and Metop-B (right) satellite data and subdivided by season (JJA: June-July-Aug, SON: September-October-November, DJF: December-January-February, MAM: March-April-May, ALL: whole period). The thresholds for CC are also shown.

Finally, the analysis is also performed through the application of the Soil Water Index (SWI) method [5] to the H16 product. The SWI method is used to obtain root-zone soil moisture information from surface measurements. This allows to match correctly the soil layer depth of satellite and modelled data. Specifically, a constant T-value, which is the unique parameter of the SWI method, equal to 4 days is adopted accordingly to previous studies (e.g., [4], [6]). Through the application of the SWI method, the median correlation values are better than the threshold requirement for all seasons, with median values for the whole period exceeding 0.74 (close to the optimal target).

Figure 6: As in Figure 5 for the Soil Water Index data (T=4 days).

Figure 6: As in Figure 5 for the Soil Water Index data (T=4 days).

Conclusions

In this report, the validation of the H16 soil moisture product through in situ, and modelled, data is carried out for the period May 2013 - December 2014. As mentioned above, in the comparison with in situ observations, the conclusions about the product accuracy should be drawn only considering the 136 sensors at 5 and 10 cm depth (Table 2 and Figure 2). On this basis, it is obtained that the H16 product overcomes the threshold accuracy requirements for 66% of the stations, 28% of the stations overcomes the target accuracy requirements and, 1% of the stations overcomes the optimal accuracy requirements (see Figure 2). The median Correlation Coefficient (CC) for the whole period is equal to 0.57 (Table 2). By excluding the in situ observations of Denmark network, characterized by lower accuracy, the median CC is equal to 0.60, quite close to the Target requirement. In the seasonal analysis, the higher values are obtained in the DJF and SON periods (CC=0.62) and the lower in the JJA period (CC=0.17). We underline that this comparison is carried out with a very large number of sensors and a long time series (20-month) thus allowing to obtain robust results. In the comparison with modelled data, better performance are obtained. For instance, in Italy it is found that 88% of pixels (99% of pixels if the SWI method is applies, see Figure 6) overcomes the threshold requirement, while in Belgium a median CC=0.84 is obtained (Table 3). Being the spatial scale of modelled and satellite data nearly the same, differently from in situ observations, the comparison with modelled data is probably more suitable for the evaluation of H16 product accuracy. Finally, as modelled and in situ observations are usually representative for a deeper soil layer with respect to satellite data, we suggest to apply the SWI method routinely in the validation procedure of H16, and all H-SAF satellite soil moisture products, to obtain more robust and consistent results.

Appendix

The Table A1, A2 and A3 can be downloaded here.

References

[1] P. Baguis, E. Roulin, P. Willems, and V. Ntegeka, “Climate change and hydrological extremes in Belgian catchments,” Hydrology and Earth System Sciences Discussions, vol. 7, no. 4, pp. 5033–5078, Jul. 2010.

[2] L. Brocca, F. Melone, and T. Moramarco, “On the estimation of antecedent wetness conditions in rainfall–runoff modelling,” Hydrological Processes, vol. 22, no. 5, pp. 629–642, Feb. 2008.

[3] L. Brocca, S. Camici, F. Melone, T. Moramarco, J. Martínez-Fernández, J.-F. Didon-Lescot, and R. Morbidelli, “Improving the representation of soil moisture by using a semi-analytical infiltration model: Improving Infiltration Modelling for Soil Moisture Simulation,” Hydrological Processes, vol. 28, no. 4, pp. 2103–2115, Feb. 2014.

[4] L. Brocca, S. Hasenauer, T. Lacava, F. Melone, T. Moramarco, W. Wagner, W. Dorigo, P. Matgen, J. Martinez-Fernandez, P. Llorens, J. Latron, C. Martin, and M. Bittelli, “Soil moisture estimation through ASCAT and AMSR-e sensors: An intercomparison and validation study across europe,” Remote Sensing of Environment, vol. 115, no. 12, pp. 3390–3408, Dec. 2011.

[5] W. Wagner, G. Lemoine, and H. Rott, “A method for estimating soil moisture from ERS scatterometer and soil data,” Remote Sens. Environ., vol. 70, no. 2, pp. 191–207, 1999.

[6] C. Albergel, C. Rüdiger, D. Carrer, J.-C. Calvet, N. Fritz, V. Naeimi, Z. Bartalis, and S. Hasenauer, “An evaluation of ASCAT surface soil moisture products with in-situ observations in Southwestern France,” Hydrology and Earth System Sciences, vol. 13, no. 2, pp. 115–124, Feb. 2009.