Round Robin

The European Space Agency and the WorldWater project with support from CNES, NASA, EARSC, CEOS and GEO invite interested parties to express their interest in participating in a Round Robin inter-comparison of inland surface water detection and monitoring algorithms using Sentinel-1, Sentinel-2 and Landsat 8 imagery.

The WorldWater project is organising a Round Robin exercise aiming at the inter-comparison of Earth Observation (EO) algorithms for surface water detection, using the latest generation of free and open satellite data from Sentinel-1, Sentinel-2 and Landsat 8. The Round Robin is organised under the auspice of the European Space Agency and being supported by a number of international organisations and initiatives including CNES, NASA, the European Association of Remote Sensing Companies (EARSC), the CEOS Ad hoc team on Sustainable Development Goals (CEOS SDG AHT) as well as GEO and their Earth Observations for the Sustainable Development Goals (EO4SDG) initiative.

The objective of the Round Robin is to enable a better understanding of the pros and cons of EO approaches for mapping and monitoring the extent of inland open waters, comparing the robustness of different algorithms and identifying shortfalls and areas of further research. The Round Robin will also contribute to the development of EO best-practices for the monitoring of SDG indicator 6.6.1 on water-related ecosystems.

The Round Robin is open to researchers, companies, and other developers of satellite-based algorithms for surface water detection.

The precondition for participating in the Round Robin is to have a peer-reviewed algorithm for surface water detection based on (or adaptable to) Sentinel-1, Sentinel-2 and/or Landsat 8. Non-peer reviewed algorithms can be considered if adequate supplementary documentation and justification can be provided.  

Interested participants will be asked to produce monthly maps of surface water presence at 10-meter spatial resolution in 5 different locations for 2 consecutive years.  The input data sets, which will be made available to all participants, will include 2 years of multi-temporal data from Sentinel 1, Sentinel 2 and Landsat 8 acquired over five globally distributed test sites (100×100 km) located in 5 different countries: Colombia, Gabon, Greenland, Mexico and Zambia. All satellite input data will be made available in both raw and Analysis Ready Data (ARD) format.

The organizers will prepare and facilitate access to an online working environment for uploading and execution of algorithms in the cloud. Users with access to their own platforms or cloud infrastructure (e.g. data cubes, Google Earth Engine etc.) can also participate on the condition that they accept to use the same input data sets from Sentinel-1, Sentinel-2 and Landsat 8. Use of ancillary datasets (such as DEMs and a priori surface water maps) are allowed, but under condition they are free and open datasets (e.g.  Copernicus DEM).

The outputs generated by the round robin participants across all 5 test sites will be evaluated individually and in cross-comparison using a harmonized independent reference data set. The reference data and algorithm evaluations will be prepared and done by a WorldWater independent validation team in a fully transparent process where all participants will gain access to the reference data.

The benefits of participation in the Round Robin are multiple:

• to be a co-author of a peer-reviewed paper summarizing the outcomes of the Round Robin exercise.
• to cross-compare the performance of applied algorithms against other leading algorithms and identify strengths and weaknesses.
• to contribute to the production of best-practice EO based methods to monitor surface water dynamics.

The call for expression of interest was open until 19^th of February 2021. 

Expressions of interest will be evaluated in terms of algorithm maturity, ability to execute (cf. transferability and scalability of algorithm) and willingness to share outputs and documentation. All accepted participants will be invited to join an online session on the Round Robin setup and guidelines. The session will also include a Q&A session to give participants the opportunity to clarify open issues.

The Round Robin execution is expected to last two months starting from 1^st of March 2021 (note: a contact point will be available for assistance during the entire period). 

The organisers: 
Christian Tøttrup (DHI GRAS, WorldWater), Mads Christensen (DHI GRAS, WorldWater), Marc Paganini (ESA, CEOS SDG AHT), Santiago Pena Luque (CNES, CEOS SDG AHT), Monica Miguel-lago (EARSC), Andreia Siqueira (GA, CEOS SDG AHT), Argie Kavvada (NASA, GEO EO4SDG) and Douglas Cripe (GEO secretariat).

Registrations are now closed.