Detecting Marine Pollutants and Sea Surface Features with Deep Learning in Sentinel-2 Imagery

Despite the significant negative impact of marine pollution on the ecosystem and humans, its automated detection and tracking from the broadly available satellite data is still a major challenge. In particular, most research and development efforts focus on one specific pollutant implementing, in most cases, binary classification tasks, e.g., detect Plastics or no Plastics, or target a limited number of classes, such as Oil Spill, Look-alikes and Water. Moreover, most developed algorithms tend to operate successfully only locally, failing to scale and generalize adequately towards operational deployments. Our aim is to address these challenges by introducing a holistic approach towards marine pollutant detection using remote sensing. We argue that constructing such operational solutions requires detectors trained and tested against different types of pollutants, various sea surface features and water-related thematic classes. We offer such a Marine Debris and Oil Spill (MADOS) dataset, composed of high-resolution multispectral Sentinel-2 (S2) data, consisting of 174 scenes captured between 2015-2022, with approximately 1.5M annotated pixels, which are globally distributed and collected under various weather conditions. Moreover, we propose a novel Deep Learning (DL) framework named MariNeXt, based on recent state-of-the-art architectural advancements for semantic segmentation, which outperforms all baselines by at least 12% in F1 and mIoU metrics. The extensive quantitative and qualitative validation justifies our choices and demonstrates the high potential of the proposed approach. We further discuss the underlying discrimination challenges among the competing thematic classes. Our dataset, code and trained models are openly available to stimulate further research.