DeepLitterAI: An AI System for Automatically Detecting and Counting Plastic Litter on the Deep Seafloor Faster detection and counting of plastic litter in large-scale video surveys to support global monitoring of marine pollution

1. Key Points

• A research team at the Japan Agency for Marine-Earth Science and Technology (JAMSTEC) has developed DeepLitterAI, an AI system that automatically detects and counts macrolitter ^※1, including plastic litter, on the deep seafloor from video footage. Global monitoring is essential to support international efforts to reduce marine plastic litter, but the many hours required for conventional visual analysis have been a major obstacle. DeepLitterAI offers a practical way to overcome this challenge.
• Developing a robust AI system requires a large and diverse set of training images. Previous AI models in this field were trained mainly on images in which litter appeared large and clearly visible. As a result, they tended to miss the small litter items commonly seen in actual survey footage. To address this problem, the research team built J-Litter, a new dataset of more than 12,000 images. It includes images in which litter appears small and distant, as well as seafloor rocks, organisms, and other features that can easily be mistaken for litter. Training the AI with this dataset substantially improved its ability to detect litter under realistic survey conditions.
• Tests using video footage from actual deep-sea surveys showed that DeepLitterAI could automatically count litter with accuracy comparable to that of experienced researchers. The system also reduced analysis time to less than half that required for manual visual analysis, processing the footage 2.1 times faster on average and up to 3.1 times faster.

Graphical Abstract

2. Overview

A research team led by Ryota Nakajima of the Japan Agency for Marine-Earth Science and Technology (JAMSTEC; President: Tomohiko Kawamura) has developed a system that uses AI to automatically detect, classify, and count macrolitter in deep-sea survey footage. To develop the system, the team extracted more than 12,000 original images containing seafloor litter from deep-sea video footage collected by JAMSTEC over more than 40 years and built a new image dataset, J-Litter ^※2. The dataset was then used to train a new AI system called DeepLitterAI.

The deep sea, which accounts for most of the ocean, is considered a major final destination, or “sink”, for litter entering the marine environment, including plastics. However, accurately assessing the amount of litter on the deep seafloor has traditionally required researchers to spend many hours visually examining long video recordings collected by ROVs and other survey platforms. This process requires an enormous amount of time and effort.

DeepLitterAI combines advanced object-detection technology trained on the newly developed J-Litter dataset with object-tracking technology that follows the same item through consecutive video frames and prevents it from being counted more than once. In particular, by training the AI on large numbers of images containing small litter items and complex seafloor backgrounds, the system can detect litter with fewer missed detections even in wide-field footage collected during actual deep-sea surveys.

The system was tested using survey footage collected at depths of 860 to 5,600 m in waters around Japan. The number of litter items counted by the AI showed a high level of agreement with counts made by experts through visual inspection, with a difference of approximately 10%. DeepLitterAI was also able to complete the analysis in less time than the actual playback duration of the footage. Human observers, by contrast, often need to pause and replay video while identifying and counting litter. The AI therefore substantially reduces the time required for analysis.

By enabling large volumes of video data to be analyzed rapidly and objectively, this technology can enable the rapid mapping of litter distribution across deep seafloors worldwide. DeepLitterAI is therefore expected to become an important tool for supporting international efforts to reduce plastic pollution.

These findings were published online as a corrected proof in Environmental Pollution on September 3rd.
This study was partially supported by the Ministry of the Environment, Japan (MOEJ), under Project No. SII-10-1(2)-JPMEERF23S21002. This work was also partially funded by the Moonshot Research and Development Program, Research and Development of Marine Biodegradable Plastics with Degradation Initiation Switch Function (JPNP18016), commissioned by the New Energy and Industrial Technology Development Organization (NEDO).

3. Background

Each year, enormous amounts of plastic waste end up in the ocean worldwide as a result of improper waste management and other factors, raising serious concerns about their impacts on marine ecosystems. In response to this global problem, negotiations are currently underway at the United Nations toward the development of an international legally binding instrument on plastic pollution, including in the marine environment^※3.

Much of the litter that enters the ocean, including plastic litter, is believed to eventually sink to the deep seafloor, where it may remain for long periods and continue to have adverse effects on the marine ecosystems. When countries implement measures to reduce plastic pollution, extensive monitoring of the seafloor will be essential to determine whether the amount of plastic litter in the ocean is actually decreasing.

Survey methods that involve dragging nets across the seafloor can damage marine ecosystems. For this reason, non-destructive surveys using underwater cameras have increasingly been recommended. However, manually reviewing the enormous amount of video footage collected during such surveys and counting individual pieces of litter requires considerable time and labor. This has been a major obstacle to conducting surveys on a large scale.

Previous studies have explored the use of AI for automated litter detection. However, conventional AI models were trained mainly on images in which litter appeared large and clear. As a result, they had difficulty detecting the small litter items commonly seen in actual wide-area survey footage. In addition, conventional AI systems were generally limited to detecting litter in individual images. A reliable method had not yet been established for tracking the same item through continuous video without counting it more than once, making it difficult to accurately determine the total number of litter items on the seafloor.

4. Results

To overcome these challenges, the research team developed DeepLitterAI, an AI system designed specifically for practical use in real-world deep-sea surveys. The study produced two major advances.

(1) Improved detection of small litter against complex seafloor backgrounds

The research team extracted images containing seafloor litter and related objects from deep-sea footage collected by JAMSTEC over more than 40 years and built a new dataset called J-Litter, consisting of 12,029 images. The dataset contains a large number of images showing small litter items occupying less than 10% of the image area, as well as background images containing marine organisms, seafloor features, and other objects that could easily be mistaken for litter. J-Litter is publicly available through the JAMSTEC website. Training the AI on this dataset substantially improved its ability to detect plastic bags and other film-like plastics, plastic bottles, beverage cans, and other litter. Its detection performance was approximately 1.6 times higher than that of conventional AI models evaluated in the study (Figure 1).

Figure 1. Examples of Seafloor Litter Detected by DeepLitterAI
DeepLitterAI accurately detects not only large and clearly visible litter items (left column: a, plastic bag; b, plastic bottle; c, beverage can) but also distant, small litter items commonly seen in actual wide-area survey footage (right column: d, plastic bag; e, plastic bottle; f, beverage can). Detecting objects are shown with bounding boxes and their associated detection probabilities.

(2) Accurate counting through video tracking and a major reduction in analysis time

The second advance is the system’s ability to accurately count litter by tracking objects through video, while substantially reducing the time required for analysis. The research team introduced an object-tracking algorithm that follows individual litter items through consecutive video frames. This prevents the same item from being counted more than once as the camera moves. The performance of DeepLitterAI was compared with manual visual analysis using actual survey footage collected at depths ranging from 860 to 5,600 m. The AI counted litter with accuracy close to that of expert researchers, with a difference of approximately 10% between the AI and expert counts. The system also substantially reduced analysis time. Human observers often need to pause and replay footage while checking and counting individual items. DeepLitterAI, in contrast, completed the analysis 2.1 times faster on average and up to 3.1 times faster than manual visual analysis. This substantially reduced the workload required to analyze footage, particularly in areas where litter is highly concentrated (Figure 2).

Figure 2. Comparison of the time required for seafloor litter analysis by humans and DeepLitterAI
The gray bars show the time required to analyze seafloor litter in survey footage collected at various locations around Japan at depths ranging from 860 to 5,600 m. From left to right, the bars show the average playback time of the survey footage at normal speed, the average time required to count litter manually through visual inspection, and the average time required for DeepLitterAI to count litter. Error bars indicate the standard deviation. The results demonstrate that AI-based analysis can substantially reduce processing time compared with the lengthy manual visual analysis traditionally performed by researchers.

5. Future Perspectives

The development of DeepLitterAI makes it possible to automatically search the enormous volume of deep-sea survey footage accumulated around the world for seafloor litter and to create high-accuracy maps of its distribution. For example, manually analyzing 100 hours of video footage could previously require nearly one month of work. With DeepLitterAI, the same amount of footage could be analyzed in only a few days. In the future, installing the AI system directly on underwater vehicles could also enable litter to be detected and counted in real time. This would allow researchers to rapidly identify hotspots where large amounts of litter are concentrated and enable more flexible and efficient monitoring of the marine environment.

Wider use of this technology could also make it possible to efficiently assess whether environmental protection measures implemented under the international plastics treaty, which is currently under discussion, are actually producing measurable results. Data obtained using DeepLitterAI could provide objective evidence of whether efforts to reduce plastic pollution are contributing to the protection of deep-sea environments. Such data would also provide important scientific knowledge for further promoting public understanding of and participation in efforts to address environmental issues.

The current version of DeepLitterAI has been developed mainly using data collected in waters around Japan. The research team therefore plans to collaborate with research institutions around the world and incorporate images of seafloor litter from different ocean regions into its training data, with the aim of developing a successor dataset to J-Litter. Through these efforts, the team aims to develop a robust, general-purpose AI model that can operate accurately and consistently across a wide range of deep-sea environments worldwide. This will contribute to more comprehensive and effective monitoring of the global marine environment.