A New Vision for Trash:
Using Remote Sensing to Address Trash Monitoring Needs
Tony Hale, PhD, San Francisco Estuary Institute
Shelly Moore, Southern California Coastal Water Research Project
Look hard enough and you will find trash. In many ways, trash is ubiquitous, forming elements in our environment almost as common as the natural vegetation, rocks, and sediment that the trash can often obscure. Plastics, metal, and paper waste are readily recognizable by everyone from the smallest child to the oldest adult, yet we still face routine challenges in locating and quantifying trash in areas that are likely to impact water quality. When it comes to assessing trash loads, transport pathways, and impacts, California and its many regions are unable to address some essential questions.
To focus our vision on trash, the Southern California Coastal Water Research Project (SCCWRP) and the San Francisco Estuary Institute (SFEI) are co-leading a project funded by the California Ocean Protection Council and supported by the State Water Board. The project seeks to provide greater measures of precision and accuracy for trash monitoring methods so that they might better address pressing management questions.
The impulse for this renewed focus on trash is not driven merely by scientific interests, water quality concerns, or management rigor. Rather, there are also new legal requirements. Recent policies put in place by state and local governments in California will soon necessitate more robust methods for measuring the amounts of trash in various habitats to both determine how much is in the environment and how effective the policies have been at limiting or reducing trash. In 2015, the California State Water Resources Control Board passed a regulation (the “Trash Amendments”) prohibiting the discharge of trash from stormwater systems into streams, rivers, lakes, and the ocean. This regulation mandates municipalities to either install full trash capture devices within their storm drain systems to limit trash making its way into receiving waters, or to establish an equivalent monitoring program.
The project at hand seeks to validate three existing monitoring methods used across the state, while also developing a new protocol to determine how technology innovations might be brought to bear on the problems. The intended result in 2020 will be a standardized set of tools to be used by all levels of monitoring practitioners from city staffers to scientists to community clean-up volunteers.
For this article, however, we will concentrate on some of the challenges and opportunities addressed by a novel monitoring method which pushes the boundaries of what is possible in a geospatial context. Essentially, to expand our vision into what is happening on the ground, we head into the air. More specifically, we deploy unmanned aerial systems (UAS) in the field to expand the footprint and temporal density of what might be otherwise achievable using larger, on-the-ground teams. (Such conventional surveys are depicted in figure 1.) This leads us to several questions: As our drones relay to us their images, can we see trash from the sky at all? Considering how dense the tree canopy can be around Bay Area creeks and rivers, is trash hidden from view? If we can see the trash, then what? How do the results from the aerial surveys compare to on-the-ground assessments?
We try to address these questions through practical application of technology. We leverage the opportunity of simultaneous assessments to compare results of the aerial surveys to the conventional trash collection results, thereby gaining some measure of certainty for our aerial method. Without the ground surveys, after all, we would have no measure of what we might call “truth.”
In the course of our surveys, we were able to identify trash items, some with great certainty. Figure 2, for instance, represents a single photograph from approximately 100 ft in the air. From that distance, we can achieve ½-cm resolution imagery, allowing us to identify coolers, boxes, cans, and sundry items deposited at the edge of a dry creek bed.
These initial results from the drone flights are highly encouraging, since they might allow surveys to be conducted remotely rather than through intimate contact with unknown or perhaps hazardous materials. We quickly discovered limitations, however. Figure 3 reveals refuse entangled in leaves and branches under a heavy canopy. We can identify cups, wrappers, and other items likely composed of plastic, but it is difficult to determine the precise nature of the items while they are entangled in the natural leaf litter. Moreover, the broad riparian tree canopy obscures part of the relatively narrow waterway, making a comprehensive assessment rather challenging or impossible from the sky.
Neighboring assessments in the same vicinity offered unimpeded views from the sky. In figure 4, you can see the same park as that in figure 3. However, in figure 4, the trees are either beyond the waterway or defoliated, revealing a view of a shiny sheet of metal in the water, some small packaging, and a plastic bag. Even submerged materials can be deciphered from the derived image.
When evaluating these early results, we learned that there are several factors to consider.
● Flight altitude vs resolution: Generally speaking, the greater your altitude, the lower your resolution. On the other hand, the lower you fly, the more imagery you must assemble to cover the same geography as a higher flight. We learned that 100 ft is the optimum flying altitude. We only fly higher when we must cover vaster swaths of land, at the sacrifice of resolution.
● Urban obstacles: Overhead lines, bridges, poles, even trees can prove to be hazards to an unmanned aerial vehicle. For some planned locations, the hazards were too numerous to attempt a flight.
● Restricted airspace: With three major airports ringing the Bay Area, there are large no-fly zones where drone flights are not permitted. Unfortunately, airports, being at the edges of the watersheds, are often where trash is most likely to collect before spilling into the Bay. Moreover, airports are often situated in exposed landscapes without much riparian canopy. Therefore, these are presently missed opportunity areas, but regulations – local, state, and federal – are quickly evolving regarding the rules governing drone flights, so we are keeping apprised of developments.
Despite these challenges, the team has performed almost 30 surveys in Southern and Northern California, 20 of the surveys accompanied by drone flights. (With another monitoring season to go, this represents the halfway point in the project.) Thus far, each flight collected between 36 and 200 individual images over the span of each assessment area at least 300 ft in length with variable width, depending on obstacles and constraints in the built environment. Figure 5, showing the SFEI team in the field, captures a drone flight in action with a licensed pilot and a spotter who looks for obstacles. Using 3DR’s Site Scan software, the team assembled the captured images into an orthomosaic that can be used to represent the assessment area in its entirety. Figure 6 illustrates the user interface for the tool, depicting hundreds of individual images, ortho-rectified and stitched together into a seamless mosaic which then lends itself to further analysis.
You might reasonably ask, if it requires time in the field to assess creeks, rivers, and wetlands using conventional methods, then aren’t we simply extending the assessment time by conducting the counting and measuring in the office? In other words, all we are doing is bringing indoors the sometimes-trashy outdoors. This is where the next phase of our analysis comes into play to drastically reduce analysis effort.
Using a combination of the orthomosaic and the individual images, the team is developing new machine learning algorithms to detect trash. A convolutional neural network (CNN) called TensorFlow is an open-source solution broadly used by Amazon Web Services, Google, Microsoft, and countless other organizations to examine images and recognize features. For instance, the facial recognition used by Facebook to find your Aunt Mabel in the pictures from your holiday party is one example of a visual machine learning algorithm. In our case, we are interested in disambiguating the rocks along a stream from the expanded polystyrene cooler nestled between them. This is not an altogether easy job, but early results are promising.
Our application developer is hard at work annotating images, indicating where he sees trash. These annotated images are used as the basis for “training” the algorithm to recognize similar features in similar environments. The general rule with these projects is that the more reliable training material you can collect (annotated images), the better your machine learning algorithm will perform in recognizing the target objects.
Meanwhile, we are also partnering with others to collect imagery that can specially target plastics. Whereas we are using a true-color sensor that yields imagery analogous to the way the human eye would perceive it, our colleagues at San Francisco State are mounting a MicaSense sensor onto their drone to determine whether its near-infrared spectral band can detect plastics as distinct from other forms of trash. This could offer a form of “special vision” to identify and quantify the amount of plastic in an assessment area, which might otherwise be very challenging to calculate via either on-the-ground or remote methods when assessing very trashy areas.
And plastics, after all, are often subject to special scrutiny with regard to trash monitoring, due to their persistence in the environment, their tendency to break down into ever smaller pieces to form microplastic, and their potential impact on humans and wildlife once entering the food chain.
Our team looks forward to offering a library of standard methodologies to promote comparability of trash levels both within and among regions across California. Many methodologies that provide detailed information are deemed cost-restrictive because of the needed resources in both staff time and equipment. In addition, many of these methods do not focus on answering the important management questions relative to determining trash policy effectiveness. With some of the work that the project team is performing, we hope to help close some of these knowledge gaps and address the barriers toward greater standardization.
The project will be completed in 2020, but in the meantime, we are offering regular updates via a newsletter and website. You can reach all the information you need by visiting trashmonitoring.org where you can sign up for the newsletter and learn about opportunities to connect with the project team.
Fall 2018 Volume 11 Issue 2