Skip to main content

Co-Designing for Deep Ocean Imaging and Analysis

Interview Synthesis, Fall 2020

Published onDec 18, 2020
Co-Designing for Deep Ocean Imaging and Analysis
·

Introduction

The Open Ocean Initiative is taking a highly collaborative, cross-disciplinary approach to tackle existing and future challenges in ocean exploration and storytelling through technological innovation, scientific discovery, and community connections. Our vision is to create a new paradigm for ocean exploration by employing collaborative, innovative research methods across a spectrum of cutting-edge tools, and making ocean discovery accessible to all communities around the world to make significantly more progress in understanding our planet than ever before.

This document synthesizes feedback on two concepts and projects initiated by the Open Ocean Initiative at the MIT Media Lab in the spring and summer of 2020, specifically:

  • Low-cost sensors & platforms for deep sea exploration and research. Open Ocean is in the process of designing and building a low-cost imaging system called Maka Niu, which will be available for users to test in fall/winter 2020. The engineering design and fabrication work is in collaboration with the MIT Future Oceans Lab.

  • AI-driven underwater video analysis. Open Ocean is doing initial research and development on an artificial intelligence-driven online platform for automated analysis of underwater video.

Seeking additional feedback from a broader global community on these projects, the Design Research Team interviewed 20 marine professionals around the world — 10 in the US and 10 located internationally (Figure 1). 

Figure 1. Ten interview participants were from the United States; ten represented Bermuda, Trinidad and Tobago, Portugal, Galapagos, Cook Islands, South Africa, Seychelles, and Sri Lanka.

Interviewees were members of the marine community including marine scientists, engineers, recreational divers, and educators. They represented a broad cross-section of society, taking into account race, gender identity, and national origin.

The interviews covered four main topics:

  • The interviewee’s background and interests

  • Low-cost deep sea imaging tools, with focus on capabilities of Maka Niu

  • AI-driven data analysis, with focus on capabilities of Ocean AI

  • Interest and availability for user testing of Maka Niu and/or Ocean AI

For marine professionals, Maka Niu offers several advantages to existing low-cost camera systems. Optional sensor modules will allow for users to collect environmental data alongside imagery. The open-source nature of the project enables repair work by users, as well as the development of additional sensor modules in the future by external collaborators. The easy integration between hardware and software enables data sharing and analysis. Lastly, as a co-design project, the Maka Niu community will provide opportunities for support and the sharing of best practices around deployment scenarios and local community involvement.

Key Quotes

This section introduces key participant quotes that highlight the need for low-cost platforms and an accessible data system, as well as design considerations for its successful deployment:

“The doors have been so closed on deep sea exploration...low-cost tech could get people excited about their home island.” 

— Dr. Jess Cramp, Executive Director of Sharks Pacific and National Geographic Explorer, Cook Islands

“People just don’t know what’s beneath the water surface. If they don’t know, they don’t love it, and they won’t protect it.” 

— Veta Wade, Founder & Director of Fish ‘N Fins, Monserrat

“People's imaginations stop at the shoreline.” 

— Nuno Lurenço, President of CoLAB +ATLANTIC, Portugal

“Here [in Sri Lanka] we don't have a conservation culture. We need to let people know there’s so much underneath...every coastline needs a hero...It's not like we have a shortage of data in general across the world. We just don’t have the manpower to work through.” 

— Dr. Asha de Vos, Founding Executive Director of Oceanswell, Sri Lanka

“It has to be low-cost...your cost-effective is different than my cost-effective.”

— Zoleka Filander, offshore ecologist with the Department of Environment, Forestry, and Fisheries, South Africa

“We should offer citizen science opportunities so communities can be part of the solution...we want to teach students to be observers and ask the right questions. We want younger learners to become change agents in their communities.” 

— Chris Blake, teacher & voyager with Kamehameha Schools and Polynesian Voyaging Society, Hawai’i

“There is a disconnect between what scientists study and what policy makers need to know...This could be a real game changer for deep sea science...This will revolutionize things. And put me out of a job!” 

— Dr. Diva Amon, deep-sea biologist and 2020 National Geographic Emerging Explorer, Trinidad & Tobago

“The single most important thing we could do as a community is get our act together. We’ve explored a ton of the deep sea but we can’t even determine biogeographic provinces. Bringing together deep sea data would be such an explosion of deep sea science. It’s a tractable problem if we just do it.”

— Dr. Craig McClain, deep-sea biologist and Executive Director of the Louisiana Universities Marine Consortium, Louisiana

Participant Archetypes

Although all of our interview participants are marine professionals, they represent a broad cross-section of the marine community, and each has different primary motivations and requirements. This section outlines motivations and requirements for each participant archetype in order to guide future ideation and decision-making processes. Our aggregate data pointed to six major participant archetypes.

Note: Many participants represent multiple archetypes. Archetypes are listed alphabetically.

Developing Nation / Community

  • Motivations: Enabling locally led science, not just parachute science. Sharing with people what’s in the ocean around them so that they will be motivated to conserve and protect it. Engaging populations not usually engaged in scientific research (e.g. fishing, tourism, etc). Prepare people for marine jobs.

  • Requirements: Low-cost, easy-to-use and deploy, and error-proof. Doesn’t require a big boat or internet access. Ability to deal with maintenance and repair issues locally. Potentially additional language support for software.

Education / Outreach

  • Motivations: Broadening access to tools, skills, and knowledge. Sharing with people what’s in the ocean around them so that they will be motivated to conserve and protect it. Preparing people for marine jobs. Making learning engaging. 

  • Requirements: Easy-to-use, error-proof, low-cost. Integrated with software, works on multiple mobile platforms, and with a variety of wifi conditions.

Engineering

  • Motivations: Having a long-term presence in the ocean. Minimizing potential losses. 

  • Requirements: Open-source, easily accessible parts. Long-duration hardware. Coordination of multiple assets. Low-cost hardware for testing.

Policy & Management

  • Motivations: Having more information for better-informed management and policy decisions. Being able to collect and analyze their own data versus relying on outside countries and companies (particularly countries/communities that don’t have deep sea assets or expertise).

  • Requirements: Data quality, data ownership, and data accessibility/comprehensibility.

Scientific Research

  • Motivations: Getting more eyes on the seafloor, and more data everywhere, especially in deep water. Ensuring data quality and making analysis easier. Being able to make more global conclusions vs hyper-localized ones. Collaborating with other researchers. 

  • Requirements: Easy-to-use and deploy without needing a research vessel. Standardization of data, accurate AI tools. Lower-cost than existing tools. More specific toolsets (e.g. additional modules). Ability to reach deeper depths.

Traditional Ecological Knowledge

  • Motivations: Recognition of marine traditional knowledge. Protection of culturally significant regions. Connections between traditional knowledge, cultural values, and scientific research. Storytelling to honor heritage and connections to the marine environment.

  • Requirements: Science should be driven by traditional knowledge and local communities. Opportunities for students and local communities to be involved and leading efforts. Tools should be easy to use and low-cost.

Hardware Design Considerations

Our interviews pointed to a variety of considerations for hardware design, including sensor recommendations, feature requests, and deployment scenarios. The priority levels reflect the relative consensus amongst participants about the need for these capabilities.

Sensor Recommendations

  • 1st Priority Sensor Recommendations: Temperature, Depth, Salinity, Imaging

  • 2nd Priority Sensor Recommendations: GPS, Oxygen, pH, Acoustic tags, Light attenuation, eDNA

  • Imaging Recommendations: High definition, Stereo, 360

Various types of water quality indicators were also recommended, but less consistently than the above measurements, including chlorophyll, methane, nitrates, phosphates, alkalinity, and turbidity.

Feature Requests

  • 1st Priority Feature: Easy to access video/database

  • 2nd Priority Features: Long duration, quick access to video/live stream

  • Additional Features: Easy to use and fix, modular, programmable missions

Deployment Scenarios

Suggested deployment scenarios, in order of participant capacity and interest:

  1. On a deployed benthic structure (lander, elevator, BRUV)

  2. Deploy from small boat (ferry, kayak, fishing vessel, vaka)

  3. On a fixed structure (buoys, moorings, FAD)

  4. By people (SCUBA divers, snorkelers)

  5. On a tethered system (ROV, fishing line)

These deployment scenarios are not mutually exclusive; for example, one might deploy a lander from a small boat. Several additional features of a deployable system include: baited; deploy/retrieve quickly and easily; drifting system; no anchor; potential for array to do swath of seafloor simultaneously.

Data Analysis Design Considerations

The Ocean AI experience must be designed with different user categories in mind. Level of data access and user experience must vary for different categories of users. The interface used by children in the classroom should look and function differently than the interface for marine researchers. The entire toolset must also be developed with the future goal of operation on mobile devices, as well as offline and real-time capabilities.

The value of Ocean AI is entirely dependent on accuracy, therefore higher accuracy identification at higher taxonomic levels is of greater importance than lower accuracy identification at the genus/species level. This focus on higher level identification is also beneficial to the broader marine community. Compatibility with existing databases and collaboration with existing identification software may expedite the development of Ocean AI and prevent the creation of yet another data silo. Access, storage, and managing data flow were also common concerns among interviewees. 

Key takeaways:

  • Assume no computer programming background.

  • Need to make software easy to use, simple.

  • Concerns: data management, access, storage, ownership, policy.

  • Participants have a desire to combine different data sets (video + environmental).

  • Higher accuracy at higher taxonomic level is better.

  • Consider different user groups, what their experiences are, and what their level of access to data could/should be, including youth, teachers, researchers, and policy makers.

Community Involvement

The ocean community includes a wide range of groups, including researchers, engineers, policy makers, coastal communities, NGOs, students, fishers, tourists, and offshore industries. Interviewees explored how each of these groups can play a role in ocean exploration using Maka Niu and Ocean AI. Participants were enthusiastic about brainstorming opportunities for community involvement in ocean exploration if the support and the tools are made available. 

Fishing

Fishers were mentioned as key stakeholders in ocean management by 14 of our 19 interviewees. Building relationships with the local fishing industry allows for an expansion of research capabilities. Using Maka Niu and Ocean AI, fishers can connect with their marine ecosystem and contribute to continued ecosystem monitoring. Participants suggested that giving fishers access to data collection methods would allow them to contextualize and value scientific research.

Offshore Industry

It was noted that unlike fishing, other offshore industries like oil and gas tend to be disconnected from the marine environment around them. Maka Niu provides an opportunity to share deep sea ecology with this community and encourage future partnerships.

Tourism

A few participants have direct relationships with local tourism. Inviting tourists to form deeper connections with the local ecology and researchers through environmental monitoring and exploration would enhance their experiences and inform them on their impacts while traveling. If partnering with tourism companies that operate in the same regions over long periods of time, this would be an opportunity to increase our understanding of regional changes over time. 

Policy Makers

It was noted that data usage may differ between scientists and decision-makers. The versatility of our system should allow data collection and use to be conducted through the lens of different sectors including management and policy making. Our systems can bridge the gap by using Maka Niu and Ocean AI to illustrate how deep sea ecosystems work, how data can be used to inform management, and how they are in turn impacted by policies. 

Local Communities

Coastal communities themselves were noted as vital assets for marine research. Suggestions for engagement included local cultural centers like village gathering places and museums. Participants proposed strengthening relationships between the local community and their underwater ecosystems to encourage sustainability and marine management. Each community has unique priorities that are driven by its cultural heritage. Accessible, low-cost oceanographic tools provide an educational platform that can empower communities to invest in long term ecological monitoring, as well as learning opportunities for local people to develop their scientific skills and lead their own projects. Participants were also excited by the possibility to build multi-generational community connections around ocean exploration. 

Students and Classrooms

All participants suggested opportunities for students ranging from K-12 through college to use Maka Niu and Ocean AI to experience the ocean, learn about marine life, and contribute to a global knowledge base. Younger learners would be able to observe life in the ocean using annotated videos to learn the important species in their area. Middle and High school students would be able to deploy a camera system, collect their own data, and begin to contribute to scientific research, and learn valuable technical and scientific skills. Classroom dialogue and partnerships with local college students can identify regional questions which can be explored using these new tools.

Aquatics and Recreation

Our aquatics and recreation interviewees addressed the role of ocean exploration in inspiring youth to consider future marine careers. Using oceanographic tools that mirror those used by researchers allows youth to contribute to scientific knowledge while building interest and enthusiasm for careers in science.

NGOs

Several interviewees lead or are strongly connected via their work to local NGOs. NGOs offer structured organizations to connect with an important community of volunteers. Interviewees noted these volunteers will be excited to participate and deploy cameras under the supervision of the NGOs and contribute to the gathering of scientific knowledge.

Discussion

Our interviews revealed a number of open design and implementation questions, highlighting participant concerns and potential trade-offs. Addressing these questions will be necessary to ensure that these systems have impact where intended. Six major categories result from the data:

Deployment

How can we support people to come up with plans for deployment? 
Some participants have a lot of experience in and around the water, but not in deploying tools in the deep ocean. Options—including both physical hardware and training—for deployment should be developed and shared with users. Duration of deployment was a question that came up a lot, as well as stability of the system in high-current or otherwise difficult environmental conditions. The community around Maka Niu may be able to help provide support and best practices for deployment-related challenges.

Software

How can we make the software (both camera mission programming and Ocean AI) easy-to-use but also robust?
We repeatedly heard the need for mobile-friendly, accessible, and simple software solutions. The software also needs to be stable and easy-to-use in classroom environments. UI/UX design is a major area for future research and design efforts. Low/no internet access must be taken into consideration.

Environmental Sensors

What sensor modules should be prioritized for future development?
The environmental sensor modules are a value-add for educators and researchers alike. While our research suggests which sensors may be most useful to work on in the immediate future, there is less consensus about prioritization of future modules.

Data Sharing

How do we balance the desire to share data with concerns about exploitation?
Our participants indicated concern about exploitation (e.g. letting people know what kind of resources their community has). There were concerns about how the data sharing would work, how quality would be ensured, and how participants would know how and by whom the data would be used. Some participants had concerns about copyright.

Funding

How can we support local researchers and collaborators to take on the work?
Our interviews pointed to the need for financial support to make the use of these systems possible by people around the world. In order for Maka Niu and Ocean AI to have the biggest impact, we will need to determine how to value and support local researchers and collaborators to take on the work.

Storytelling

How can storytelling be integrated into the use and deployment of these tools?
Information from research programs shared with the public is too often limited to the final output of an entire process of scientific and tool development, data gathering and analyzing. Scientists don’t have many opportunities to share why they became scientists and what their journey during this process was. By documenting and filming the evolution of the development of the Maka Niu/Ocean AI system and its use on the ground by the different participants, we want to use storytelling to catalyze mutual understanding among the different communities and scientists, narrating a long process of learning that allows to place the gathered information in its context and render it more concrete and impactful for everyone.  

***Visualization tools to display contribution and covered areas (e.g., geographic regions, depth, taxonomic group) by individuals and communities could also encourage participation, as well as facilitate identification of areas that may require more attention.

Acknowledgements

Many thanks to our interview participants and engineering and design teams for their time and expertise. 

Design Research Team

  • Dr. Jennifer Chow, MIT Media Lab

  • Alexis Hope, MIT Media Lab

  • Kat Cantner, MIT Media Lab

  • Dr. Maud Quinzin, MIT Media Lab

  • Dr. Katy Croff Bell, MIT Media Lab

Maka Niu Team

  • Dr. Dan Novy, MIT Media Lab

  • Dr. Allan Adams, MIT Future Oceans Lab

  • Lui Kawasumi, MIT Media Lab

  • Margaret Sullivan, MIT

  • Peter Bell, MIT Media Lab

  • Dr. Katy Croff Bell, MIT Media Lab

Ocean AI Team

  • Susan Poulton, Door 44 Digital

  • Oceane Boulais, MIT Media Lab

  • Jon Brendsel, QCatalyst

  • Bogdan Nechita, QCatalyst

  • Benjamin Woodward, CVision AI

  • Dr. Katy Croff Bell, MIT Media Lab

Comments
0
comment

No comments here

Why not start the discussion?