Boat to Plate

SIEF supports technologies improving traceability in food supply chains

Background

Globally the seafood industry provides approximately 171 million tons of product from thousands of species, with about 91 million tons from wild-capture production. Australia’s low-production, high-value wild-capture fisheries are some of the most sustainably managed in the world, providing seafood to domestic and international markets, worth $2.5 billion per year.  Seafood supply chains however face uncertainty. Lack of traceability from the point of capture on vessels is seen as one of the key challenges.

The objectives of this project have been to develop prototype solutions for vessel-level data capture using onboard cameras, video analytics for automated species identification, counting, size estimation, colour measurement and tagging of catch. The project aims to i) provide trusted information of fish origin at the root of the seafood supply chain, ii) potentially increase economic productivity, and iii) provide assurance to consumers of Australian seafood quality and provenance. The work aligns with CSIRO’s Food Security and Quality challenge and is highly relevant to the thematic context of connecting the physical and digital world.

The Science and Industry Endowment Fund has extended support for development and translation of this work through a funding of 450K over a period of eight months. The Boat to Plate project team states SIEF’s support played an instrumental role in driving the transition from R&D to uptake and potential commercialisation of this work.

The challenge

Traceability is a critical challenge for the industry. It is important for industry to know fish quality is maintained throughout the supply chain and that fish are not substituted for poorer quality product. There is also an increasing expectation from consumers that fish are caught sustainably. To this end, imagery from point of capture is increasingly being used to manage Australian fisheries, determine sustainable fishing levels, and is seen as basis for the supply chain management. Similar to face recognition, however there are several research challenges including species that look similar and are difficult, even for humans, to identify such as yellowfin tuna and juvenile bigeye tuna. Also, vessel operators would like an ability to label, count and measure fish, and integrate these data into an on-line data management systems to efficiently provide vessel crew, skippers, and fleet managers real-time or near-real time information on the state of the catch providing better information for fish markets and assurance of a quality product on sale.

The response

There are currently no available commercial solutions to address this challenge. With the emerging deep video analytics, the Boat to Plate team is developing novel solutions for automated video analysis by leveraging capabilities in computer vision, machine learning, catch species identification, system integration and connections with the Australian fishing industry and fisheries management authorities.

The team has developed innovative solutions in the form of a software tool and tagging system. The product offers a convenient, cost and time effective solution for seafood producers to monitor and manage their product throughout the supply chain. The team’s initial machine learning product called SNAPPER automatically locates fish from on-vessel footage. Another more advanced system, called WANDA, identifies species, size and colour using image analysis techniques. A third product, FLIPPER, allows identification and counting to be achieved in real, or near-real, time. A tagging system tags each fish box, which can be used to scan and retrieve catch information. The tag, or a connected tag further down the supply chain, can also be used for end consumers to access catch information.This provides trusted information of fish origin at the root of the seafood supply chain and lays the ground for developing a prototype solution for other varieties of seafood.

Outputs

• Deep learning tools for fish detection and species identification (SNAPPER, WANDA and FLIPPER)
• Image analysis tools for fish size and colour measurement
• Tagging system for tracking seafood product
• A mock-up system that has been designed and tested in the lab. It integrates NFC tag readers, a trained machine learning model to identify and recognise caught fish, a database to record events, and a web based graphical user interface. The system has been tested with market bought fish, and will be tested on an operational fishing vessel when it becomes available. Several companies have been impressed with the mock up system.
• In the toothfish project, the uptake of technology will both allow the skipper to better concentrate on important safety and operational considerations during the haul, and the 100% human observer coverage on board to be retasked for more pressing research issues over and above simply counting and weighing fish
• Experiments to identify different fish species through AI automation have shown an average classification accuracy of about 85%.
• A demo developed by the team that demonstrates how a fish can be tagged and its capture information can be retrieved from a smartphone.

Publications, Patents and Awards

Outcomes

The Boat to Plate team has active engagement with Mures Tasmania. The team is working with the Mures fishing operations and strategy team to get insights about their challenges and used their fish species for the testing of the mock-up system. The mock-up system was designed based on the operations of a typical commercial fishing vessel, such as Mures’s fishing vessel, “Diana”.  This engagement has helped refine testing procedures; the mock-up system that has been designed is being used as a prototype to build similar systems for other fish species. Demonstrations have been presented at D61+ Live, and Seafood Directions, trade-shows in 2019.

Impacts

The on-board automated visual identification of fish catch will help markets gain greater clarity of products and provide regulators with greater assurances of sustainability, faster and more cost-effectively than available practices. This will lead to enhanced seafood supply chain management, traceability and food security for industry, wholesalers and retailers that purchase Australian seafood products. This will boost trust between buyers and sellers of Australian seafood, and has the potential to create a substantially improved product-market fit.

Illustration of costs saved (focus: Tuna industry)

Tuna is the third most widely traded international seafood commodity, with a global market worth $45b USD. The Boat to Plate team is commercially partnering to address traceability and supply chain inefficiencies, which will enable Australian producers to sustainably use, and capture a greater amount of the value of this precious Australian resource.

Market inefficiencies cause $9b USD in annual wastage in the tuna industry, and supply chain inefficiencies add another $8.7b. This presents a total addressable market of > $17.7b USD worth of supply chain inefficiencies and tuna wastage alone. Adoption of “Boat to Plate” technologies presents an opportunity to generate cost-savings associated with these kinds of losses in the seafood industry.

SIEF’s Role

SIEF played an instrumental role to transition this work from research project to uptake. The funding enabled bringing together multidisciplinary teams for the development and testing of the end to end mock-up system. The demonstrated success of the mock-up system has led to interest in the commercialisation of this technology. SIEF has helped the team identify their value proposition and the project team firmly believes they would not have advanced to this stage without this support.

Future Work

The funding and support from collaborators has been valuable in completing ideation stage and developing a prototype. The ultimate success of this work is heavily dependent upon execution beyond this stage for successful translation and commercialization of this work.

Boat to Plate PDF (127kb)