Deep Learning for Automated Shark Detection and Biometrics Without Keypoints

Abstract

This study introduces a pipeline designed to aid biologists via automatic detection and biometric analysis of marine animals. Our approach uses detection transformer (DETR) to detect subjects in an image, then generates a segmentation mask over the animal. We also introduce a new method to measure the center line of segmentations, which can be used to assess length during tail movement of animals in images. We test our system on a new dataset of aerial drone imagery of Pacific Nurse Sharks ( Ginglymostoma unami ). The detection model was trained on a dataset of drone-captured images under diverse environmental conditions of varying water clarity and lighting conditions, achieving a recall of 0.96 and precision of 0.80 at an IOU of 0.35. Notably, our method does not require labeled segmentations or keypoints in the dataset, as we find Segment Anything Model (SAM) has strong zero-shot performance. The efficiency of the pipeline was benchmarked against non-expert human annotators, showing a 91% decrease in data analysis time.