Yolo11 Segment Fish Grayscale

Yolo11n-seg Fish Segmentation

Model Overview

This model was trained to detect and segment fish in underwater Grayscale Imagery using the YOLO11n-seg architecture, leveraging automatic training with the Segment Anything Model (SAM) for generating segmentation masks. The combination of detection and SAM-powered segmentation enhances the model's ability to outline fish boundaries.

• Model Architecture: YOLO11n-seg
• Task: Fish Segmentation
• Footage Type: Grayscale Underwater Footage
• Classes: 1 (Fish)

Test Results

GIF description

Model Weights

Download the model weights here

Auto-Training Process

The segmentation dataset was generated using an automated pipeline:

• Detection Model: A pre-trained YOLO model (https://huggingface.co/akridge/yolo11-fish-detector-grayscale/) was used to detect fish.
• Segmentation: The SAM model (sam_b.pt) was applied to generate precise segmentation masks around detected fish.
• Output: The dataset was saved at /content/sam_dataset/.

This automated process allowed for efficient mask generation without manual annotation, facilitating faster dataset creation.

Intended Use

• Real-time fish detection and segmentation on grayscale underwater imagery.
• Post-processing of video or images for research purposes in marine biology and ecosystem monitoring.

Training Configuration

• Dataset: SAM asisted segmentation dataset.
• Training/Validation Split: 80% training, 20% validation.
• Number of Epochs: 50
• Learning Rate: 0.001
• Batch Size: 16
• Image Size: 640x640

Results and Metrics

The model was trained and evaluated on the generated segmentation dataset with the following results:

Confusion Matrix

Confusion Matrix

How to Use the Model

To use the trained YOLO11n-seg model for fish segmentation:

1. Load the Model:

   from ultralytics import YOLO

   # Load YOLO11n-seg model
   model = YOLO("yolo11n_fish_seg_trained.pt")

   # Perform inference on an image
   results = model("/content/test_image.jpg")
   results.show()

---

Limitations

The model was trained on black-and-white underwater footage, and may not generalize well to color images or videos with different lighting conditions. The unsupervised learning nature of this model may lead to some incorrect detections, particularly in noisy environments where it may confuse other underwater objects for fish. Images with complex backgrounds, occlusions, or poor resolution may affect the model's performance.

Additional Notes:

• Grayscale Imagery: The model may perform better on grayscale images and might not generalize well to color underwater footage or images with different lighting conditions.
• Unsupervised Learning: Since using an unsupervised approach, it's worth noting that this can make the model more flexible but also more prone to errors or misclassifications without annotated data.
• Ethical Considerations The unsupervised learning approach could lead to biases in detections, especially in new environments or types of marine life that were not represented in the training dataset. This model should not be used in critical applications without thorough validation to ensure it doesn't miss key detections or produce incorrect results in sensitive scenarios. Consider the potential environmental or societal impact when using the model for marine conservation or research, and ensure that the detections are verified.

Disclaimer

This repository is a scientific product and is not official communication of the National Oceanic and Atmospheric Administration, or the United States Department of Commerce. All NOAA project content is provided on an ‘as is’ basis and the user assumes responsibility for its use. Any claims against the Department of Commerce or Department of Commerce bureaus stemming from the use of this GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.