Sam2 Cells Seg

Usage

import numpy as np, torch
from PIL import Image
from sam2.sam2_image_predictor import SAM2ImagePredictor

predictor = SAM2ImagePredictor.from_pretrained("DnaRnaProteins/sam2-cells-seg")

image = np.array(Image.open("cell_image.png").convert("RGB"))
predictor.set_image(image)

with torch.inference_mode():
    masks, scores, _ = predictor.predict(
        point_coords=np.array([[128, 256]]),  # [x, y] prompt point
        point_labels=np.array([1]),
        multimask_output=True,
    )
# masks: (N, H, W) bool array
# scores: (N,) float confidence per mask

Via Modal endpoint

import base64, modal

segment = modal.Function.from_name("biomech-inference-serving", "segment")
with open("cell_image.png", "rb") as f:
    b64 = base64.b64encode(f.read()).decode()
result = segment.remote(b64)
# {"masks": [[...]], "scores": [0.94, ...]}

Limitations

• Optimised for fluorescence cell images; performance on brightfield or H&E may vary.
• Point prompts improve precision — promptless predictions use a default center point.

--- license: apache-2.0 language: - en tags: - cell-segmentation - biology - microscopy - sam2 - biomedical-imaging - instance-segmentation base_model: facebook/sam2.1-hiera-tiny datasets: - DnaRnaProteins/cell_seg_labeled pipeline_tag: image-segmentation --- # SAM2-tiny — Cell Segmentation Fine-tuned [SAM2-tiny](https://huggingface.co/facebook/sam2.1-hiera-tiny) for instance segmentation of cells in fluorescence microscopy images. Part of the [biomech-inference-serving](https://github.com/czi-ai/biomech-inference-serving-feedback) pipeline (internal research project). ## Training | | | |---|---| | Base model | `facebook/sam2.1-hiera-tiny` | | Training data | [`DnaRnaProteins/cell_seg_labeled`](https://huggingface.co/datasets/DnaRnaProteins/cell_seg_labeled) | | Fine-tuning | Full decoder fine-tune | | Framework | [sam2](https://github.com/facebookresearch/sam2) | ## Usage ```python import numpy as np, torch from PIL import Image from sam2.sam2_image_predictor import SAM2ImagePredictor predictor = SAM2ImagePredictor.from_pretrained("DnaRnaProteins/sam2-cells-seg") image = np.array(Image.open("cell_image.png").convert("RGB")) predictor.set_image(image) with torch.inference_mode(): masks, scores, _ = predictor.predict( point_coords=np.array([[128, 256]]), # [x, y] prompt point point_labels=np.array([1]), multimask_output=True, ) # masks: (N, H, W) bool array # scores: (N,) float confidence per mask ``` ## Via Modal endpoint ```python import base64, modal segment = modal.Function.from_name("biomech-inference-serving", "segment") with open("cell_image.png", "rb") as f: b64 = base64.b64encode(f.read()).decode() result = segment.remote(b64) # {"masks": [[...]], "scores": [0.94, ...]} ``` ## Limitations - Optimised for fluorescence cell images; performance on brightfield or H&E may vary. - Point prompts improve precision — promptless predictions use a default center point.