ModelPublicadoExterno

Sam2.1 Hiera Tiny Mlx 16bit

MLX-native ports of Meta/Facebook SAM 2.1 models for Apple Silicon.

Fuentehuggingface.co/avbiswas/sam2.1-hiera-tiny-mlx-16bit/tree/3cd20137d61efd06c9e0abaae709aad610925a6c

TipoOtherModelo basefacebook/sam2.1-hiera-base-plusVersiónRevision 3cd2013Ccalidad

Fuente del modelo

Tipo: Other
Tamaño del archivo: 86,3 MB
Modelo base: facebook/sam2.1-hiera-base-plus
Versión: vRevision 3cd2013
Actualizado: 9 oct 2026
Descargas: 0
Fuente: Hugging Face

Descripción de la fuente

SAM 2.1 MLX

MLX-native ports of Meta/Facebook SAM 2.1 models for Apple Silicon.

This model is converted from Meta's SAM 2.1 checkpoints and the official facebookresearch/sam2 implementation. It is intended for local image segmentation and video object tracking with MLX, without requiring PyTorch at runtime.

Project repo: https://github.com/avbiswas/sam2-mlx
Model collection: https://huggingface.co/collections/avbiswas/sam2-mlx-6a0a0dcfbbbcb089d13d23cd
Original SAM2 repo: https://github.com/facebookresearch/sam2
Original models: https://huggingface.co/facebook

Install

pip install mlx-sam

or with uv:

uv pip install mlx-sam

Usage

import numpy as np
from mlx_sam import SAM2VideoPredictor

predictor = SAM2VideoPredictor.from_pretrained(
    "avbiswas/sam2.1-hiera-small-mlx"  # replace with this model repo id
)

state = predictor.init_state("path/to/video_or_frames")

predictor.add_new_points_or_box(
    state,
    frame_idx=0,
    obj_id=1,
    points=np.array([[625.0, 429.0]], dtype=np.float32),
    labels=np.array([1], dtype=np.int32),
)

for frame_idx, obj_ids, masks in predictor.propagate_in_video(state):
    # masks: NumPy float32 array shaped [objects, 1, height, width]
    pass

Benchmarks

Benchmarks were run on an Apple M2 Max with 32 GB unified memory. Video tests use the SAM2 dog demo clip: 1280x720, 289 frames, 29.97 FPS, 9.64 s.

FP32 MLX vs Torch/MPS

Prompted first-frame fixture at 1024x1024 internal resolution.

Model	Size	Torch/MPS	MLX	Speedup	Parity vs Torch
`sam2.1-hiera-tiny-mlx`	`172.6 MiB`	`96.6 ms`	`71.3 ms`	`1.36x`	mask mean abs `1.17e-05`
`sam2.1-hiera-small-mlx`	`199.7 MiB`	`112.5 ms`	`84.5 ms`	`1.33x`	mask mean abs `8.14e-06`
`sam2.1-hiera-base-plus-mlx`	`336.4 MiB`	`203.5 ms`	`144.7 ms`	`1.41x`	mask mean abs `5.04e-06`
`sam2.1-hiera-large-mlx`	`892.2 MiB`	`433.0 ms`	`341.1 ms`	`1.27x`	mask mean abs `7.84e-06`

Video Tracking

For sam2.1-hiera-small-mlx on the 9.64 second dog clip:

Workload	Torch/MPS	MLX	Result
Full video, post-prompt propagation	`331 ms/frame`	`189 ms/frame`	MLX `1.75x` faster
Full video, total run	`100.5 s`	`94.8 s`	MLX faster end to end
Raw propagation, no save/overlay/final resize	`407 ms/frame`	`287 ms/frame`	MLX `1.42x` faster

Experimental preview mode at 768x768 internal resolution:

Setting	Propagation	Quality vs 1024
`1024x1024` baseline	`268.5 ms/frame`	reference
`768x768`, fp16 memory attention	`52.9 ms/frame`	mean IoU `0.949`, presence `80 / 80` on 80-frame dog clip

Quantized Variants

Quantized models reduce download size and memory footprint. On current MLX kernels, quantization should not be assumed to speed up video tracking; it primarily helps memory and distribution size.

Variant	Typical Size Reduction	Notes
`*-mlx-16bit`	about `2x` smaller	fp16 weights, closest quantized parity
`*-mlx-8bit`	about `2.5x-3x` smaller	int8 linear quantization
`*-mlx-4bit`	about `3.5x` smaller	mixed recipe: int8 trunk/mask decoder, int4 memory/object-pointer layers

Example small model parity vs fp32 MLX:

Model	Size	Parity vs fp32 MLX
`sam2.1-hiera-small-mlx-16bit`	`99.9 MiB`	mask mean abs `8.24e-03`
`sam2.1-hiera-small-mlx-8bit`	`76.7 MiB`	mask mean abs `2.99e-02`
`sam2.1-hiera-small-mlx-4bit`	`56.4 MiB`	mask mean abs `2.87e-02`

License

This MLX port is released under the Apache 2.0 license.

The original SAM 2 repository and source models are from Meta/Facebook and are also Apache 2.0 licensed.

Original SAM2 license: https://github.com/facebookresearch/sam2/blob/main/LICENSE
Original SAM2 repo: https://github.com/facebookresearch/sam2

Fuentes

1 fuente

Hugging FacePrincipalVersión 3cd20137d61efd06c9e0abaae709aad610925a6c

huggingface.co/avbiswas/sam2.1-hiera-tiny-mlx-16bit/tree/3cd20137d61efd06c9e0abaae709aad610925a6c

Verificado 25 ago

Artefactos del modelo

1 artefacto

sam2.1_hiera_tiny_image_segmenter_fp16.safetensors

safetensors · 86,3 MB · SHA-256 d97f27bd15bb…c6b5 · Hugging Face

Descargar

Extractos de fuentes

2 extractos

Readme25 agohuggingface.co/avbiswas/sam2.1-hiera-tiny-mlx-16bit

--- license: apache-2.0 library_name: mlx pipeline_tag: image-segmentation tags: - mlx - sam2 - segment-anything - image-segmentation - video-segmentation - video-object-tracking - apple-silicon base_model: - facebook/sam2.1-hiera-tiny - facebook/sam2.1-hiera-small - facebook/sam2.1-hiera-base-plus - facebook/sam2.1-hiera-large --- # SAM 2.1 MLX MLX-native ports of Meta/Facebook SAM 2.1 models for Apple Silicon. This model is converted from Meta's SAM 2.1 checkpoints and the official `facebookresearch/sam2` implementation. It is intended for local image segmentation and video object tracking with MLX, without requiring PyTorch at runtime. - Project repo: https://github.com/avbiswas/sam2-mlx - Model collection: https://huggingface.co/collections/avbiswas/sam2-mlx-6a0a0dcfbbbcb089d13d23cd - Original SAM2 repo: https://github.com/facebookresearch/sam2 - Original models: https://huggingface.co/facebook ## Install ```bash pip install mlx-sam ``` or with uv: ```bash uv pip install mlx-sam ``` ## Usage ```python import numpy as np from mlx_sam import SAM2VideoPredictor predictor = SAM2VideoPredictor.from_pretrained( "avbiswas/sam2.1-hiera-small-mlx" # replace with this model repo id ) state = predictor.init_state("path/to/video_or_frames") predictor.add_new_points_or_box( state, frame_idx=0, obj_id=1, points=np.array([[625.0, 429.0]], dtype=np.float32), labels=np.array([1], dtype=np.int32), ) for frame_idx, obj_ids, masks in predictor.propagate_in_video(state): # masks: NumPy float32 array shaped [objects, 1, height, width] pass ``` ## Benchmarks Benchmarks were run on an Apple M2 Max with 32 GB unified memory. Video tests use the SAM2 dog demo clip: `1280x720`, 289 frames, 29.97 FPS, `9.64 s`. ### FP32 MLX vs Torch/MPS Prompted first-frame fixture at `1024x1024` internal resolution. | Model | Size | Torch/MPS | MLX | Speedup | Parity vs Torch | | --- | ---: | ---: | ---: | ---: | --- | | `sam2.1-hiera-tiny-mlx` | `172.6 MiB` | `96.6 ms` | `71.3 ms` | `1.36x` | mask mean abs `1.17e-05` | | `sam2.1-hiera-small-mlx` | `199.7 MiB` | `112.5 ms` | `84.5 ms` | `1.33x` | mask mean abs `8.14e-06` | | `sam2.1-hiera-base-plus-mlx` | `336.4 MiB` | `203.5 ms` | `144.7 ms` | `1.41x` | mask mean abs `5.04e-06` | | `sam2.1-hiera-large-mlx` | `892.2 MiB` | `433.0 ms` | `341.1 ms` | `1.27x` | mask mean abs `7.84e-06` | ### Video Tracking F...

Note25 agohuggingface.co/avbiswas/sam2.1-hiera-tiny-mlx-16bit

Source context: 0 downloads · 0 likes · Pipeline image-segmentation · Library mlx · Repo avbiswas/sam2.1-hiera-tiny-mlx-16bit