Tem um workflow do ComfyUI? Gere uma documentação legível direto no navegador.Torne um workflow legível.Experimentar agora ↗
MLX-converted weights of docling-project/docling-layout-heron, the default document-layout model of the Docling project. Apache-2.0, same as upstream.
Fonte do modelo
Descrição da fonte
MLX-converted weights of docling-project/docling-layout-heron, the default document-layout model of the Docling project. Apache-2.0, same as upstream.
The architecture is RT-DETRv2 with a ResNet-50-vd backbone, 300 queries, and 17 layout classes (, , , , , , , , , , , , , , , , ).
Fontes
1 fonteVerificado 27 de set.
Artefatos de modelo
1 artefatoTrechos de fonte
2 trechos
captionfootnoteformulalist_itempage_footerpage_headerpicturesection_headertabletexttitledocument_indexcodecheckbox_selectedcheckbox_unselectedformkey_value_regionRequires the RT-DETRv2 MLX port in mlx-vlm.
from pathlib import Path
from PIL import Image
from huggingface_hub import snapshot_download
from transformers import AutoProcessor
from mlx_vlm.utils import load_model
from mlx_vlm.models.rt_detr_v2.generate import RTDetrV2Predictor
import mlx_vlm.models.rt_detr_v2 # registers the processor with AutoProcessor
path = Path(snapshot_download("mlx-community/docling-layout-heron-mlx-bf16"))
model = load_model(path)
processor = AutoProcessor.from_pretrained(path)
predictor = RTDetrV2Predictor(model, processor, threshold=0.3)
result = predictor.predict(Image.open("page.png"))
for name, score, box in zip(result.class_names, result.scores, result.boxes):
print(f"{name:20s} {score:.3f} {box.tolist()}")
result is a DetectionResult with vectorized fields: boxes (N, 4) xyxy in original-image pixels, scores (N,), labels (N,) integer class ids, and class_names.
Produced with:
python -m mlx_vlm.models.rt_detr_v2.convert \
--hf-path docling-project/docling-layout-heron \
--output ./docling-layout-heron-mlx-bf16 \
--dtype bfloat16
Numerical validation against transformers.RTDetrV2ForObjectDetection on real document inputs: max abs error ~2e-5 on logits, sub-pixel on bboxes.
Apache-2.0. The original work is described in "Advanced Layout Analysis Models for Docling" by Livathinos et al. (arXiv:2509.11720); please cite the upstream paper if you use this model.
--- license: apache-2.0 library_name: mlx pipeline_tag: object-detection base_model: docling-project/docling-layout-heron base_model_relation: quantized tags: - mlx - rt_detr_v2 - object-detection - layout-analysis - docling - document-understanding --- # docling-layout-heron — MLX (bfloat16) MLX-converted weights of [`docling-project/docling-layout-heron`](https://huggingface.co/docling-project/docling-layout-heron), the default document-layout model of the [Docling](https://github.com/docling-project/docling) project. Apache-2.0, same as upstream. The architecture is RT-DETRv2 with a ResNet-50-vd backbone, 300 queries, and 17 layout classes (`caption`, `footnote`, `formula`, `list_item`, `page_footer`, `page_header`, `picture`, `section_header`, `table`, `text`, `title`, `document_index`, `code`, `checkbox_selected`, `checkbox_unselected`, `form`, `key_value_region`). ## Inference Requires the RT-DETRv2 MLX port in [`mlx-vlm`](https://github.com/Blaizzy/mlx-vlm). ```python from pathlib import Path from PIL import Image from huggingface_hub import snapshot_download from transformers import AutoProcessor from mlx_vlm.utils import load_model from mlx_vlm.models.rt_detr_v2.generate import RTDetrV2Predictor import mlx_vlm.models.rt_detr_v2 # registers the processor with AutoProcessor path = Path(snapshot_download("mlx-community/docling-layout-heron-mlx-bf16")) model = load_model(path) processor = AutoProcessor.from_pretrained(path) predictor = RTDetrV2Predictor(model, processor, threshold=0.3) result = predictor.predict(Image.open("page.png")) for name, score, box in zip(result.class_names, result.scores, result.boxes): print(f"{name:20s} {score:.3f} {box.tolist()}") ``` `result` is a `DetectionResult` with vectorized fields: `boxes` (N, 4) xyxy in original-image pixels, `scores` (N,), `labels` (N,) integer class ids, and `class_names`. ## Conversion Produced with: ```bash python -m mlx_vlm.models.rt_detr_v2.convert \ --hf-path docling-project/docling-layout-heron \ --output ./docling-layout-heron-mlx-bf16 \ --dtype bfloat16 ``` Numerical validation against `transformers.RTDetrV2ForObjectDetection` on real document inputs: max abs error ~2e-5 on logits, sub-pixel on bboxes. ## License and citation Apache-2.0. The original work is described in *"Advanced Layout Analysis Models for Docling"* by Livathinos et al. ([arXiv:2509.11720](htt...
Source context: 47 downloads · 0 likes · Pipeline object-detection · Library mlx · Repo mlx-community/docling-layout-heron-mlx-bf16