ModelPublicadoExterno

VLM Cholecystectomie

This repository contains models for Surgical Phase, Step, Target and Tools Recognition in Laparoscopic Cholecystectomy videos.\ It features two distinct approaches: a lightweight custom ViT (ResNet+Transformer) and a...

Fuentehuggingface.co/Bopa-Boptech/VLM-Cholecystectomie/tree/610b3c3a6aedef329dd789a2dfd8651bdaac76e6

TipoOtherModelo baseBopa-Boptech/VLM-CholecystectomieVersiónAdapter modelCcalidad

Fuente del modelo

Tipo: Other
Tamaño del archivo: 784 MB
Modelo base: Bopa-Boptech/VLM-Cholecystectomie
Versión: vAdapter model
Actualizado: 24 oct 2026
Fuente: Hugging Face

Descripción de la fuente

VLM-Cholecystectomie

This repository contains models for Surgical Phase, Step, Target and Tools Recognition in Laparoscopic Cholecystectomy videos.
It features two distinct approaches: a lightweight custom ViT (ResNet+Transformer) and a large-scale finetuned Qwen3-VL.

Models

1. ViT-ResNet

A lightweight, specialized architecture designed for efficient video classification.

Backbone: ResNet50 (Frozen)
Aggregator: Temporal Transformer Encoder
MLP Heads: for Phase, Step, Target and Tool prediction.

2. Qwen3-VL

A Vision-Language Model finetuned for surgical understanding.

Base Model: unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit
Method: LoRA Finetuning (Vision & Language layers)

Tasks

The models are trained to predict three levels of surgical granularity simultaneously:

Phase: High-level surgical stages (e.g., PREPARATION, CALOT_TRIANGLE_DISSECTION).
Step: Fine-grained surgical actions (e.g., CYSTIC_DUCT_DISSECTION, CLIPPING).
Target: The anatomical structure or object being operated on (e.g., CYSTIC_ARTERY, GALLBLADDER).
Tool(s): The list of tool(s) being actively used in the surgery (e.g., GRASPER HOOK, GRASPER).

Usage

Inference with ViT-ResNet

The ViT model requires the specific architecture definition (available in the src folder of the associated code repository or the Space).

import torch
from model_utils import SurgicalTransformer # Custom class

# Load Model
model = SurgicalTransformer(vocab_size_dict={"phase": 7, "step": 30, "target": 29})
checkpoint = torch.load("models/vit_v1/vit_resampling_v1.pt")
model.load_state_dict(checkpoint["model_state_dict"])
model.eval()

# Inference
# input_tensor: [Batch, Time, Channels, Height, Width]
output = model(input_tensor)

Fuentes

1 fuente

Hugging FacePrincipalVersión 610b3c3a6aedef329dd789a2dfd8651bdaac76e6

huggingface.co/Bopa-Boptech/VLM-Cholecystectomie/tree/610b3c3a6aedef329dd789a2dfd8651bdaac76e6

Verificado 24 oct

Download files

Artefactos del modelo

64 artefactos

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_0/checkpoint-100/adapter_model.safetensors

safetensors · 784 MB · SHA-256 fe97f7b58501…2bc3 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_0/checkpoint-150/adapter_model.safetensors

safetensors · 784 MB · SHA-256 b4b602be79c7…7baf · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_0/checkpoint-200/adapter_model.safetensors

safetensors · 784 MB · SHA-256 7eadbc3c158f…dbc2 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_0/checkpoint-238/adapter_model.safetensors

safetensors · 784 MB · SHA-256 ff29c38ef22f…8967 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_0/checkpoint-50/adapter_model.safetensors

safetensors · 784 MB · SHA-256 94e10bca8259…477f · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_0/model/adapter_model.safetensors

safetensors · 784 MB · SHA-256 ff29c38ef22f…8967 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_1/checkpoint-100/adapter_model.safetensors

safetensors · 784 MB · SHA-256 276542b30414…b1bc · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_1/checkpoint-150/adapter_model.safetensors

safetensors · 784 MB · SHA-256 35665576bc66…6ed4 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_1/checkpoint-200/adapter_model.safetensors

safetensors · 784 MB · SHA-256 b46b892bfa1d…8d2e · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_1/checkpoint-210/adapter_model.safetensors

safetensors · 784 MB · SHA-256 5588226a3e07…dbd0 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_1/checkpoint-50/adapter_model.safetensors

safetensors · 784 MB · SHA-256 1d576e85083d…9d4f · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_1/model/adapter_model.safetensors

safetensors · 784 MB · SHA-256 5588226a3e07…dbd0 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_2/checkpoint-100/adapter_model.safetensors

safetensors · 784 MB · SHA-256 2c0175064871…edd6 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_2/checkpoint-150/adapter_model.safetensors

safetensors · 784 MB · SHA-256 e085ebbc4b2b…2976 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_2/checkpoint-161/adapter_model.safetensors

safetensors · 784 MB · SHA-256 ee8843534344…8ad7 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_2/checkpoint-50/adapter_model.safetensors

safetensors · 784 MB · SHA-256 c30798babc77…675f · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_2/model/adapter_model.safetensors

safetensors · 784 MB · SHA-256 ee8843534344…8ad7 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_3/checkpoint-100/adapter_model.safetensors

safetensors · 784 MB · SHA-256 c011bc781521…3144 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_3/checkpoint-150/adapter_model.safetensors

safetensors · 784 MB · SHA-256 29301b7e8c75…d9a2 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_3/checkpoint-189/adapter_model.safetensors

safetensors · 784 MB · SHA-256 95519324202e…f13d · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_3/checkpoint-50/adapter_model.safetensors

safetensors · 784 MB · SHA-256 fc618c315d35…94c8 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_3/model/adapter_model.safetensors

safetensors · 784 MB · SHA-256 95519324202e…f13d · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_4/checkpoint-100/adapter_model.safetensors

safetensors · 784 MB · SHA-256 94bbc69bfeab…d7a3 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_4/checkpoint-126/adapter_model.safetensors

safetensors · 784 MB · SHA-256 c858e8b48ebb…d953 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_4/checkpoint-50/adapter_model.safetensors

safetensors · 784 MB · SHA-256 b6c16d9eb08b…2b51 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_4/model/adapter_model.safetensors

safetensors · 784 MB · SHA-256 c858e8b48ebb…d953 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_5/checkpoint-100/adapter_model.safetensors

safetensors · 784 MB · SHA-256 94bbc69bfeab…d7a3 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_5/checkpoint-126/adapter_model.safetensors

safetensors · 784 MB · SHA-256 c858e8b48ebb…d953 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_5/checkpoint-50/adapter_model.safetensors

safetensors · 784 MB · SHA-256 b6c16d9eb08b…2b51 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_cleaned_v0/fold_5/model/adapter_model.safetensors

safetensors · 784 MB · SHA-256 c858e8b48ebb…d953 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_0/checkpoint-100/adapter_model.safetensors

safetensors · 784 MB · SHA-256 745287cf520f…2418 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_0/checkpoint-150/adapter_model.safetensors

safetensors · 784 MB · SHA-256 3de708894d55…1299 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_0/checkpoint-200/adapter_model.safetensors

safetensors · 784 MB · SHA-256 1753cefe31ca…0336 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_0/checkpoint-250/adapter_model.safetensors

safetensors · 784 MB · SHA-256 17ea2523a8df…6b59 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_0/checkpoint-300/adapter_model.safetensors

safetensors · 784 MB · SHA-256 57ea5a35998d…2550 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_0/checkpoint-350/adapter_model.safetensors

safetensors · 784 MB · SHA-256 177a1ed36da2…b455 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_0/checkpoint-400/adapter_model.safetensors

safetensors · 784 MB · SHA-256 4403d2cceb84…d724 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_0/checkpoint-450/adapter_model.safetensors

safetensors · 784 MB · SHA-256 8c9a07c3d733…ba9b · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_0/checkpoint-50/adapter_model.safetensors

safetensors · 784 MB · SHA-256 6f8277548489…e64c · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_0/checkpoint-500/adapter_model.safetensors

safetensors · 784 MB · SHA-256 eb6f42ace32e…68e3 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_0/checkpoint-504/adapter_model.safetensors

safetensors · 784 MB · SHA-256 761c905580ec…12a8 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_0/model/adapter_model.safetensors

safetensors · 784 MB · SHA-256 761c905580ec…12a8 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_1/checkpoint-100/adapter_model.safetensors

safetensors · 784 MB · SHA-256 b4665888adb8…0186 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_1/checkpoint-150/adapter_model.safetensors

safetensors · 784 MB · SHA-256 64fff09735c7…86d6 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_1/checkpoint-200/adapter_model.safetensors

safetensors · 784 MB · SHA-256 e941073cbc28…af24 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_1/checkpoint-250/adapter_model.safetensors

safetensors · 784 MB · SHA-256 d491f0b2d67c…63f0 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_1/checkpoint-300/adapter_model.safetensors

safetensors · 784 MB · SHA-256 ab9d1e7eff16…69b0 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_1/checkpoint-350/adapter_model.safetensors

safetensors · 784 MB · SHA-256 586145fa9ed1…849b · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_1/checkpoint-400/adapter_model.safetensors

safetensors · 784 MB · SHA-256 6279665cdbc1…f2c5 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_1/checkpoint-434/adapter_model.safetensors

safetensors · 784 MB · SHA-256 75757a4a69dd…71ef · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_1/checkpoint-50/adapter_model.safetensors

safetensors · 784 MB · SHA-256 e777c3b0b5a9…6219 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_1/model/adapter_model.safetensors

safetensors · 784 MB · SHA-256 75757a4a69dd…71ef · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_2/checkpoint-100/adapter_model.safetensors

safetensors · 784 MB · SHA-256 7f61ba9823a4…9085 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_2/checkpoint-150/adapter_model.safetensors

safetensors · 784 MB · SHA-256 246928e303a5…40ca · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_2/checkpoint-200/adapter_model.safetensors

safetensors · 784 MB · SHA-256 6d7affbf5da3…99ac · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_2/checkpoint-250/adapter_model.safetensors

safetensors · 784 MB · SHA-256 fa89db378d55…44e1 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_2/checkpoint-300/adapter_model.safetensors

safetensors · 784 MB · SHA-256 17fa77a631c1…e9cd · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_2/checkpoint-350/adapter_model.safetensors

safetensors · 784 MB · SHA-256 f6908722ca8c…9ed4 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_2/checkpoint-378/adapter_model.safetensors

safetensors · 784 MB · SHA-256 e9fce2836fc9…5fd2 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_2/checkpoint-50/adapter_model.safetensors

safetensors · 784 MB · SHA-256 7a743606f21a…bad5 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_2/model/adapter_model.safetensors

safetensors · 784 MB · SHA-256 e9fce2836fc9…5fd2 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_3/checkpoint-100/adapter_model.safetensors

safetensors · 784 MB · SHA-256 d4db12f9906c…4ad9 · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_3/checkpoint-150/adapter_model.safetensors

safetensors · 784 MB · SHA-256 cb4f767ca081…0efe · Hugging Face

Download file

not_used_in_demo/unsloth_Qwen3-VL-8B-Instruct-unsloth-bnb-4bit_aug_v0/fold_3/checkpoint-200/adapter_model.safetensors

safetensors · 784 MB · SHA-256 4a58fd840f70…2eed · Hugging Face

Download file

Extractos de fuentes

2 extractos

Readme6 agohuggingface.co/Bopa-Boptech/VLM-Cholecystectomie

--- license: mit language: - en base_model: - Bopa-Boptech/VLM-Cholecystectomie pipeline_tag: video-classification tags: - medical - surgery - cholecystectomie - wip --- # VLM-Cholecystectomie This repository contains models for **Surgical Phase, Step, Target and Tools Recognition** in Laparoscopic Cholecystectomy videos.\ It features two distinct approaches: a lightweight custom **ViT (ResNet+Transformer)** and a large-scale finetuned **Qwen3-VL**. ## Models ### 1. ViT-ResNet A lightweight, specialized architecture designed for efficient video classification. - **Backbone:** ResNet50 (Frozen) - **Aggregator:** Temporal Transformer Encoder - **MLP Heads:** for **Phase**, **Step**, **Target** and **Tool** prediction. ### 2. Qwen3-VL A Vision-Language Model finetuned for surgical understanding. - **Base Model:** `unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit` - **Method:** LoRA Finetuning (Vision & Language layers) ## Tasks The models are trained to predict three levels of surgical granularity simultaneously: 1. **Phase:** High-level surgical stages (e.g., `PREPARATION`, `CALOT_TRIANGLE_DISSECTION`). 2. **Step:** Fine-grained surgical actions (e.g., `CYSTIC_DUCT_DISSECTION`, `CLIPPING`). 3. **Target:** The anatomical structure or object being operated on (e.g., `CYSTIC_ARTERY`, `GALLBLADDER`). 4. **Tool(s):** The list of tool(s) being actively used in the surgery (e.g., `GRASPER HOOK`, `GRASPER`). ## Usage ### Inference with ViT-ResNet The ViT model requires the specific architecture definition (available in the `src` folder of the associated code repository or the Space). ```python import torch from model_utils import SurgicalTransformer # Custom class # Load Model model = SurgicalTransformer(vocab_size_dict={"phase": 7, "step": 30, "target": 29}) checkpoint = torch.load("models/vit_v1/vit_resampling_v1.pt") model.load_state_dict(checkpoint["model_state_dict"]) model.eval() # Inference # input_tensor: [Batch, Time, Channels, Height, Width] output = model(input_tensor) ```

Note6 agohuggingface.co/Bopa-Boptech/VLM-Cholecystectomie

Source context: 0 downloads · 0 likes · Pipeline video-classification · Repo Bopa-Boptech/VLM-Cholecystectomie