Loading
Vous avez un workflow ComfyUI ? Obtenez une documentation lisible directement dans votre navigateur.Rendre un workflow lisible.Essayer maintenant ↗
A vision-language model built upon the BLIP-2 framework using BioGPT and HIPT for pathology report generation and cross-modal retrieval of melanocytic lesions.
Source du modèle
Description de la source
A vision-language model built upon the BLIP-2 framework using BioGPT and HIPT for pathology report generation and cross-modal retrieval of melanocytic lesions.
This repository contains multiple checkpoints for the model which was used for the experiments in the paper. The model was trained and evaluated on a dataset of 19,636 melanocytic lesion cases, consisting of one or more whole slide images (WSIs) and a pathology report, using different training configurations. The supporting code is available from the corresponding GitHub repository. We refer to the paper for more information regarding the dataset, training, evaluation, and limitations.
Sources
1 sourceVérifié 17 sept.
Artefacts du modèle
4 artefactsstage_1_pretrained_stage_2/checkpoint.pth
pth · 2,06 GB · SHA-256 503e703daeb9…a6b1 · Hugging Face
TéléchargerExtraits de sources
2 extraitsstage_2_full_report/checkpoint.pth
pth · 1,54 GB · SHA-256 0d68f48e0747…74f4 · Hugging Face
--- license: apache-2.0 language: - en pipeline_tag: image-to-text tags: - medical --- # Model Card for PathBLIP-2 A vision-language model built upon the BLIP-2 framework using BioGPT and HIPT for pathology report generation and cross-modal retrieval of melanocytic lesions. ## Model Details This repository contains multiple checkpoints for the model which was used for the experiments in the paper. The model was trained and evaluated on a dataset of 19,636 melanocytic lesion cases, consisting of one or more whole slide images (WSIs) and a pathology report, using different training configurations. The supporting code is available from the corresponding GitHub repository. We refer to the paper for more information regarding the dataset, training, evaluation, and limitations. - **Paper: *["On the Importance of Text Preprocessing for Multimodal Representation Learning and Pathology Report Generation"](https://openreview.net/forum?id=5fQchwJgQr&referrer=%5Bthe%20profile%20of%20Ruben%20Lucassen%5D(%2Fprofile%3Fid%3D~Ruben_Lucassen1))*** - **Repository:** [GitHub](https://github.com/nuldertien/PathBLIP-2) - **Framework:** [BLIP-2](https://github.com/salesforce/LAVIS/tree/main/projects/blip2) - **Base language model:** [BioGPT](https://huggingface.co/microsoft/biogpt) - **WSI feature extractor:** [HIPT](https://github.com/mahmoodlab/HIPT) - **License:** Apache-2.0
Source context: 0 downloads · 1 likes · Pipeline image-to-text · Repo RTLucassen/PathBLIP-2
