Ir para o conteúdo
ModelListadosExterno
Browse files

MiniCPM Llama3 V 2 5 Int4

This is the int4 quantized version of MiniCPM-Llama3-V 2.5. Running with int4 version would use lower GPU memory (about 9GB).

Fontehuggingface.co/DavidLanz/MiniCPM-Llama3-V-2_5-int4/tree/b95530587c7582e9ce603b2702615ddb46a5221e
TipoModeloVersãoRevision b955305Bnota

Fonte do modelo

Tipo
Modelo
Tamanho do arquivo
1,40–4,33 GB
Versão
vRevision b955305
Atualizado
30 de out. de 2026
Downloads
15
Fonte
Hugging Face

Descrição da fonte

MiniCPM-Llama3-V 2.5 int4

This is the int4 quantized version of MiniCPM-Llama3-V 2.5.
Running with int4 version would use lower GPU memory (about 9GB).

Usage

Inference using Huggingface transformers on NVIDIA GPUs. Requirements tested on python 3.10:

Pillow==10.1.0
torch==2.1.2
torchvision==0.16.2
transformers==4.40.0
sentencepiece==0.1.99
accelerate==0.30.1
bitsandbytes==0.43.1

# test.py
import torch
from PIL import Image
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained('DavidLanz/MiniCPM-Llama3-V-2_5-int4', trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained('DavidLanz/MiniCPM-Llama3-V-2_5-int4', trust_remote_code=True)
model.eval()

image = Image.open('xx.jpg').convert('RGB')
question = 'What is in the image?'
msgs = [{'role': 'user', 'content': question}]

res = model.chat(
    image=image,
    msgs=msgs,
    tokenizer=tokenizer,
    sampling=True, # if sampling=False, beam_search will be used by default
    temperature=0.7,
    # system_prompt='' # pass system_prompt if needed
)
print(res)

## if you want to use streaming, please make sure sampling=True and stream=True
## the model.chat will return a generator
res = model.chat(
    image=image,
    msgs=msgs,
    tokenizer=tokenizer,
    sampling=True,
    temperature=0.7,
    stream=True
)

generated_text = ""
for new_text in res:
    generated_text += new_text
    print(new_text, flush=True, end='')

Fontes

1 fonte
Hugging FacePrimárioVersão b95530587c7582e9ce603b2702615ddb46a5221e
huggingface.co/DavidLanz/MiniCPM-Llama3-V-2_5-int4/tree/b95530587c7582e9ce603b2702615ddb46a5221e

Verificado 30 de out.

Download files

Model · 2 weight files

All files listed in the provider index are present.

Repository files

File inventory only; completeness is unverified.

Artefatos de modelo

2 artefatos

model-00001-of-00002.safetensors

safetensors · 4,33 GB · SHA-256 5e20e08a758d…b3fc · Hugging Face

Download shard

model-00002-of-00002.safetensors

safetensors · 1,40 GB · SHA-256 8600aa101dcb…95ce · Hugging Face

Download shard

Trechos de fonte

3 trechos
Readme14 de ago.huggingface.co/DavidLanz/MiniCPM-Llama3-V-2_5-int4

--- pipeline_tag: visual-question-answering --- ## MiniCPM-Llama3-V 2.5 int4 This is the int4 quantized version of [MiniCPM-Llama3-V 2.5](https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5). Running with int4 version would use lower GPU memory (about 9GB). ## Usage Inference using Huggingface transformers on NVIDIA GPUs. Requirements tested on python 3.10： ``` Pillow==10.1.0 torch==2.1.2 torchvision==0.16.2 transformers==4.40.0 sentencepiece==0.1.99 accelerate==0.30.1 bitsandbytes==0.43.1 ``` ```python # test.py import torch from PIL import Image from transformers import AutoModel, AutoTokenizer model = AutoModel.from_pretrained('DavidLanz/MiniCPM-Llama3-V-2_5-int4', trust_remote_code=True) tokenizer = AutoTokenizer.from_pretrained('DavidLanz/MiniCPM-Llama3-V-2_5-int4', trust_remote_code=True) model.eval() image = Image.open('xx.jpg').convert('RGB') question = 'What is in the image?' msgs = [{'role': 'user', 'content': question}] res = model.chat( image=image, msgs=msgs, tokenizer=tokenizer, sampling=True, # if sampling=False, beam_search will be used by default temperature=0.7, # system_prompt='' # pass system_prompt if needed ) print(res) ## if you want to use streaming, please make sure sampling=True and stream=True ## the model.chat will return a generator res = model.chat( image=image, msgs=msgs, tokenizer=tokenizer, sampling=True, temperature=0.7, stream=True ) generated_text = "" for new_text in res: generated_text += new_text print(new_text, flush=True, end='') ```

Note30 de out.huggingface.co/DavidLanz/MiniCPM-Llama3-V-2_5-int4

Source context: 15 downloads · 2 likes · Pipeline visual-question-answering · Repo DavidLanz/MiniCPM-Llama3-V-2_5-int4

Note14 de ago.huggingface.co/DavidLanz/MiniCPM-Llama3-V-2_5-int4

Source context: 17 downloads · 2 likes · Pipeline visual-question-answering · Repo DavidLanz/MiniCPM-Llama3-V-2_5-int4