Internlm Xcomposer2d5 7b 4bit

--- license: other pipeline_tag: visual-question-answering --- <p align="center"> <img src="logo_en.png" width="600"/> <p> <p align="center"> <b><font size="6">InternLM-XComposer-2.5</font></b> <p> <div align="center"> [💻Github Repo](https://github.com/InternLM/InternLM-XComposer) [Online Demo](https://huggingface.co/spaces/Willow123/InternLM-XComposer) [Paper](https://huggingface.co/papers/2407.03320) </div> **InternLM-XComposer2.5** excels in various text-image comprehension and composition applications, achieving GPT-4V level capabilities with merely 7B LLM backend. IXC2.5 is trained with 24K interleaved image-text contexts, it can seamlessly extend to 96K long contexts via RoPE extrapolation. This long-context capability allows IXC-2.5 to excel in tasks requiring extensive input and output contexts. ## 4-Bit Model We offer 4-bit quantized models via LMDeploy to reduce memory requirements. For a memory usage comparison, please refer to [here](example_code/4bit/README.md). ```python from lmdeploy import TurbomindEngineConfig, pipeline from lmdeploy.vl import load_image engine_config = TurbomindEngineConfig(model_format='awq') pipe = pipeline('internlm/internlm-xcomposer2d5-7b-4bit', backend_config=engine_config) image = load_image('examples/dubai.png') response = pipe(('describe this image', image)) print(response.text) ``` ### Import from Transformers To load the InternLM-XComposer2.5 model using Transformers, use the following code: ```python import torch from transformers import AutoTokenizer, AutoModelForCausalLM ckpt_path = "internlm/internlm-xcomposer2d5-7b" tokenizer = AutoTokenizer.from_pretrained(ckpt_path, trust_remote_code=True).cuda() # Set `torch_dtype=torch.floatb16` to load model in bfloat16, otherwise it will be loaded as float32 and might cause OOM Error. model = AutoModelForCausalLM.from_pretrained(ckpt_path, torch_dtype=torch.bfloat16, trust_remote_code=True).cuda() model = model.eval() ``` ## Quickstart We provide a simple example to show how to use InternLM-XComposer2.5 with 🤗 Transformers. <details> <summary> <b>Video Understanding</b> </summary> ```python import torch from transformers import AutoModel, AutoTokenizer torch.set_grad_enabled(False) # init model and tokenizer model = AutoModel.from_pretrained('internlm/internlm-xcomposer2d5-7b', torch_dtype=torch.bfloat16, trust_remote_code=True).cuda().eval()...

Source context: 109 downloads · 13 likes · Pipeline visual-question-answering · Library transformers · Repo internlm/internlm-xcomposer2d5-7b-4bit