Surgical Instruments LoRA Adapter

--- library_name: peft pipeline_tag: image-feature-extraction tags: - peft - lora - open-clip - image-feature-extraction base_model: laion/CLIP-ViT-B-32-DataComp.XL-s13B-b90K --- # PEFT LoRA Adapter This repository contains a LoRA adapter for OpenCLIP vision inference. - Base model: `open-clip:laion/CLIP-ViT-B-32-DataComp.XL-s13B-b90K` - Adapter target: `visual.transformer` - Task: `image_embedding` ## Usage ```python import open_clip import torch from peft import PeftModel MODEL_NAME = 'hf-hub:laion/CLIP-ViT-B-32-DataComp.XL-s13B-b90K' ADAPTER_REPO = 'mounkie/surgical_instruments-lora-adapter' model, _, preprocess = open_clip.create_model_and_transforms(MODEL_NAME) model.visual.transformer = PeftModel.from_pretrained( model.visual.transformer, ADAPTER_REPO, ) model.eval() # image = preprocess(pil_image).unsqueeze(0) # with torch.no_grad(): # image_features = model.encode_image(image) # image_features = image_features / image_features.norm(dim=-1, keepdim=True) ```

Source context: 5 downloads · 0 likes · Pipeline image-feature-extraction · Library peft · Repo mounkie/surgical_instruments-lora-adapter