Loading
Vous avez un workflow ComfyUI ? Obtenez une documentation lisible directement dans votre navigateur.Rendre un workflow lisible.Essayer maintenant ↗
This repository contains a math-focused PEFT LoRA adapter based on [openbmb/MiniCPM5-1B]( openbmb/MiniCPM5-1B).
Source du modèle
Description de la source
This repository contains a math-focused PEFT LoRA adapter based on [openbmb/MiniCPM5-1B](https://huggingface.co/ openbmb/MiniCPM5-1B).
Sources
1 sourceVérifié 22 sept.
Artefacts du modèle
4 artefactsadapter_model.safetensors
safetensors · 85,5 MB · SHA-256 c05345873ae1…8284 · Hugging Face
Téléchargercheckpoint-100/adapter_model.safetensors
safetensors · 42,8 MB · SHA-256 86cab059d6d4…f39d · Hugging Face
TéléchargerExtraits de sources
2 extraits
openbmb/MiniCPM5-1Bloraadapter32645.0150This model is intended for research and experimentation with math reasoning fine-tuning. It is not intended for high-stakes educational, financial, legal, medical, or safety-critical decisions.
The adapter was trained on math reasoning data, including MetaMathQA and Hendrycks MATH-style algebra/geometry examples, depending on the notebook configuration used for this run.
The run used lora fine-tuning with PEFT/LoRA-style adaptation where applicable. Training configuration and adapter metadata are included in adapter_enhancement_metadata.json.
Before/after evaluation was run with the same base model and PEFT disable_adapter() for the baseline.
| Task | Role | N | Before | After | Delta |
|---|---|---|---|---|---|
| GSM8K | primary | 30 | 0.133 | 0.200 | +0.067 |
| PIQA | guard | 150 | 0.620 | 0.647 | +0.027 |
Gate status: PASS.
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model = "openbmb/MiniCPM5-1B"
adapter_id = "MSGEncrypted/minicpm5-1b-math-lora"
tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(
base_model,
torch_dtype="auto",
device_map="auto",
trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, adapter_id)
prompt = "Solve: If there are 12 apples and 5 are eaten, how many remain?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
The model may produce incorrect reasoning, hallucinated steps, or invalid arithmetic. Evaluation results from small notebook samples should be treated as smoke tests, not full benchmark claims.
eval_results.json: captured GSM8K/PIQA notebook resultsadapter_enhancement_metadata.json: training and publish metadata2.10.0+cu128checkpoint-120/adapter_model.safetensors
safetensors · 85,5 MB · SHA-256 9feee39de6c6…d7e2 · Hugging Face
checkpoint-150/adapter_model.safetensors
safetensors · 85,5 MB · SHA-256 c05345873ae1…8284 · Hugging Face
Télécharger--- base_model: openbmb/MiniCPM5-1B library_name: peft pipeline_tag: text-generation license: apache-2.0 tags: - text-generation - math - gsm8k - piqa - lora - peft - adapter - minicpm --- # minicpm5-1b-math-lora ## Model Details This repository contains a math-focused PEFT LoRA adapter based on [openbmb/MiniCPM5-1B](https://huggingface.co/ openbmb/MiniCPM5-1B). - **Base model:** `openbmb/MiniCPM5-1B` - **Training mode:** `lora` - **Published artifact:** `adapter` - **Primary task:** grade-school math reasoning - **Primary eval:** GSM8K - **Guard eval:** PIQA - **LoRA rank:** `32` - **LoRA alpha:** `64` - **NEFTune alpha:** `5.0` - **Max steps:** `150` ## Intended Use This model is intended for research and experimentation with math reasoning fine-tuning. It is not intended for high-stakes educational, financial, legal, medical, or safety-critical decisions. ## Training Data The adapter was trained on math reasoning data, including MetaMathQA and Hendrycks MATH-style algebra/geometry examples, depending on the notebook configuration used for this run. ## Training Procedure The run used `lora` fine-tuning with PEFT/LoRA-style adaptation where applicable. Training configuration and adapter metadata are included in `adapter_enhancement_metadata.json`. ## Evaluation Before/after evaluation was run with the same base model and PEFT `disable_adapter()` for the baseline. | Task | Role | N | Before | After | Delta | |---|---:|---:|---:|---:|---:| | GSM8K | primary | 30 | 0.133 | 0.200 | +0.067 | | PIQA | guard | 150 | 0.620 | 0.647 | +0.027 | Gate status: **PASS**. ## How to Use ```python from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel base_model = "openbmb/MiniCPM5-1B" adapter_id = "MSGEncrypted/minicpm5-1b-math-lora" tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True) base = AutoModelForCausalLM.from_pretrained( base_model, torch_dtype="auto", device_map="auto", trust_remote_code=True, ) model = PeftModel.from_pretrained(base, adapter_id) prompt = "Solve: If there are 12 apples and 5 are eaten, how many remain?" inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ``` ## Limitations The model may produce incorrect reasoning, hal...
Source context: 9 downloads · 1 likes · Pipeline text-generation · Library peft · Repo MSGEncrypted/minicpm5-1b-math-lora