Minicpm5 1b Math LoRA

--- base_model: openbmb/MiniCPM5-1B library_name: peft pipeline_tag: text-generation license: apache-2.0 tags: - text-generation - math - gsm8k - piqa - lora - peft - adapter - minicpm --- # minicpm5-1b-math-lora ## Model Details This repository contains a math-focused PEFT LoRA adapter based on [openbmb/MiniCPM5-1B](https://huggingface.co/ openbmb/MiniCPM5-1B). - **Base model:** `openbmb/MiniCPM5-1B` - **Training mode:** `lora` - **Published artifact:** `adapter` - **Primary task:** grade-school math reasoning - **Primary eval:** GSM8K - **Guard eval:** PIQA - **LoRA rank:** `32` - **LoRA alpha:** `64` - **NEFTune alpha:** `5.0` - **Max steps:** `150` ## Intended Use This model is intended for research and experimentation with math reasoning fine-tuning. It is not intended for high-stakes educational, financial, legal, medical, or safety-critical decisions. ## Training Data The adapter was trained on math reasoning data, including MetaMathQA and Hendrycks MATH-style algebra/geometry examples, depending on the notebook configuration used for this run. ## Training Procedure The run used `lora` fine-tuning with PEFT/LoRA-style adaptation where applicable. Training configuration and adapter metadata are included in `adapter_enhancement_metadata.json`. ## Evaluation Before/after evaluation was run with the same base model and PEFT `disable_adapter()` for the baseline. | Task | Role | N | Before | After | Delta | |---|---:|---:|---:|---:|---:| | GSM8K | primary | 30 | 0.133 | 0.200 | +0.067 | | PIQA | guard | 150 | 0.620 | 0.647 | +0.027 | Gate status: **PASS**. ## How to Use ```python from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel base_model = "openbmb/MiniCPM5-1B" adapter_id = "MSGEncrypted/minicpm5-1b-math-lora" tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True) base = AutoModelForCausalLM.from_pretrained( base_model, torch_dtype="auto", device_map="auto", trust_remote_code=True, ) model = PeftModel.from_pretrained(base, adapter_id) prompt = "Solve: If there are 12 apples and 5 are eaten, how many remain?" inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ``` ## Limitations The model may produce incorrect reasoning, hal...