ModelVerifiedExternal

UltraFeedback Openbmb TinyLlama 1.1B Aligned With Semantic MARS Deberta RM

KindOtherPublisher@payelbCgrade

Model source

Kind: Other
File size: 9.90 KB–48.2 MB
Updated: Sep 19, 2026
Publisher: @payelb
Parameters: 1B

Source excerpt

Model artifacts

2 artifacts

adapter_model.safetensors

safetensors · 48.2 MB · SHA-256 ce175bdde740…7b7c

pytorch_model.bin

bin · 9.90 KB · SHA-256 442928b96e1b…ca6f

Source excerpts

3 excerpts

ReadmeJul 11huggingface.co/payelb/UltraFeedback_openbmb_TinyLlama-1.1B_aligned_with_semantic_MARS_deberta_RM

# payelb/UltraFeedback_openbmb_TinyLlama-1.1B_aligned_with_semantic_MARS_deberta_RM Base model: `TinyLlama/TinyLlama-1.1B-Chat-v1.0` Alignment dataset: `openbmb/UltraFeedback` Reward model: `payelb/UltraFeedback_openbmb_reward-model-deberta-v3-base_1k_fixed_MARS_semantic_refined` Method: PPO alignment with LoRA adapters. Reward model type: semantic-distance-aware MARS DeBERTa-v3-base reward model. Training details: - NUM_TRAIN_SAMPLES: 1000 - MAX_PROMPT_TOKENS: 256 - MIN_NEW_TOKENS: 32 - MAX_NEW_TOKENS: 64 - TOTAL_PPO_STEPS: 250 - PPO_EPOCHS: 2 - LR: 5e-06 - Batch size: 16 - Mini-batch size: 4 - Gradient accumulation: 4 - INIT_KL_COEF: 0.02 - TARGET_KL: 6.0 - ADAP_KL_CTRL: True - Reward normalization and clipping enabled, clip=5.0 - LoRA r=16, alpha=32, dropout=0.05 - Generation during PPO: do_sample=True, top_p=0.9, temperature=0.7

NoteJul 11huggingface.co/payelb/UltraFeedback_openbmb_TinyLlama-1.1B_aligned_with_semantic_MARS_deberta_RM

Source context: 0 downloads · 0 likes · Repo payelb/UltraFeedback_openbmb_TinyLlama-1.1B_aligned_with_semantic_MARS_deberta_RM

NoteMay 14huggingface.co/api/models/payelb/UltraFeedback_openbmb_TinyLlama-1.1B_aligned_with_semantic_MARS_deberta_RM

Source context: 0 downloads · 0 likes · Repo payelb/UltraFeedback_openbmb_TinyLlama-1.1B_aligned_with_semantic_MARS_deberta_RM