UltraFeedback Openbmb TinyLlama 1.1B Aligned With Semantic MARS Deberta RM | Sweet Tea StudioUltraFeedback Openbmb TinyLlama 1.1B Aligned With Semantic MARS Deberta RM
payelb/UltraFeedback openbmb TinyLlama-1.1B aligned with semantic MARS deberta RM
Verified source
KindOtherVersionv8e82a0cd4d41a0605dbf4ccb095ee9ba44960a50Publisher@payelbCgrade Model source
- Kind
- Other
- Version
- v8e82a0cd4d41a0605dbf4ccb095ee9ba44960a50
- Parameters
- 1B
- Source
- Hugging Face
# payelb/UltraFeedback_openbmb_TinyLlama-1.1B_aligned_with_semantic_MARS_deberta_RM Base model: `TinyLlama/TinyLlama-1.1B-Chat-v1.0` Alignment dataset: `openbmb/UltraFeedback` Reward model: `payelb/UltraFeedback_openbmb_reward-model-deberta-v3-base_1k_fixed_MARS_semantic_refined` Method: PPO alignment with LoRA adapters. Reward model type: semantic-distance-aware MARS DeBERTa-v3-base reward model. Training details: - NUM_TRAIN_SAMPLES: 1000 - MAX_PROMPT_TOKENS: 256 - MIN_NEW_TOKENS: 32 - MAX_NEW_TOKENS: 64 - TOTAL_PPO_STEPS: 250 - PPO_EPOCHS: 2 - LR: 5e-06 - Batch size: 16 - Mini-batch size: 4 - Gradient accumulation: 4 - INIT_KL_COEF: 0.02 - TARGET_KL: 6.0 - ADAP_KL_CTRL: True - Reward normalization and clipping enabled, clip=5.0 - LoRA r=16, alpha=32, dropout=0.05 - Generation during PPO: do_sample=True, top_p=0.9, temperature=0.7
Sources & provenance
1 active source
Source evidence
3 excerpts
Method: PPO alignment with LoRA adapters.
payelb/UltraFeedback openbmb TinyLlama-1.1B aligned with semantic MARS deberta RM
payelb/UltraFeedback_openbmb_TinyLlama-1.1B_aligned_with_semantic_MARS_deberta_RM
