ModelVerificadoExterno

UltraFeedback Openbmb TinyLlama 1.1B Aligned With Semantic MARS Deberta RM

TipoOtherPublicador@payelbCnota

Fonte do modelo

Tipo: Other
Tamanho do arquivo: 9,90 KB–48,2 MB
Atualizado: 19 de set. de 2026
Publicador: @payelb
Parâmetros: 1B

Trecho da fonte

Artefatos de modelo

2 artefatos

adapter_model.safetensors

safetensors · 48,2 MB · SHA-256 ce175bdde740…7b7c

pytorch_model.bin

bin · 9,90 KB · SHA-256 442928b96e1b…ca6f

Trechos de fonte

3 trechos

Readme11 de jul.huggingface.co/payelb/UltraFeedback_openbmb_TinyLlama-1.1B_aligned_with_semantic_MARS_deberta_RM

# payelb/UltraFeedback_openbmb_TinyLlama-1.1B_aligned_with_semantic_MARS_deberta_RM Base model: `TinyLlama/TinyLlama-1.1B-Chat-v1.0` Alignment dataset: `openbmb/UltraFeedback` Reward model: `payelb/UltraFeedback_openbmb_reward-model-deberta-v3-base_1k_fixed_MARS_semantic_refined` Method: PPO alignment with LoRA adapters. Reward model type: semantic-distance-aware MARS DeBERTa-v3-base reward model. Training details: - NUM_TRAIN_SAMPLES: 1000 - MAX_PROMPT_TOKENS: 256 - MIN_NEW_TOKENS: 32 - MAX_NEW_TOKENS: 64 - TOTAL_PPO_STEPS: 250 - PPO_EPOCHS: 2 - LR: 5e-06 - Batch size: 16 - Mini-batch size: 4 - Gradient accumulation: 4 - INIT_KL_COEF: 0.02 - TARGET_KL: 6.0 - ADAP_KL_CTRL: True - Reward normalization and clipping enabled, clip=5.0 - LoRA r=16, alpha=32, dropout=0.05 - Generation during PPO: do_sample=True, top_p=0.9, temperature=0.7

Note11 de jul.huggingface.co/payelb/UltraFeedback_openbmb_TinyLlama-1.1B_aligned_with_semantic_MARS_deberta_RM

Source context: 0 downloads · 0 likes · Repo payelb/UltraFeedback_openbmb_TinyLlama-1.1B_aligned_with_semantic_MARS_deberta_RM

Note14 de mai.huggingface.co/api/models/payelb/UltraFeedback_openbmb_TinyLlama-1.1B_aligned_with_semantic_MARS_deberta_RM

Source context: 0 downloads · 0 likes · Repo payelb/UltraFeedback_openbmb_TinyLlama-1.1B_aligned_with_semantic_MARS_deberta_RM