Phonebabylm Phone Deu Eng Sequential Interleaved
Trained with trainclm.py, a Hugging Face Accelerate causal-LM training script (no Trainer).
Source du modèle
- Type
- Other
- Taille du fichier
- 326 MB
- Version
- vRevision 4570ba0
- Mis à jour
- 26 sept. 2026
- Téléchargements
- 72
- Tâches
- Génération de texte
- Source
- Hugging Face
Description de la source
nikitastheo/phonebabylm-phone-deu-eng-sequential_interleaved
Trained with train_clm.py, a Hugging Face Accelerate causal-LM training script (no Trainer).
Training details
- Base config:
gpt_base_config.json - Tokenizer:
nikitastheo/phonelm-phone-deu-eng-tokenizer - Max steps: 30000
- Learning rate: 0.0001
- LR scheduler: linear
- Warmup steps: 3000
- Batch size (per device): 32
- Gradient accumulation steps: 1
- Total train batch size: 32
- Language switch steps: 10000
Sources
1 source
Sources
1 sourceVérifié 26 sept.
Artefacts du modèle
1 artefact
Artefacts du modèle
1 artefactExtraits de sources
2 extraits
Extraits de sources
2 extraits--- tags: - causal-lm - text-generation library_name: transformers --- # nikitastheo/phonebabylm-phone-deu-eng-sequential_interleaved Trained with train_clm.py, a Hugging Face Accelerate causal-LM training script (no `Trainer`). ## Training details - **Base config**: `gpt_base_config.json` - **Tokenizer**: `nikitastheo/phonelm-phone-deu-eng-tokenizer` - **Max steps**: 30000 - **Learning rate**: 0.0001 - **LR scheduler**: linear - **Warmup steps**: 3000 - **Batch size (per device)**: 32 - **Gradient accumulation steps**: 1 - **Total train batch size**: 32 - **Language switch steps**: 10000
Source context: 72 downloads · 0 likes · Pipeline text-generation · Library transformers · Repo nikitastheo/phonebabylm-phone-deu-eng-sequential_interleaved
