Skip to content
ModelListedExternal
Download

Phonebabylm Text Deu Eng Sequential Interleaved

Trained with trainclm.py, a Hugging Face Accelerate causal-LM training script (no Trainer).

Sourcehuggingface.co/nikitastheo/phonebabylm-text-deu-eng-sequential_interleaved/tree/1b20ff891d042a08d0f0be5a72658c58ee8d6766
KindOtherVersionRevision 1b20ff8Dgrade

Model source

Kind
Other
File size
414 MB
Version
vRevision 1b20ff8
Updated
Oct 20, 2026
Downloads
24
Source
Hugging Face

Source description

nikitastheo/phonebabylm-text-deu-eng-sequential_interleaved

Trained with train_clm.py, a Hugging Face Accelerate causal-LM training script (no Trainer).

Training details

  • Base config: gpt_base_config.json
  • Tokenizer: nikitastheo/phonelm-text-deu-eng-tokenizer
  • Max steps: 30000
  • Learning rate: 0.0001
  • LR scheduler: linear
  • Warmup steps: 3000
  • Batch size (per device): 32
  • Gradient accumulation steps: 1
  • Total train batch size: 32
  • Language switch steps: 10000

Sources

1 source
Hugging FacePrimaryVersion 1b20ff891d042a08d0f0be5a72658c58ee8d6766
huggingface.co/nikitastheo/phonebabylm-text-deu-eng-sequential_interleaved/tree/1b20ff891d042a08d0f0be5a72658c58ee8d6766

Verified Oct 20

Model artifacts

1 artifact

model.safetensors

safetensors · 414 MB · SHA-256 f034347bee2f…d669 · Hugging Face

Download

Source excerpts

3 excerpts
ReadmeAug 4huggingface.co/nikitastheo/phonebabylm-text-deu-eng-sequential_interleaved

--- tags: - causal-lm - text-generation library_name: transformers --- # nikitastheo/phonebabylm-text-deu-eng-sequential_interleaved Trained with train_clm.py, a Hugging Face Accelerate causal-LM training script (no `Trainer`). ## Training details - **Base config**: `gpt_base_config.json` - **Tokenizer**: `nikitastheo/phonelm-text-deu-eng-tokenizer` - **Max steps**: 30000 - **Learning rate**: 0.0001 - **LR scheduler**: linear - **Warmup steps**: 3000 - **Batch size (per device)**: 32 - **Gradient accumulation steps**: 1 - **Total train batch size**: 32 - **Language switch steps**: 10000

NoteOct 20huggingface.co/nikitastheo/phonebabylm-text-deu-eng-sequential_interleaved

Source context: 24 downloads · 0 likes · Pipeline text-generation · Library transformers · Repo nikitastheo/phonebabylm-text-deu-eng-sequential_interleaved

NoteAug 4huggingface.co/nikitastheo/phonebabylm-text-deu-eng-sequential_interleaved

Source context: 3232 downloads · 0 likes · Pipeline text-generation · Library transformers · Repo nikitastheo/phonebabylm-text-deu-eng-sequential_interleaved