Gpt2 Small ExpandedbabyLM 100M Exp3 Conj Ratio 1p00 Mix 0p10 43

--- tags: - generated_from_trainer model-index: - name: gpt2_small_expandedbabyLM_100M_exp3_conj_ratio_1p00_mix_0p10_43 results: [] --- <!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. --> # gpt2_small_expandedbabyLM_100M_exp3_conj_ratio_1p00_mix_0p10_43 This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 3.5092 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0001 - train_batch_size: 256 - eval_batch_size: 256 - seed: 43 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 4000 - num_epochs: 20 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:-----:|:---------------:| | 3.6726 | 1.0 | 4591 | 4.1032 | | 3.3224 | 2.0 | 9182 | 3.6579 | | 3.1927 | 3.0 | 13773 | 3.5092 | | 3.1062 | 4.0 | 18364 | 3.5539 | | 3.0487 | 5.0 | 22955 | 3.5774 | | 3.0058 | 6.0 | 27546 | 3.5962 | ### Framework versions - Transformers 4.30.2 - Pytorch 2.11.0+cu130 - Datasets 4.1.1 - Tokenizers 0.13.3

Source context: 7 downloads · 0 likes · Repo devika-tiwari/gpt2_small_expandedbabyLM_100M_exp3_conj_ratio_1p00_mix_0p10_43