lambda-1-160m-base lambda-1-160m-base is an experimental language model created with a custom decoder-only Transformer implementation. All training code is publicly available at . ## Model Details | Item | Value | |---|---:| | Parameters | 164.5M | | Architecture | Decoder-only Transformer | | Context length | 1024 tokens | | Tokenizer | Byte-level BPE | | Vocabulary size | 65,536 | | Layers | 16 | | Hidden size | 768 | | Attention heads | 12 | | FFN size | 3,072 | ## Training Data The model was pretrained on a Japanese text mixture. | Dataset | Notes | |---|---| | | Filtered Japanese web corpus | | | Synthetic Japanese corpus | ## Usage ## Limitations This model is not instruction-tuned or safety-aligned. It may generate incorrect, biased, unsafe, or low-quality text. The model was trained on a limited Japanese corpus mixture and has not been evaluated on standard benchmarks.
