Lumia 50m | Sweet Tea StudioLumia 50m
Custom reasoning model with Gated Reasoning Memory (GRM) + Stateful Inference Pipeline (SIP). Params: 46.35M Architecture: 12-layer Transformer, GQA (10Q/2KV), SwiGLU, RoPE, RMSNorm + LayerScale Memory: 6 RMC cells (K=16 each), 3 RRB blocks with convergence detection Inference: SIP with prefill/extend, exact output equivalence Training: Pure PyTorch, CPU/T4/TPU support, fp16 AMP, activation...
Verified source
Kindtext-generationVersionv5166798c3fc67db74c2b573973b2ae6441d76edeLicenseapache-2.0Publisher@samcheng0Cgrade Model source
- Kind
- text-generation
- Version
- v5166798c3fc67db74c2b573973b2ae6441d76ede
- License
- apache-2.0
- Parameters
- 50M
- Tasks
- text-generation
- Source
- Hugging Face
--- license: apache-2.0 language: - en tags: - lumia - reasoning - grm - sip pipeline_tag: text-generation --- # Lumia-50M Custom reasoning model with Gated Reasoning Memory (GRM) + Stateful Inference Pipeline (SIP). - **Params:** 46.35M - **Architecture:** 12-layer Transformer, GQA (10Q/2KV), SwiGLU, RoPE, RMSNorm + LayerScale - **Memory:** 6 RMC cells (K=16 each), 3 RRB blocks with convergence detection - **Inference:** SIP with prefill/extend, exact output equivalence - **Training:** Pure PyTorch, CPU/T4/TPU support, fp16 AMP, activation checkpointing - **Tokenizer:** BPE (12288 vocab, 12017 merges)
Sources & provenance
1 active source
Source evidence
3 excerpts
license: apache-2.0 language: en tags: lumia reasoning grm sip pipelinetag: text-generation
Custom reasoning model with Gated Reasoning Memory (GRM) + Stateful Inference Pipeline (SIP). Params: 46.35M Architecture: 12-layer Transformer, GQA (10Q/2KV), SwiGLU, RoPE, RMSNorm + LayerScale Memory: 6 RMC cells (K=16 each), 3 RRB blocks with convergence…
