Qwonkeau v0.2 0.9B

Qwonkeau-v0.2-0.9B Fully linearized Qwen3.5-0.8B with RWKV-7 ("Goose") and MesaNet layers in place of full attention. - State at v0.2: closer to pre-surgery capability at short seq len, loses track past a few hundred tokens in specific conditions, making that the point of focus for v0.3. - Changes from v0.1: - Numerous minor changes - Revised architecture and data - +110M parameters (from 6 new layers) - See later revision(s): TBD python from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("amkdg/Qwonkeau-v0.2-0.9B", trust_remote_code=True, dtype="bfloat16").cuda() tok = AutoTokenizer.from_pretrained("amkdg/Qwonkeau-v0.2-0.9B") ids = tok.apply_chat_template([{"role":"user","content":"hi"}], add_generation_prompt=True, return_tensors="pt").cuda() print(tok.decode(model.generate(ids, max_new_tokens=128, use_cache=False)[0, ids.shape[1]:])) Needs fla