DorsetHeatwaveLLM2

--- license: agpl-3.0 datasets: - JuDDGES/en-appealcourt-coded-instruct_v02 language: - en base_model: unsloth/Qwen3-1.7B-Base library_name: peft tags: - lora - rs-lora - text-generation-inference - unsloth --- # **A little about me** I am 13, am in a normal non-grammar school and I have SEND (Special Educational Needs and Disabilities). So I decided that as it is a heatwave (albeit less intense), so it is DorsetHeatwaveLLM time! My idea was simple: Make the sequel. And I made this model on Sunday 5th July 2026. And to the question **"Also, have any Grammar students my time period beat me to making a similar sized model first?"**, nobody told me so, so I will assume the answer is **no**. Or at least no in Dorset from what I know. # **The *not so* grand return of DorsetHeatwaveLLM** **And as the UK Health Security Agency pushed a yellow heat warning** *(yes, I know, not as bad as the original DorsetHeatwaveLLM had it)*, as you can see from the image below...  ...**this gives us the** *(not so)* **grand return of DorsetHeatwaveLLM with DorsetHeatwaveLLM2!** ## **NOTE: THIS IS A ~2B MODEL! SMALL MODELS LIKE THIS CAN HALLUCINATE, ROLEPLAY OR GET THINGS WRONG. DO NOT USE THIS FOR MISSION CRITICAL APPLICATIONS!** # **Specifications** This is the data you can find from this that I will give over.  **Hyperparams** - Batch size: 2 - Learning rate: 0.0002 - Optimizer: AdamW 8-bit - Max steps: 450 - Context length: 2048 - Warmup steps: 5 **LoRA** - Rank: 32 - Alpha: 32 - Dropout: 0 - Variant: lora **Other info** - Seed: 1326 # **I decided not to do tests.** So if you want them ... ...make your own tests! I can be biased to show my model is better than it actually is so try it for yourself. # **GGUF Versions** [Find info on those here](GGUF_options.md). # **Some nice questions to ask, maybe nobody cares:** Note: "I" and "Me" in this secrtion refer to *you*, and "You" refers to *me*. This note only applies for these questions in the Q, not in the answers, otherwise known in a bullet point starting with A below the question starting with Q next to a number. Q1: Is this software Free or Nonfree? - A1: Same as DorsetHeatwaveLLM. It is Nonfree because the Free Software Foundatio...