--- base_model: Qwen/Qwen3-4B datasets: - AbijahKaj/kicad-netlist-sft-dataset language: - en license: apache-2.0 library_name: transformers tags: - electronics - kicad - netlist - schematic - eda - circuit-design - skidl - pcb - trl - sft - peft - lora --- # qwen3-4b-skidl A fine-tune of [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) for generating **executable SKiDL Python netlists** from natural-language circuit descriptions. Given a description like *"ESP32 board with BME280 on I2C and USB-C"*, the model outputs valid Python using `Part()`, `Net()`, and `generate_netlist()` — producing a `.net` file KiCad can open directly. This repo has two branches: | Branch | Contents | Size | Use | |---|---|---|---| | **`main`** (this) | Merged weights — LoRA baked into base | 7.5 GB | Load directly, no base model needed | | [`lora`](https://huggingface.co/AbijahKaj/qwen3-4b-skidl/tree/lora) | Adapter only | 504 MB | Lightweight; load on top of Qwen3-4B | ## Quick start ### Merged (main branch — recommended) ```python from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained( "AbijahKaj/qwen3-4b-skidl", dtype="auto", device_map="auto", ) tokenizer = AutoTokenizer.from_pretrained("AbijahKaj/qwen3-4b-skidl") ``` ### LoRA adapter (lora branch) ```python from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B", dtype="auto", device_map="auto") tokenizer = AutoTokenizer.from_pretrained("AbijahKaj/qwen3-4b-skidl", revision="lora") model = PeftModel.from_pretrained(base, "AbijahKaj/qwen3-4b-skidl", revision="lora") ``` ### Generating a circuit ```python SYSTEM = ( "You are an expert electronics engineer and KiCad schematic designer. " "When given a description of an electronic circuit, generate executable " "SKiDL Python code that defines the circuit using the SKiDL library." ) messages = [ {"role": "system", "content": SYSTEM}, {"role": "user", "content": "Design a CAN bus interface using MCP2515 controller and TJA1050 transceiver on SPI, with 120 ohm termination resistor."}, ] text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = tokenizer(text, return_tensors="pt").to(model.device) out = model.generate(**inputs, max_new_tokens=1024, do_sample=False) print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)) ``` ## Training | | | |---|---| | **Base model** | Qwen/Qwen3-4B | | **Method** | SFT + LoRA | | **Dataset** | [AbijahKaj/kicad-netlist-sft-dataset](https://huggingface.co/datasets/AbijahKaj/kicad-netlist-sft-dataset) — 100,179 SKiDL Python circuit examples | | **LoRA rank / alpha** | r=64, α=32, dropout=0.05 | | **Target modules** | q/k/v/o\_proj, gate/up/down\_proj | | **Epochs** | 2 | | **Peak LR** | 2e-4 (cosine decay) | | **Effective batch** | 8 | | **Max length** | 8192 tokens | | **Trainable params** | ~132M (5.65% of 4B) | ### Loss curve | Phase | Train loss | Token accuracy | |---|---|---| | Start | 0.86 | 75% | | Epoch 1 end | 0.17 | 95% | | Epoch 2 end | **0.15** | **95–96%** | | Eval loss (final) | **0.1581** | **95.3%** | ### SKiDL validation (final checkpoint) Functional scoring across 5 held-out circuits: Python syntax + SKiDL import + net count + part count + connectivity + GND checks. | Circuit | Score | |---|---| | LED blink — ATtiny85 | 0.85 | | USB power meter — ATmega328P + INA219 | 0.85 | | CAN bus — MCP2515 + TJA1050 | **0.95** | | **Average** | **0.883** | ## Why SKiDL? KiCad's native s-expression netlist is deeply nested and pathological for LLMs. [PCBSchemaGen (arXiv:2602.00510)](https://arxiv.org/abs/2602.00510) shows SKiDL Python achieves **87% Pass@1 with GPT-4o** on circuit generation — because Python-based HDLs align with LLM pretraining data and are 3× more compact than s-expressions. ## Dataset [AbijahKaj/kicad-netlist-sft-dataset](https://huggingface.co/datasets/AbijahKaj/kicad-netlist-sft-dataset) — 100,179 ChatML examples: | Source | Count | |---|---| | Si7li/ltspice-spice-circuits (LTspice → SKiDL) | ~54,000 | | bshada/open-schematics (GitHub `.kicad_sch` files) | ~45,000 | | Ashed00/SPICE-Circuits | ~800 | | Synthetic + tool-augmented | ~308 | ## References - [PCBSchemaGen (arXiv:2602.00510)](https://arxiv.org/abs/2602.00510) - [CircuitLM (arXiv:2601.04505)](https://arxiv.org/abs/2601.04505) - [SKiDL](https://github.com/devbisme/skidl)
