--- base_model: unsloth/gpt-oss-20b-unsloth-bnb-4bit library_name: peft pipeline_tag: text-generation license: apache-2.0 tags: - base_model:adapter:unsloth/gpt-oss-20b-unsloth-bnb-4bit - lora - sft - preference-derived-sft - transformers - trl - unsloth - epistemic-maturity - 42ndalignment --- # 42ndAlignment gpt-oss-20b Preference-SFT LoRA v0.2 This is a LoRA adapter for `unsloth/gpt-oss-20b-unsloth-bnb-4bit`, trained as part of the 42ndAlignment project. 42ndAlignment tests whether the Epistemic Octahedron framework can be operationalized into training data for epistemic maturity: reality contact, self-correction, contradiction handling, non-strawmanning, calibrated certainty, integrated dimension consideration, and agency-preserving usefulness. ## Model Details - Base model: `unsloth/gpt-oss-20b-unsloth-bnb-4bit` - Starting point: v0.1 SFT adapter - Adapter type: LoRA - Training method: preference-derived SFT - Training examples: 300 chosen responses from a DPO-style preference dataset - Project repo: https://github.com/raulmiguelparasiii/42ndAlignment - License: Apache-2.0 ## Method Note This adapter is not true DPO. True DPO was attempted using the free Colab T4 `gpt-oss + Unsloth + TRL` stack, but training failed with a dtype mismatch during the forward pass: ```text RuntimeError: expected mat1 and mat2 to have the same dtype, but got: float != c10::Half ``` As a practical fallback, v0.2 was trained on the chosen side of the DPO-style preference dataset using supervised fine-tuning. This preserves the preference-selected target responses, but it does not directly optimize a DPO loss. Do not describe this adapter as a DPO model. Describe it as preference-derived SFT. ## Intended Use This adapter is intended for research and evaluation of epistemic-maturity-oriented fine-tuning. The goal is to test whether an open-weight model can be steered toward more stable judgment behavior: directness, calibrated certainty, resistance to weak framing, contradiction handling, non-strawmanning, and reduced sycophancy. This is not a production safety model. ## Training Data The adapter was trained using the chosen responses from `epistemic_maturity_dpo_pairs_v1_300.jsonl`. The original preference dataset contains: ```text prompt chosen rejected chosen_score rejected_score preference_reason collapse_flags_rejected ``` Only the `chosen` side was used for this v0.2 fallback run. ## Evaluation This v0.2 adapter has not yet received a full public benchmark evaluation. The project currently includes preliminary v0.1 before/after evaluation artifacts in the GitHub repository. TruthfulQA, IFEval, sycophancy, and a larger Epistemic Maturity Benchmark are planned next. ## Limitations - Not true DPO. - Small dataset. - Preliminary evaluation only. - Not a production safety model. - May affect general capability. - Should be tested against matched baselines before strong claims. - Does not prove broad generalization or mature internal cognition. ## Loading This adapter was trained and tested through Unsloth. ```python from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( model_name="rmpiii/42ndAlignment-gpt-oss-20b-pref-sft-lora-v0.2", max_seq_length=512, dtype=None, load_in_4bit=True, full_finetuning=False, ) ``` ## Project GitHub repository: https://github.com/raulmiguelparasiii/42ndAlignment ## Citation / Attribution Theory, rubric, datasets, and adapter work by Raul Miguel P. Paras III.
