Contract Drafting Assistant v2

# Contract Drafting Assistant — Complete System A RAG-augmented LLM pipeline for drafting full legal contracts from deal context, party position, and business constraints. ## System Architecture ``` Deal Context → Playbook Rules → Dense Retriever (37K clauses) → SaulLM-7B (4-bit) → NLI Verifier → Final Contract ↑ ↑ ↑ ↑ Party Position Clause Dependency Graph Precedent Clauses Per-Clause NLI ``` ## Supported Contract Types - SaaS Agreement - Master Services Agreement (MSA) - Non-Disclosure Agreement (NDA) - Data Processing Agreement (DPA) - Statement of Work (SOW) - Vendor Agreement - Consulting Agreement - IP Assignment - Employment/Contractor Agreement ## Key Components ### Playbook Rules (`playbook.py`) Encodes expert knowledge as structured rules: - **Required clauses** per contract type (9 contract types, ~12 clauses each) - **Fallback positions** per clause × party position (pro_company, balanced, pro_counterparty) - **Risk flags** with severity levels (HIGH/MEDIUM) - **Drafting checklist** per contract type (~8-10 items each) ### Dense Retriever (`dense_retriever.py`) Hybrid BM25 + cosine similarity over 37,508 real contract clauses from EDGAR/ALM. Used to provide precedent examples when drafting each clause. ### Clause Dependency Graph (`clause_graph.py`) 14 directed edges modeling how clauses interact: - CONSTRAINS: liability cap limits indemnification - REQUIRES: termination needs defined breach conditions - TRIGGERS: data protection activates subprocessor rules - MATCHES: governing law must match arbitration seat ### NLI Verifier Per-clause entailment checking against 100+ legal hypotheses using roberta-large-mnli. Future: swap to contract-specific NLI (contract-nli-v1). ### Fine-tuned Model (`contract-drafter-v1`) Saul-7B-Instruct-v1 fine-tuned on 126 contracting examples across 9 types. See [narcolepticchicken/contract-drafter-v1](https://huggingface.co/narcolepticchicken/contract-drafter-v1). ## Evaluation Results Evaluated on 7 gold-standard drafting tasks with rubric scoring across 8 metrics. | Version | Model | Overall | NLI | Semantic Recall | |---------|-------|:-------:|:---:|:---------------:| | V11 | Base SaulLM-7B | 0.646 | 0.553 | 0.167 | | V12 (running) | Fine-tuned | TBD | TBD | TBD | See [contract-drafts-v1](https://huggingface.co/datasets/narcolepticchicken/contract-drafts-v1) for full results. ## Known Issues (V11) 1. **key_term_recall_semantic = 0.167**: NLI truncates full contract to 2000 chars 2. **roberta-large-mnli**: General-purpose NLI misses legal entailments (~55% precision) 3. **Semantic recall**: Only checks first ~4 clauses, misses terms in later clauses ## Planned Improvements 1. Per-clause key term matching (+0.086 overall) 2. Swap to contract-nli-v1 (+0.022 overall) 3. Lower eval temperature to 0.1 for cleaner measurements 4. Seed deal memory for counterparty history ## Hub Assets | Repo | Type | Description | |------|------|-------------| | [contract-drafter-v1](https://huggingface.co/narcolepticchicken/contract-drafter-v1) | Model | Fine-tuned drafting model (3.8GB) | | [contract-clause-index-v1](https://huggingface.co/datasets/narcolepticchicken/contract-clause-index-v1) | Dataset | 37.5K clauses + embeddings | | [contract-nli-v1](https://huggingface.co/narcolepticchicken/contract-nli-v1) | Model | Legal-domain NLI | | [contract-drafts-v1](https://huggingface.co/datasets/narcolepticchicken/contract-drafts-v1) | Dataset | Eval results (v8-v12) | | [contract-drafting-assistant](https://huggingface.co/narcolepticchicken/contract-drafting-assistant) | Model | Playbook + fallback positions | | [contract-drafting-assistant](https://huggingface.co/spaces/narcolepticchicken/contract-drafting-assistant) | Space | Interactive Gradio UI | ## Usage ```python from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained( "narcolepticchicken/contract-drafter-v1", device_map="auto", load_in_4bit=True, ) tokenizer = AutoTokenizer.from_pretrained("narcolepticchicken/contract-drafter-v1") # See run_v12.py for the full drafting pipeline ```