Hopcoder Mini 9B Native ToolCall LoRA H200

Per-tool breakdown

Failure analysis (3 cases)

Native Tool-Call Format

The adapter emits tool calls in a compact XML-like format using special tokens:

VALUE
VALUE

Key characteristics:

• No JSON wrapping — parameters are individual XML-like tags, not a JSON object
• No markdown fences — output is pure tool-call blocks
• Balanced tags — every opening tag has a matching closing tag
• Arrays and objects — JSON is used only for complex parameter values (arrays/objects)
• Multiple calls — the model can emit multiple blocks in sequence

Example output

**/*.py

Parsing the output

import re

FUNCTION_RE = re.compile(
r" \s* \s*"
r"(.*?)\s* \s* ",
flags=re.DOTALL,
)
PARAMETER_RE = re.compile(
r" \s*"
r"(.*?)\s* ",
flags=re.DOTALL,
)

def parse_tool_calls(text):
calls = []
for match in FUNCTION_RE.finditer(text):
function_name = match.group(1)
body = match.group(2)
params = {}
for key, value in PARAMETER_RE.findall(body):
value = value.strip()
if value.startswith("[") or value.startswith("{"):
import json
params[key] = json.loads(value)
else:
params[key] = value
calls.append({"function": function_name, "parameters": params})
return calls

Model Details

Training Details

Hardware

Training data

Targeted CLI tools (8)

These are the real CLI agent tools the adapter was specifically trained on:

1. ask_user_question — Show interactive questions in the CLI
2. todo_write — Create or update a structured task list
3. read_file — Read a UTF-8 text file
4. search_code — Search source files for a text or regex pattern
5. glob — Find files by glob pattern
6. grep_search — Search file contents for a regex pattern
7. edit — Replace text in a file with new content
8. run_shell_command — Execute a shell command and return output

Training hyperparameters

Training metrics

Tool prompt format

The system prompt uses compact tool signatures instead of verbose JSON schemas:

Find files by glob pattern (e.g., **/*.py).
Search file contents for a regex pattern.

• Required parameters have no suffix; optional parameters are marked with ?
• Type annotations are compact (e.g., array[object{label,description}])
• Descriptions are truncated to 120 characters (tools) / 60 characters (parameters)

How to Use

Installation

pip install torch transformers peft

Quick start

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

MODEL_ID = "TaimoorSiddiqui/Hopcoder-Mini-9B"
ADAPTER_ID = "TaimoorSiddiqui/Hopcoder-Mini-9B-Native-ToolCall-LoRA-H200"

processor = AutoProcessor.from_pretrained(MODEL_ID, trust_remote_code=True)
tokenizer = processor.tokenizer
if tokenizer.pad_token_id is None:
tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForImageTextToText.from_pretrained(
MODEL_ID,
trust_remote_code=True,
dtype=torch.bfloat16,
device_map="auto",
)
model = PeftModel.from_pretrained(model, ADAPTER_ID)
model.eval()

Generating a tool call

SYSTEM_PROMPT = (
"Use the provided tools whenever the request requires one.\n\n"
"For a tool request, emit only complete native tool-call blocks. "
"Never emit a function name as a top-level tag. Never leave unmatched "
"parameter, function, or tool_call tags. Arrays and objects inside "
"parameter blocks must be valid JSON. Do not use Markdown fences.\n\n"
)

TOOLS_XML = (
" \n"
" "
"Find files by glob pattern (e.g., **/*.py). \n"
" "
)

messages = [
{"role": "system", "content": SYSTEM_PROMPT + TOOLS_XML},
{"role": "user", "content": "Find all Python files in the project."},
]

prompt = processor.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
enable_thinking=False,
)

inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)

with torch.inference_mode():
outputs = model.generate(
**inputs,
max_new_tokens=384,
do_sample=False,
repetition_penalty=1.05,
eos_token_id=tokenizer.eos_token_id,
pad_token_id=tokenizer.pad_token_id,
)

generated = outputs[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(generated, skip_special_tokens=True).strip())

Generation parameters

Benchmark

The benchmark evaluates the adapter on 40 cases:

• 30 targeted cases — 5 per CLI tool x 6 tools (ask_user_question, todo_write, glob, grep_search, edit, run_shell_command)
• 10 general cases — xLAM-style queries with 10 different tools (get_weather, search_flights, calculate_mortgage, send_email, book_restaurant, get_stock_price, create_event, translate_text, get_directions, set_reminder)

Validation criteria

Each generated tool call is validated on:

1. Complete tool-call block — must contain at least one valid block
2. No extra prose — no text outside tool-call blocks
3. No markdown fences — no code blocks
4. Balanced tags — matching counts of opening/closing tags for tool_call, function, and parameter
5. Correct function name — the called function matches the expected one
6. Valid JSON — array/object parameter values must be valid JSON
7. Required parameters — all required parameters must be present
8. No top-level function tags — function name must not appear as a standalone XML tag

Running the benchmark

The benchmark runs on Modal with an H200 GPU:

python -m modal run --detach hopcoder_benchmark.py

The benchmark script (hopcoder_benchmark.py) is included in this repository.

Limitations

1. Function confusion — The model occasionally confuses similar tools (e.g., search_code vs grep_search, read_file vs edit)
2. Missing parameters — Rare cases of omitting required parameters for complex tools
3. Single-turn only — The adapter was trained on single-turn examples; multi-turn conversations may require additional fine-tuning
4. CLI-focused — The 8 targeted tools are CLI agent tools; the adapter has not been tested with real-world API tools
5. Compact schema format — The system prompt uses a compact XML-like tool signature format, not standard JSON schemas. This may not be compatible with all tool-calling frameworks

Training Infrastructure

Files

Citation

@misc{hopcoder-mini-9b-native-toolcall-lora-h200,
author = {Taimoor Siddiqui},
title = {HopCoder-Mini-9B Native Tool-Call LoRA Adapter (H200)},
year = {2026},
publisher = {HuggingFace},
url = {https://huggingface.co/TaimoorSiddiqui/Hopcoder-Mini-9B-Native-ToolCall-LoRA-H200}
}

License

Apache 2.0