Hs Roberta v2 Onnx

hs-roberta-v2-onnx

A 3-class English hate-speech / offensive-language classifier. RoBERTa fine-tuned with LoRA on the Davidson dataset, then merged and exported to ONNX (fp32 + dynamic int8) so it's small enough to run entirely in the browser via transformers.js.

Live demo: https://karpit0499.github.io/hate-speech-adv-v2/ (classification runs client-side — text never leaves the device)

Labels

id2label (Davidson ClassLabel order):

ID	Label	Meaning
0	hate speech	Attacks or dehumanizes people based on a protected attribute (religion, ethnicity, gender, sexual orientation, disability, etc.).
1	offensive	Rude, insulting, or profane, but not targeting a protected group. In this dataset the class is defined largely by profanity.
2	neither	Neutral, positive, or otherwise fine.

Model description

• Base model: unitary/unbiased-toxic-roberta (a RobertaForSequenceClassification with 16 multi-label toxicity heads).
• Adaptation: LoRA fine-tuning with the 16-head classifier swapped for a fresh 3-class head (ignore_mismatched_sizes=True), LoRA applied to the query and value projections, and the new classifier head trained via modules_to_save.
• Export: merged LoRA weights → ONNX (Optimum), with dynamic int8 quantization, arranged into the onnx/ subfolder that transformers.js expects. Quantized weights are ~125 MB (which is why this lives on the Hub, not in a GitHub repo).

This repo is the ONNX export. The full-precision PyTorch model is at kmrarpit2704/hs-roberta-v2.

How to use

Browser / Node (transformers.js):

import { pipeline } from '@huggingface/transformers';

const classify = await pipeline('text-classification', 'kmrarpit2704/hs-roberta-v2-onnx');
const result = await classify('put your sentence here');
console.log(result); // [{ label: 'offensive', score: 0.97 }]

Python (ONNX Runtime via Optimum):

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, pipeline

model = ORTModelForSequenceClassification.from_pretrained("kmrarpit2704/hs-roberta-v2-onnx")
tokenizer = AutoTokenizer.from_pretrained("kmrarpit2704/hs-roberta-v2-onnx")

clf = pipeline("text-classification", model=model, tokenizer=tokenizer)
print(clf("put your sentence here"))

Training data

tdavidson/hate_speech_offensive (Davidson et al., 2017) — ~25k English tweets labelled hate speech / offensive language / neither. Split roughly 80/10/10 (stratified), with a held-out test set of 2,479 examples.

Evaluation

Reported on the held-out Davidson test set (2,479 examples).

Class	Precision	Recall	F1	Support
hate speech	0.421	0.629	0.504	143
offensive	0.967	0.913	0.939	1,919
neither	0.846	0.921	0.882	417
accuracy			0.898	2,479
macro avg	0.745	0.821	0.775	2,479
weighted avg	0.915	0.898	0.905	2,479

Confusion matrix (rows = true, columns = predicted):

                 pred:hate  pred:off  pred:neither
true:hate           90         38          15
true:offensive     112       1752          55
true:neither        12         21         384

Limitations & bias

• hate speech is the weak class (F1 0.504). It is both under-detected (recall 0.629) and over-predicted (precision 0.421), with most of the error being confusion with offensive — the well-known hate↔offensive boundary problem in this dataset. Do not rely on this model to reliably catch hate speech in isolation.
• The offensive class is profanity-driven. Because the Davidson offensive label is defined largely by swearing, profanity-free insults tend to be classified as neither. This is expected behaviour, not a bug, but it means the model is a poor fit for "polite" harassment.
• Documented dataset bias. Models trained on the Davidson dataset are known in the literature to over-flag African American English and in-group/reclaimed language as offensive or hateful. Expect this failure mode.
• English-only, short-text, tweet-domain. Accuracy drops on longer text, formal registers, or other domains and languages.
• Not a moderation decision-maker. Treat the output as a signal for human review, not an automated verdict, and never as the basis for punitive action.

License

Released under the MIT License.