Gus Net Bert

GUS-Net (BERT) Token-level built on . Given a sentence, it tags each token with one of four bias categories following a 7-label BIO scheme, so a downstream system can highlight carry bias, not just whether a sentence is biased. This is the flagship / default checkpoint of the project (a master's thesis on interpretable bias detection through transformer attention). This published checkpoint is the training run. - - multi-label token classification (per-token sigmoid, thresholded) - English - , , ## Label scheme The model outputs 7 BIO labels. , and are the three bias categories; is "no bias". | Index | Label | Category | |-------|-----------|-----------------------------------| | 0 | O | none | | 1 | B-STEREO | Stereotype (span start) | | 2 | I-STEREO | Stereotype (span inside) | | 3 | B-GEN | Generalisation (span start) | | 4 | I-GEN | Generalisation (span inside) | | 5 | B-UNFAIR | Unfair language (span start) | | 6 | I-UNFAIR | Unfair language (span inside) | - — a blanket generalisation about a group. - — unfair / disparaging language toward a group. - — a stereotype attributed to a group. ## Important: multi-label + per-label thresholds Outputs are (multi-label), not a softmax. Predictions are obtained by thresholding each label. The F1-optimised thresholds for this checkpoint (order ) are: Using a flat 0.5 threshold will under-detect bias — use the values above. ## Usage ## Training data Fine-tuned on the — a token-level social-bias corpus annotated for eneralisations, nfairness and tereotypes, from which this model's 7-label scheme is derived (). punctuation tokens that were annotated with bias labels in the original data were (relabelled to the non-bias class), since punctuation is not a social-bias carrier. Bias spans predicted by these models therefore exclude leading/trailing punctuation. ## Evaluation ### StereoSet (intersentence split, 2123 examples) | Metric | Score | |--------|-------| | LMS (language-modeling score, higher is better) | | | SS (stereotype score, ) | | | ICAT (bias-adjusted quality) | | Per-category SS: gender 58.68 · race 50.10 · religion 46.15 · profession 55.62. ### Token classification (GUS-Net held-out test set) Held-out 20% split (seed 42, 748 sentences) of , scored with the per-label thresholds above; each category aggregates its B-/I- labels. | Category | Precision | Recall | F1 | |----------|-----------|--------|----| | O (non-bias) | 0.834 | 0.915 | 0.873 | | GEN | 0.713 | 0.581 | 0.641 | | UNFAIR | 0.307 | 0.315 | 0.311 | | STEREO | 0.701 | 0.576 | 0.632 | | | 0.640 | 0.542 | 0.587 | ## Limitations & intended use - , not a content-moderation oracle. Predictions reflect a specific operationalisation of bias (clear generalisations, unfairness and stereotypes about a group); subtle, implicit or context-dependent bias may be missed. - English only. - Labels are not error-free; treat spans as evidence to review, not ground truth. - Do use for automated decisions about individuals. ## Citation If you use these models, please cite the GUS-Net dataset and benchmark: ## License Weights released under (matching the base model). The Attention Atlas code is MIT-licensed.