ModelVérifiéExterne

Wmt22 Cometkiwi Da Int8

A compressed version of Unbabel/wmt22-cometkiwi-da — a reference-free machine-translation quality estimation model (source + MT only, no human reference required).

Sourcehuggingface.co/solailabs/wmt22-cometkiwi-da-int8/tree/f8f1beb464dd6f836af9bbd51986f3a50356e85d

TypeOtherModèle de baseUnbabel/wmt22-cometkiwi-daVersionRevision f8f1bebÉditeur@solailabsCClasse

Source du modèle

Type: Other
Taille du fichier: 1,05 GB
Modèle de base: Unbabel/wmt22-cometkiwi-da
Version: vRevision f8f1beb
Mis à jour: 1 oct. 2026
Éditeur: @solailabs
Téléchargements: 28
Source: Hugging Face

Description de la source

wmt22-cometkiwi-da-int8

A compressed version of Unbabel/wmt22-cometkiwi-da — a reference-free machine-translation quality estimation model (source + MT only, no human reference required).

Lossless compression — zero human-Pearson loss, ~40% smaller on disk via int8 alone.

What's different from the base model

No layer pruning — all 24 XLM-R encoder layers retained. Compression comes entirely from dynamic int8 quantization + fp16 storage.
layerwise_attention rebuilt to mix only the surviving layers (embeddings + kept layer outputs).
Dynamic int8 quantization on the XLM-R encoder + fp16 storage (cast back to fp32 at load before quant). No layer pruning — all 24 encoder layers retained.

Accuracy

Benchmarked on 1200 stratified segments from RicardoRei/wmt-da-human-evaluation (reference-free, src+mt only):

Metric	This variant	Full cometkiwi
Pearson r vs human DA	0.6404	0.6402
Spearman vs human DA	0.6703	0.6698
Pearson r vs full	0.9919	1.0000
MAE vs full	0.0138	0.0000
Params	565.1M	565.1M
On-disk size	~1130 MB	~2200 MB

All variants at a glance

Variant	Pearson(human)	Pearson(full)	Size	When to use
full base	0.6402	1.0000	~2200 MB	reference quality
`-int8`	0.6404	0.9919	~1300 MB	lossless compression
`-pruned-k2`	0.6300	0.9784	~2100 MB	best-quality pruned
`-pruned-k4`	0.5642	0.8316	~2060 MB	aggressive prune
`-pruned-k4-xs`	0.5544	0.8113	~1030 MB	smallest footprint

Usage

Standalone — no gated base-model download. The repo ships everything the loader needs (hparams.yaml + state_dict.pt); the loader instantiates an empty COMET architecture via load_pretrained_weights=False and overlays the fine-tuned weights. Only the ungated microsoft/infoxlm-large tokenizer/config (~5 MB) is fetched on first load and cached.

# pip install "unbabel-comet" "setuptools<81" huggingface_hub pyyaml

from huggingface_hub import snapshot_download
import sys
folder = snapshot_download(repo_id="solailabs/wmt22-cometkiwi-da-int8")
sys.path.insert(0, folder)
from load import load_model

model = load_model(folder)
out = model.predict(
    [{{"src": "The meeting has been postponed until next week.",
       "mt":  "La réunion a été reportée à la semaine prochaine."}}],
    batch_size=8, gpus=0, progress_bar=False, num_workers=2,
)
print(out["scores"])

No HF_TOKEN required. No license acceptance on Unbabel/wmt22-cometkiwi-da needed.

Files

state_dict.pt — model weights (fp32 for -pruned-k2 / -pruned-k4, fp16 for -int8 / -pruned-k4-xs)
hparams.yaml — COMET hyper-parameters (encoder model, regressor shape, loss config)
config.json — kept/dropped layer indices, quant flag, benchmarked accuracy
load.py — drop-in standalone loader
README.md — this file

Citation

Base model: Unbabel/wmt22-cometkiwi-da by Unbabel.

@inproceedings{{rei-etal-2022-cometkiwi,
    title = "{{C}}omet{{K}}iwi: {{IST}}-{{U}}nbabel 2022 Submission for the Quality Estimation Shared Task",
    author = "Rei, Ricardo  and others",
    booktitle = "WMT 2022",
}}

Released under the same license as the base model (Apache 2.0).

Sources

1 source

Hugging FacePrincipalVersion f8f1beb464dd6f836af9bbd51986f3a50356e85d

huggingface.co/solailabs/wmt22-cometkiwi-da-int8/tree/f8f1beb464dd6f836af9bbd51986f3a50356e85d

Vérifié 1 oct.

Artefacts du modèle

1 artefact

state_dict.pt

pt · 1,05 GB · SHA-256 720b028f0fc0…b929 · Hugging Face

Télécharger

Extraits de sources

3 extraits

Readme11 juil.huggingface.co/solailabs/wmt22-cometkiwi-da-int8

--- license: apache-2.0 language: - multilingual tags: - translation - quality-estimation - reference-free - comet - cometkiwi - pruning base_model: Unbabel/wmt22-cometkiwi-da pipeline_tag: translation --- # wmt22-cometkiwi-da-int8 A compressed version of [Unbabel/wmt22-cometkiwi-da](https://huggingface.co/Unbabel/wmt22-cometkiwi-da) — a reference-free machine-translation quality estimation model (source + MT only, no human reference required). **Lossless compression** — zero human-Pearson loss, ~40% smaller on disk via int8 alone. ## What's different from the base model - ****No layer pruning** — all 24 XLM-R encoder layers retained. Compression comes entirely from dynamic int8 quantization + fp16 storage.** - `layerwise_attention` rebuilt to mix only the surviving layers (embeddings + kept layer outputs). - **Dynamic int8 quantization** on the XLM-R encoder + fp16 storage (cast back to fp32 at load before quant). No layer pruning — all 24 encoder layers retained. ## Accuracy Benchmarked on 1200 stratified segments from [RicardoRei/wmt-da-human-evaluation](https://huggingface.co/datasets/RicardoRei/wmt-da-human-evaluation) (reference-free, src+mt only): | Metric | This variant | Full cometkiwi | |---|---|---| | Pearson r vs human DA | **0.6404** | 0.6402 | | Spearman vs human DA | **0.6703** | 0.6698 | | Pearson r vs full | **0.9919** | 1.0000 | | MAE vs full | **0.0138** | 0.0000 | | Params | **565.1M** | 565.1M | | On-disk size | **~1130 MB** | ~2200 MB | ### All variants at a glance | Variant | Pearson(human) | Pearson(full) | Size | When to use | |---|---|---|---|---| | [full base](https://huggingface.co/Unbabel/wmt22-cometkiwi-da) | 0.6402 | 1.0000 | ~2200 MB | reference quality | | [`-int8`](https://huggingface.co/solailabs/wmt22-cometkiwi-da-int8) | **0.6404** | 0.9919 | ~1300 MB | **lossless compression** | | [`-pruned-k2`](https://huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k2) | **0.6300** | 0.9784 | ~2100 MB | best-quality pruned | | [`-pruned-k4`](https://huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4) | 0.5642 | 0.8316 | ~2060 MB | aggressive prune | | [`-pruned-k4-xs`](https://huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4-xs) | 0.5544 | 0.8113 | ~1030 MB | smallest footprint | ## Usage **Standalone — no gated base-model download.** The repo ships everything the loader needs (`hparams.yaml` + `state_dict.pt`...

Note1 oct.huggingface.co/solailabs/wmt22-cometkiwi-da-int8

Source context: 28 downloads · 0 likes · Pipeline translation · Library comet · Repo solailabs/wmt22-cometkiwi-da-int8

Note16 juil.huggingface.co/solailabs/wmt22-cometkiwi-da-int8

Source context: 8 downloads · 0 likes · Pipeline translation · Library comet · Repo solailabs/wmt22-cometkiwi-da-int8