ModelVerificadoExterno

Wmt22 Cometkiwi Da Pruned K4 Xs

A compressed version of Unbabel/wmt22-cometkiwi-da — a reference-free machine-translation quality estimation model (source + MT only, no human reference required).

Fontehuggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4-xs/tree/a56501f7eed4909d12328c46f2d1d36a74e28865

TipoOtherModelo baseUnbabel/wmt22-cometkiwi-daVersãoRevision a56501fPublicador@solailabsCnota

Fonte do modelo

Tipo: Other
Tamanho do arquivo: 982 MB
Modelo base: Unbabel/wmt22-cometkiwi-da
Versão: vRevision a56501f
Atualizado: 1 de out. de 2026
Publicador: @solailabs
Downloads: 6
Fonte: Hugging Face

Descrição da fonte

wmt22-cometkiwi-da-pruned-k4-xs

A compressed version of Unbabel/wmt22-cometkiwi-da — a reference-free machine-translation quality estimation model (source + MT only, no human reference required).

Smallest variant — K=4 prune + int8 + fp16 storage, ~1 GB on disk. Aggressive quality/size trade-off.

What's different from the base model

4 encoder layers dropped (indices 4, 5, 6, 7) out of 24. Layer selection by cosine similarity between each layer's input and output hidden states on a small multilingual calibration set.
layerwise_attention rebuilt to mix only the surviving layers (embeddings + kept layer outputs).
Dynamic int8 quantization on the XLM-R encoder + fp16 storage (cast back to fp32 at load before quant).

Accuracy

Benchmarked on 1200 stratified segments from RicardoRei/wmt-da-human-evaluation (reference-free, src+mt only):

Metric	This variant	Full cometkiwi
Pearson r vs human DA	0.5544	0.6402
Spearman vs human DA	0.5727	0.6698
Pearson r vs full	0.8113	1.0000
MAE vs full	0.0994	0.0000
Params	514.8M	565.1M
On-disk size	~1030 MB	~2200 MB

All variants at a glance

Variant	Pearson(human)	Pearson(full)	Size	When to use
full base	0.6402	1.0000	~2200 MB	reference quality
`-int8`	0.6404	0.9919	~1300 MB	lossless compression
`-pruned-k2`	0.6300	0.9784	~2100 MB	best-quality pruned
`-pruned-k4`	0.5642	0.8316	~2060 MB	aggressive prune
`-pruned-k4-xs`	0.5544	0.8113	~1030 MB	smallest footprint

Usage

Standalone — no gated base-model download. The repo ships everything the loader needs (hparams.yaml + state_dict.pt); the loader instantiates an empty COMET architecture via load_pretrained_weights=False and overlays the fine-tuned weights. Only the ungated microsoft/infoxlm-large tokenizer/config (~5 MB) is fetched on first load and cached.

# pip install "unbabel-comet" "setuptools<81" huggingface_hub pyyaml

from huggingface_hub import snapshot_download
import sys
folder = snapshot_download(repo_id="solailabs/wmt22-cometkiwi-da-pruned-k4-xs")
sys.path.insert(0, folder)
from load import load_model

model = load_model(folder)
out = model.predict(
    [{{"src": "The meeting has been postponed until next week.",
       "mt":  "La réunion a été reportée à la semaine prochaine."}}],
    batch_size=8, gpus=0, progress_bar=False, num_workers=2,
)
print(out["scores"])

No HF_TOKEN required. No license acceptance on Unbabel/wmt22-cometkiwi-da needed.

Files

state_dict.pt — model weights (fp32 for -pruned-k2 / -pruned-k4, fp16 for -int8 / -pruned-k4-xs)
hparams.yaml — COMET hyper-parameters (encoder model, regressor shape, loss config)
config.json — kept/dropped layer indices, quant flag, benchmarked accuracy
load.py — drop-in standalone loader
README.md — this file

Citation

Base model: Unbabel/wmt22-cometkiwi-da by Unbabel.

@inproceedings{{rei-etal-2022-cometkiwi,
    title = "{{C}}omet{{K}}iwi: {{IST}}-{{U}}nbabel 2022 Submission for the Quality Estimation Shared Task",
    author = "Rei, Ricardo  and others",
    booktitle = "WMT 2022",
}}

Released under the same license as the base model (Apache 2.0).

Fontes

1 fonte

Hugging FacePrimárioVersão a56501f7eed4909d12328c46f2d1d36a74e28865

huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4-xs/tree/a56501f7eed4909d12328c46f2d1d36a74e28865

Verificado 1 de out.

Artefatos de modelo

1 artefato

state_dict.pt

pt · 982 MB · SHA-256 a0a11ded2cb8…fae5 · Hugging Face

Baixar

Trechos de fonte

3 trechos

Readme11 de jul.huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4-xs

--- license: apache-2.0 language: - multilingual tags: - translation - quality-estimation - reference-free - comet - cometkiwi - pruning base_model: Unbabel/wmt22-cometkiwi-da pipeline_tag: translation --- # wmt22-cometkiwi-da-pruned-k4-xs A compressed version of [Unbabel/wmt22-cometkiwi-da](https://huggingface.co/Unbabel/wmt22-cometkiwi-da) — a reference-free machine-translation quality estimation model (source + MT only, no human reference required). **Smallest variant** — K=4 prune + int8 + fp16 storage, ~1 GB on disk. Aggressive quality/size trade-off. ## What's different from the base model - ****4 encoder layers dropped** (indices 4, 5, 6, 7) out of 24. Layer selection by cosine similarity between each layer's input and output hidden states on a small multilingual calibration set.** - `layerwise_attention` rebuilt to mix only the surviving layers (embeddings + kept layer outputs). - **Dynamic int8 quantization** on the XLM-R encoder + fp16 storage (cast back to fp32 at load before quant). ## Accuracy Benchmarked on 1200 stratified segments from [RicardoRei/wmt-da-human-evaluation](https://huggingface.co/datasets/RicardoRei/wmt-da-human-evaluation) (reference-free, src+mt only): | Metric | This variant | Full cometkiwi | |---|---|---| | Pearson r vs human DA | **0.5544** | 0.6402 | | Spearman vs human DA | **0.5727** | 0.6698 | | Pearson r vs full | **0.8113** | 1.0000 | | MAE vs full | **0.0994** | 0.0000 | | Params | **514.8M** | 565.1M | | On-disk size | **~1030 MB** | ~2200 MB | ### All variants at a glance | Variant | Pearson(human) | Pearson(full) | Size | When to use | |---|---|---|---|---| | [full base](https://huggingface.co/Unbabel/wmt22-cometkiwi-da) | 0.6402 | 1.0000 | ~2200 MB | reference quality | | [`-int8`](https://huggingface.co/solailabs/wmt22-cometkiwi-da-int8) | **0.6404** | 0.9919 | ~1300 MB | **lossless compression** | | [`-pruned-k2`](https://huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k2) | **0.6300** | 0.9784 | ~2100 MB | best-quality pruned | | [`-pruned-k4`](https://huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4) | 0.5642 | 0.8316 | ~2060 MB | aggressive prune | | [`-pruned-k4-xs`](https://huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4-xs) | 0.5544 | 0.8113 | ~1030 MB | smallest footprint | ## Usage **Standalone — no gated base-model download.** The repo ships everything the loader needs (`h...

Note1 de out.huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4-xs

Source context: 6 downloads · 0 likes · Pipeline translation · Library comet · Repo solailabs/wmt22-cometkiwi-da-pruned-k4-xs

Note16 de jul.huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4-xs

Source context: 2 downloads · 0 likes · Pipeline translation · Library comet · Repo solailabs/wmt22-cometkiwi-da-pruned-k4-xs