ModelVerificadoExterno

Wmt22 Cometkiwi Da Pruned K4

A compressed version of Unbabel/wmt22-cometkiwi-da — a reference-free machine-translation quality estimation model (source + MT only, no human reference required).

Fontehuggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4/tree/26f935a351631b954a5f224cd0da766b9af2dacd

TipoOtherModelo baseUnbabel/wmt22-cometkiwi-daVersãoRevision 26f935aPublicador@solailabsCnota

Fonte do modelo

Tipo: Other
Tamanho do arquivo: 1,92 GB
Modelo base: Unbabel/wmt22-cometkiwi-da
Versão: vRevision 26f935a
Atualizado: 20 de out. de 2026
Publicador: @solailabs
Downloads: 16
Fonte: Hugging Face

Descrição da fonte

wmt22-cometkiwi-da-pruned-k4

A compressed version of Unbabel/wmt22-cometkiwi-da — a reference-free machine-translation quality estimation model (source + MT only, no human reference required).

Aggressive pruned variant — 4 layers removed. Trades ~8 pt of human Pearson for a smaller model.

What's different from the base model

4 encoder layers dropped (indices 4, 5, 6, 7) out of 24. Layer selection by cosine similarity between each layer's input and output hidden states on a small multilingual calibration set.
layerwise_attention rebuilt to mix only the surviving layers (embeddings + kept layer outputs).
No quantization — encoder weights remain fp32.

Accuracy

Benchmarked on 1200 stratified segments from RicardoRei/wmt-da-human-evaluation (reference-free, src+mt only):

Metric	This variant	Full cometkiwi
Pearson r vs human DA	0.5642	0.6402
Spearman vs human DA	0.5832	0.6698
Pearson r vs full	0.8316	1.0000
MAE vs full	0.0932	0.0000
Params	514.8M	565.1M
On-disk size	~2059 MB	~2200 MB

All variants at a glance

Variant	Pearson(human)	Pearson(full)	Size	When to use
full base	0.6402	1.0000	~2200 MB	reference quality
`-int8`	0.6404	0.9919	~1300 MB	lossless compression
`-pruned-k2`	0.6300	0.9784	~2100 MB	best-quality pruned
`-pruned-k4`	0.5642	0.8316	~2060 MB	aggressive prune
`-pruned-k4-xs`	0.5544	0.8113	~1030 MB	smallest footprint

Usage

Standalone — no gated base-model download. The repo ships everything the loader needs (hparams.yaml + state_dict.pt); the loader instantiates an empty COMET architecture via load_pretrained_weights=False and overlays the fine-tuned weights. Only the ungated microsoft/infoxlm-large tokenizer/config (~5 MB) is fetched on first load and cached.

# pip install "unbabel-comet" "setuptools<81" huggingface_hub pyyaml

from huggingface_hub import snapshot_download
import sys
folder = snapshot_download(repo_id="solailabs/wmt22-cometkiwi-da-pruned-k4")
sys.path.insert(0, folder)
from load import load_model

model = load_model(folder)
out = model.predict(
    [{{"src": "The meeting has been postponed until next week.",
       "mt":  "La réunion a été reportée à la semaine prochaine."}}],
    batch_size=8, gpus=0, progress_bar=False, num_workers=2,
)
print(out["scores"])

No HF_TOKEN required. No license acceptance on Unbabel/wmt22-cometkiwi-da needed.

Files

state_dict.pt — model weights (fp32 for -pruned-k2 / -pruned-k4, fp16 for -int8 / -pruned-k4-xs)
hparams.yaml — COMET hyper-parameters (encoder model, regressor shape, loss config)
config.json — kept/dropped layer indices, quant flag, benchmarked accuracy
load.py — drop-in standalone loader
README.md — this file

Citation

Base model: Unbabel/wmt22-cometkiwi-da by Unbabel.

@inproceedings{{rei-etal-2022-cometkiwi,
    title = "{{C}}omet{{K}}iwi: {{IST}}-{{U}}nbabel 2022 Submission for the Quality Estimation Shared Task",
    author = "Rei, Ricardo  and others",
    booktitle = "WMT 2022",
}}

Released under the same license as the base model (Apache 2.0).

Fontes

1 fonte

Hugging FacePrimárioVersão 26f935a351631b954a5f224cd0da766b9af2dacd

huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4/tree/26f935a351631b954a5f224cd0da766b9af2dacd

Verificado 8 de ago.

Artefatos de modelo

1 artefato

state_dict.pt

pt · 1,92 GB · SHA-256 24122e85c343…be88 · Hugging Face

Baixar

Trechos de fonte

3 trechos

Readme11 de jul.huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4

--- license: apache-2.0 language: - multilingual tags: - translation - quality-estimation - reference-free - comet - cometkiwi - pruning base_model: Unbabel/wmt22-cometkiwi-da pipeline_tag: translation --- # wmt22-cometkiwi-da-pruned-k4 A compressed version of [Unbabel/wmt22-cometkiwi-da](https://huggingface.co/Unbabel/wmt22-cometkiwi-da) — a reference-free machine-translation quality estimation model (source + MT only, no human reference required). **Aggressive pruned variant** — 4 layers removed. Trades ~8 pt of human Pearson for a smaller model. ## What's different from the base model - ****4 encoder layers dropped** (indices 4, 5, 6, 7) out of 24. Layer selection by cosine similarity between each layer's input and output hidden states on a small multilingual calibration set.** - `layerwise_attention` rebuilt to mix only the surviving layers (embeddings + kept layer outputs). - No quantization — encoder weights remain fp32. ## Accuracy Benchmarked on 1200 stratified segments from [RicardoRei/wmt-da-human-evaluation](https://huggingface.co/datasets/RicardoRei/wmt-da-human-evaluation) (reference-free, src+mt only): | Metric | This variant | Full cometkiwi | |---|---|---| | Pearson r vs human DA | **0.5642** | 0.6402 | | Spearman vs human DA | **0.5832** | 0.6698 | | Pearson r vs full | **0.8316** | 1.0000 | | MAE vs full | **0.0932** | 0.0000 | | Params | **514.8M** | 565.1M | | On-disk size | **~2059 MB** | ~2200 MB | ### All variants at a glance | Variant | Pearson(human) | Pearson(full) | Size | When to use | |---|---|---|---|---| | [full base](https://huggingface.co/Unbabel/wmt22-cometkiwi-da) | 0.6402 | 1.0000 | ~2200 MB | reference quality | | [`-int8`](https://huggingface.co/solailabs/wmt22-cometkiwi-da-int8) | **0.6404** | 0.9919 | ~1300 MB | **lossless compression** | | [`-pruned-k2`](https://huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k2) | **0.6300** | 0.9784 | ~2100 MB | best-quality pruned | | [`-pruned-k4`](https://huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4) | 0.5642 | 0.8316 | ~2060 MB | aggressive prune | | [`-pruned-k4-xs`](https://huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4-xs) | 0.5544 | 0.8113 | ~1030 MB | smallest footprint | ## Usage **Standalone — no gated base-model download.** The repo ships everything the loader needs (`hparams.yaml` + `state_dict.pt`); the loader instantiates an empty COM...

Note8 de ago.huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4

Source context: 16 downloads · 0 likes · Pipeline translation · Library comet · Repo solailabs/wmt22-cometkiwi-da-pruned-k4

Note16 de jul.huggingface.co/solailabs/wmt22-cometkiwi-da-pruned-k4

Source context: 18 downloads · 0 likes · Pipeline translation · Library comet · Repo solailabs/wmt22-cometkiwi-da-pruned-k4