Herculaneum Legibility Proxy

Herculaneum legibility proxy (proxy_v4 / proxy_s2s3_v1)

A small ResNet-18 binary classifier that scores ~1 cm windows of Vesuvius Challenge ink-probability maps for legible-text likelihood: does this window contain connected Greek letterforms, or just fiber texture / noise / damage?

It is a triage tool, not an ink detector. It runs downstream of the official ink-detection models, on their output maps, and answers a different question: "of all this detected ink, where should a human look first?"

Built and used to produce a full legibility index of Scroll 1 (PHerc. Paris 4): 125,298 windows scored across all 78 of Scroll 1's currently-published official ink maps (complete coverage, verified live against the S3 bucket 2026-07-08 — see "Coverage" below for how this grew from an initial 16-panel pass). 5,678 high-confidence text windows on the deduplicated wrap-series union (w010–w129); classic-lineage segments add 2,460 more on 18 physical surfaces (kept separate — their overlap with the series is unresolved), and 7,248 further gold windows come from redundant re-renders, counted only as consistency checks (see Coverage below). The same checkpoint, unmodified, was then run on three more scrolls with no fine-tuning in between:

Scroll	Windows scored	Segments/maps used	Coverage of official maps	Gold (≥0.9)	Independent check
Scroll 1 (PHerc. Paris 4)	125,298	78 official segments (16 original panels + 62 added 2026-07-08)	78 of 78 — complete	5,678 on the deduplicated wrap-series union w010–w129 (+2,460 on classic-lineage segments, kept separate; +7,248 redundant re-renders — see Coverage)	3 held-out panels, AUROC 0.985; wrap w028-037 independently gold-confirmed in 2 unrelated renders (50.9% and 40.0% gold density)
PHerc 0139	4,905	38	38 of 38 — complete	63	title segment ranked 2nd of 38 by gold density; 63/63 human-reviewed (single reviewer): 54 clear-text / 9 possible / 0 rejected
Scroll 5 (PHerc 172)	24,528	53 × 2 ink-detector models	53 of 53 — complete	3 (by july_retreat-model score; 0 by the more conservative november19 model — see cross-render caveat below)	max score 0.938 on a window with plainly visible, large uppercase Greek letters (segment w066)
Scroll 4 (PHerc 1667)	4,761	19 official maps	19 of 19 segments that carry an official ink map (1 of the 20 public segments has none) — complete	430	per-wrap gold fraction correlates at Spearman r = 0.843 (p < 0.0001, n = 18) with the count of Greek letters already transcribed in that wrap — ground truth external to this model

Coverage — how the Scroll 1 index reached 78/78

The index originally covered only 16 of Scroll 1's 81 public segments (one continuous panel series, w010–w100, uploaded 2026-06-23) — 29,853 windows, 4,491 gold. A live S3 check on 2026-07-08 found the scroll had grown to 78 populated official maps (3 of the 81 public segments still have none). The remaining 62 were scored with the identical unmodified pipeline (same proxy_v4.pt, same WIN=512/STRIDE=256 grid, same ds8-resolution preprocessing — verified byte-for-byte SHA256-identical against the official pre-made ink-detection/downsampled/*-ds8.jpg product before trusting it on new segments), split into four groups:

Group	Segments	Windows	Gold (≥0.9)	Territory
23 "classic"-lineage panels (name-level dedup: 18 physical surfaces)	23	8,978	2,684 (2,460 on representatives + 224 on re-renders)	separate bucket — overlap with the panel series unresolved, so never summed into the wrap-series denominator
11 more panels from the same June-2026 series, w101–w129	11	14,998	1,056	distinct — extends past the original w100 boundary
2 alternate-processing panels, w046-052_jordi / w053-058_jordi	2	3,929	739	overlaps — re-renders of w046-052/w053-058, already in the original 16 panels (confirmed by near-identical gold-density-by-row fingerprint against the originals)
26 panels from a second complete series uploaded 2026-07-01, covering w010–w129	26	67,540	6,416	one panel is unique territory: the June w104-106 segment has no published ink map; the July w104-106 panel (2,648 windows, 131 gold) covers that range and joins the deduplicated union. The other 25 panels (6,285 gold) re-cover indexed wraps and stay redundant

Correction (2026-07-08, caught in adversarial review): the _jordi panels were originally folded into the "distinct" count because their segment IDs looked like part of the w101–w129 extension. They aren't — they cover wrap ranges already present in the original 16-panel index. Moved to the overlap bucket; the headline distinct-territory number dropped from an incorrect 8,970 to a correct 8,231 as a result. The July-2026 series is confirmed not a byte-identical duplicate of the June one (e.g. the w010-027 file is 682 MB in June vs 1.07 GB in July, same recipe tag), but it covers the same physical wraps, so — like the _jordi panels — its gold windows are reported separately rather than folded into the "distinct territory" total, to avoid double-counting the same underlying text. [Update, 2026-07-11: with one exception found later — the July w104-106 panel is unique territory; see the correction below.] Both overlap groups are useful anyway as independent consistency checks: wrap range w028-037 (June vs July) scores gold at 50.9%/40.0%, and w046-052/w053-058 (original vs _jordi) show matching gold-density fingerprints band-by-band — unrelated renders of the same regions agreeing on where the text is.

Correction (2026-07-11, peer review): an earlier version of this card reported "8,231 gold on distinct territory" by summing the wrap-series panels with the classic segments. Two problems, both fixed here: the June w104-106 segment has no published ink map, so the July panel covering those wraps is unique territory (131 gold) wrongly excluded as redundant; and the classic bucket contained re-renders of the same physical surfaces (a _copy, a _v14, a _v2_flatboi, a _v8, an offset-0 re-render — 23 panels are 18 surfaces) and its physical overlap with the panel series is unresolved. Canonical accounting: deduplicated wrap-series union (June w010–w129 + July w104-106) = 5,678 gold windows; classic-lineage representatives = 2,460 gold on 18 surfaces, reported separately; redundant re-renders = 7,248 gold (224 classic + 739 _jordi + 6,285 July), used only as consistency checks. Cross-check invariant: 5,678 + 2,460 + 224 + 739 + 6,285 = 15,386 gold rows in the full 125,298-window index — shipped as full_index_complete.json in the companion dataset repo (SHA256 4d393d70ce886ed62b7e73e365f1d01cbe7f6efa37168fb3f27ade2b89d6e7a8; every bucket and denominator above is recomputed with asserts by its summarize_index.py). Note the score key in that file is named v3 for legacy reasons; every score in it was produced by proxy_v4.pt.

Files

File	What it is	MD5 (first 12)
proxy_v4.pt	main checkpoint, trained on official Scroll 1 panel maps	c869ce189f2a
proxy_s2s3_v1.pt	domain-calibrated variant: same S1 knowledge + 192 windows of self-rendered Scroll 2/3 maps as explicit negatives	95b6839c58c8

Both are plain state_dicts for torchvision.models.resnet18() with fc = nn.Linear(512, 1) (122 tensors).

Which one to use — the split is by rendering pipeline, not by scroll

proxy_v4 generalizes across scrolls, as long as the map came out of the official/community rendering pipeline (the texture statistics of an "official-style" ink map are consistent regardless of which scroll it's of). See the table above for the four-scroll validation: one native, one blind positive control (0139), one cross-render-calibration caveat that's about the ink detector, not this classifier (Scroll 5), and one check against fully independent, non-visual ground truth — transcribed-letter counts, not just human eyeballing (Scroll 4; project-internal validation note — the correlation, n, and p reported above are the complete result).

proxy_s2s3_v1 exists because that assumption breaks on maps rendered by a custom / non-official pipeline — which for us happened to be Scrolls 2 and 3, only because no official ink maps existed yet for the regions we needed and we rendered our own. On those self-rendered mosaics, proxy_v4 alone gives an inflated background-noise floor (~0.04); the 192 self-rendered-domain negatives used to fine-tune proxy_s2s3_v1 drop that to ~0.003 without hurting Scroll 1 performance (AUROC 0.984 vs 0.985). It was never evaluated on official maps of other scrolls (0139, Scroll 5) — there's no reason to prefer it over proxy_v4 there, since proxy_v4 already has evidence of working on that domain and proxy_s2s3_v1 doesn't.

Map came from	Use
Official / community rendering pipeline (any scroll)	proxy_v4.pt
Your own custom rendering pipeline	proxy_s2s3_v1.pt, or better: fine-tune your own domain-negatives the same way (see Training summary)

How to use

The preprocessing is part of the model. Deviate from it and the scores shift silently — this exact recipe is what the training used:

import numpy as np
import torch, torch.nn as nn, torchvision
from PIL import Image

m = torchvision.models.resnet18()
m.fc = nn.Linear(512, 1)
m.load_state_dict(torch.load("proxy_v4.pt", map_location="cpu"))
m.eval()

def score_window(win_u8):          # win_u8: 2D uint8 crop of an ink map
    c = win_u8.astype(np.float32)
    active = c[c > 10]
    if len(active) > 50:           # percentile stretch over active pixels
        lo, hi = np.percentile(active, [2, 99.5])
        c = np.clip((c - lo) / max(hi - lo, 1e-6), 0, 1)
    else:
        c = c / 255.0
    img = Image.fromarray((c * 255).astype(np.uint8)).resize((160, 160), Image.BILINEAR)
    t = torch.from_numpy(np.array(img, np.float32) / 255.0)[None].repeat(3, 1, 1)
    with torch.no_grad():
        return torch.sigmoid(m(t[None])).item()

The window must be ~1 cm physical — this is the #1 usage error

The model was trained on 512 px windows of ds8 panels at ≈19.2 μm/px ≈ 9,830 μm ≈ 1 cm per window. On any other map, compute the window size from the segment's real physical scale, e.g.:

px_um = (area_cm2 * 1e8 / (H * W)) ** 0.5   # from the mesh meta.json area
WIN   = round(9830 / px_um)

Two failure modes we hit ourselves, so you don't have to:

1. Do not assume 512 px. On a 7.91 μm-scan ds8 map the right window is ~155 px; on a 2.4 μm-scan ds8 map it's ~512–560 px.
2. Do not stretch undersized crops. If a fragment is smaller than WIN, resizing what you have up to 160×160 changes the fiber texture's apparent frequency and inflates scores (we measured score anti-correlating with true window size, r ≈ −0.45, before fixing this). Pad onto a black 160×160 canvas at the training scale (61.4 μm per classifier pixel) instead, and treat such windows with extra suspicion.

Thresholds we used

• ≥ 0.9 — "gold": on Scroll 1, round-2 human review confirmed 120/120 of the model's top-ranked proposals. On PHerc 0139, all 63 gold windows were human-reviewed (single reviewer): 54 clear text (85.7%), 9 possible, 0 rejected as noise/artifact. These are one reviewer's visual verdicts on model-selected windows, not population precision estimates.
• 0.35–0.70 — genuinely ambiguous band; useful for uncertainty sampling if you're fine-tuning.
• Scores are not calibrated probabilities, and they do not transfer across ink-model renders: on Scroll 5, the same three candidate windows (segment w066...