A fitted Jacobian lens for zai-org/GLM-4-9B-0414 , from Anthropic's Verbalizable Representations Form a Global Workspace in Language Models , fitted with the reference implementation anthropics/jacobian-lens . Contents field value --- --- base model zai-org/GLM-4-9B-0414 (40 layers, d model=4096 ) source layers 0–38 (all 39 layers below the final) target layer 39 (final) Jacobian shape [4096,...
Model source
Source description
# Jacobian Lens for GLM-4-9B-0414 A fitted Jacobian lens for , from Anthropic's , fitted with the reference implementation . ## Contents | field | value | |---|---| | base model | (40 layers, ) | | source layers | 0–38 (all 39 layers below the final) | | target layer | 39 (final) | | Jacobian shape | per layer, fp16 on disk | | file size | ~1.22 GiB | | fitting corpus | 100 WikiText-103 prompts × 128 tokens (first 16 positions skipped as attention sinks) | | estimator | , mean over valid source positions | | fit hardware | 1× NVIDIA H100 80GB, ~85 min, peak 24.5 GiB VRAM | ## License Apache-2.0, matching the reference implementation. The base model is MIT-licensed.
Sources
1 sourceSource excerpts
3 excerptslicense: apache-2.0 basemodel: zai-org/GLM-4-9B-0414 tags: interpretability jacobian-lens logit-lens glm4 libraryname: jlens
zai-org/GLM-4-9B-0414d_model=4096[4096, 4096]dim_batch=8zai-org/GLM-4-9B-0414A fitted Jacobian lens for zai-org/GLM-4-9B-0414 , from Anthropic's Verbalizable Representations Form a Global Workspace in Language Models , fitted with the reference implementation anthropics/jacobian-lens . Contents field value --- --- base model…
dormantx/jacobian-lens-glm4-9b