SymbolicCapsuleNetwork

Live Demo

Try the interactive demo ↗

Example Results

Key Ideas

Standard convolutional detectors reduce every visual entity to a scalar confidence score, discarding the compositional structure that makes objects recognisable. SCN addresses this with three tightly integrated contributions:

1. Part-Whole Relation Modelling CapsRoute layers propagate evidence upward from low-level part capsules — encoding local features such as wheels, windows, and body panels — to high-level object capsules through dynamic routing-by-agreement. Agreement is only reached when the geometric votes from multiple parts are mutually consistent, giving the model an inductive bias toward spatially coherent detections.

2. Symbolic Routing Paths The routing coefficients produced at each capsule layer form an explicit, directed evidence graph. Unlike Grad-CAM or SHAP, which reconstruct explanations after the fact, SCN's routing weights are native model outputs — first-class signals that describe the model's reasoning as it happens, without any additional computation.

3. Concept-Based Detection Auditing Routing paths enable structured inspection that scalar networks cannot support:

• Verify that a predicted "car" is grounded in consistent wheel, body, and windshield part activations.
• Diagnose which part capsule collapsed when the model misses an object under occlusion or viewpoint change.
• Detect bias by aggregating routing statistics across a dataset to reveal which visual parts the model over-relies on.

Architecture

Architecture Overview

The pipeline flows through four capsule-specific modules:

Performance

Detection — COCO 2017 val

SCN sets a new state of the art among nano-scale detectors, surpassing every YOLO variant at comparable FLOPs.

SCN-n achieves +0.3% mAP50 and +0.8% mAP50:95 over the previous best (YOLO26n) at the same 6.5B FLOPs budget — accuracy gains that come entirely from structural reasoning, not extra capacity.

Accuracy–Efficiency Frontier

COCO mAP50:95 vs FLOPs

SCN occupies the top of the accuracy–efficiency frontier across all model scales (n / s / m / l / x). At every FLOPs level, SCN variants outperform their YOLO counterparts, demonstrating that part-whole routing is a principled and scalable improvement.

Instance Segmentation — COCO 2017 val

Quick Start

pip install ultralytics huggingface_hub

from huggingface_hub import hf_hub_download
from ultralytics import YOLO
from models import register_ultralytics_modules

weights = hf_hub_download(
    repo_id="zpyuan/SymbolicCapsuleNetwork",
    filename="weights/symbolic_capsule_network_segmentation.pt",
)
register_ultralytics_modules()
model = YOLO(weights)
results = model.predict("image.jpg", imgsz=640, conf=0.25)
results[0].show()

Command-line:

python predict.py path/to/image.jpg
python predict.py path/to/image.jpg --conf 0.3 --imgsz 1280

Repository Structure