3dcvt Lrw1000

--- license: mit library_name: pytorch tags: - pytorch - lip-reading - computer-vision - video-classification - reproduction - 3dcvt --- # 3DCvT on LRW-1000 This repository provides the released checkpoint and evaluation artifacts for an unofficial PyTorch reproduction of: **A Lip Reading Method Based on 3D Convolutional Vision Transformer** Code repository: - https://github.com/DPInnovationWorks/3DCvT_LipReading ## Model Summary - Task: Chinese word-level lip reading - Dataset: LRW-1000 - Number of classes: 1184 in this processed split - Framework: PyTorch - Architecture: 3D CNN + CvT + BiGRU ## Released Files - `best_model.pth`: released checkpoint - `sha256.txt`: checksum for the checkpoint - `logs/train.log`: selected training log - `results/per_class_acc_lrw1000_val.csv`: per-class validation summary - `plots/learning_curve.png`: learning curve exported from training ## Training Setup Training settings from the released run: - GPUs: 1 GPU - Per-step batch size: 128 - Gradient accumulation: 2 - Effective batch size: 256 - Epochs: 120 - Optimizer: Adam - Weight decay: 1e-4 - Learning rate: 6e-4 - Warmup epochs: 5 - Mixed precision: AMP enabled - `torch.compile`: disabled ## Evaluation Result | Dataset | Split | Metric | Value | | --- | --- | --- | --- | | LRW-1000 | Validation | Top-1 Accuracy | 55.29% | ## Intended Use This checkpoint is intended for: - research reproduction - benchmark comparison - qualitative inference demos It is not intended as a production-ready commercial lip-reading system. ## Limitations - Performance depends on using the matching preprocessing pipeline - This release does not include the raw LRW-1000 dataset - Users must obtain the dataset according to its own terms - This processed split uses 1184 classes in the generated vocabulary ## Usage Example inference command: ```bash python inference.py \ --dataset lrw1000 \ --pkl_path /path/to/sample.pkl \ --checkpoint /path/to/best_model.pth \ --gpu 0 ``` ## Notes - The checkpoint is released for reproducibility - Please use the matching code version when possible - Local source artifact names were `best_model_for_lrw1000.pth` and `train_lrw1000.log` ## Citation If you use this release, please cite the original paper: ```bibtex @article{wu2022lip, title={A Lip Reading Method Based on 3D Convolutional Vision Transformer}, author={Wu, Jiafeng...

Source context: 0 downloads · 0 likes · Pipeline video-classification · Library pytorch · Repo RaikkonenWu/3dcvt-lrw1000