OpenBEATs ICME SOUND

--- datasets: - as2m license: cc-by-4.0 library_name: espnet pipeline_tag: audio-classification tags: - espnet - audio - self-supervised-learning --- # OpenBEATs-ICME-SOUND This model is part of the CMU-AIST submission for the ICME 2025 Audio Encoder Challenge. It is a scaled-up version of the **BEATs** (audio encoder) architecture, featuring 300 million parameters. - **Paper:** [The CMU-AIST submission for the ICME 2025 Audio Encoder Challenge](https://huggingface.co/papers/2601.16273) - **Library:** [espnet](https://github.com/espnet/espnet/) ## Model Description The system is built on BEATs, a masked speech token prediction-based audio encoder. This checkpoint was trained on 74,000 hours of diverse audio data covering speech, music, and sound corpora. It explores the impact of balanced pre-training mixtures on final performance across various audio domains. ## ESPnet2 SSL model ### `shikhar7ssu/OpenBEATs-ICME-SOUND` This model was trained by Shikhar Bharadwaj using as2m recipe in [espnet](https://github.com/espnet/espnet/). ## SSL config <details><summary>expand</summary> ``` config: conf/pretrain_large_icme.yaml print_config: false log_level: INFO drop_last_iter: false dry_run: false iterator_type: sequence valid_iterator_type: null output_dir: /work/nvme/bbjs/sbharadwaj/icme_challenge/exp/beats_iter3_large.soundmix.icme ngpu: 1 seed: 0 num_workers: 4 num_att_plot: 0 dist_backend: nccl dist_init_method: env:// dist_world_size: 4 dist_rank: 0 local_rank: 0 dist_master_addr: localhost dist_master_port: 35949 dist_launcher: null multiprocessing_distributed: true unused_parameters: false sharded_ddp: false use_deepspeed: true deepspeed_config: conf/ds_ear_large.json gradient_as_bucket_view: true ddp_comm_hook: null cudnn_enabled: true cudnn_benchmark: false cudnn_deterministic: true use_tf32: false collect_stats: false write_collected_feats: false max_epoch: 60 patience: null val_scheduler_criterion: - valid - loss early_stopping_criterion: - valid - loss - min best_model_criterion: - - train - loss - min - - valid - loss - min - - train - acc - max - - valid - acc - max keep_nbest_models: - 10 nbest_averaging_interval: 0 grad_clip: 5.0 grad_clip_type: 2.0 grad_noise: false accum_grad: 1 no_forward_run: false resume: true train_dtype: float32 use_amp: false log_interval: null use_matplotlib: true use_tensorboard: true create_graph_in_ten...