MaxSup Regularized ResNet50

--- license: apache-2.0 pipeline_tag: image-feature-extraction --- # MaxSup: Overcoming Representation Collapse in Label Smoothing **Max Suppression (MaxSup)** is a novel regularization technique that overcomes the shortcomings of traditional **Label Smoothing (LS)**. While LS prevents overconfidence by softening one-hot labels, it inadvertently collapses intra-class feature diversity and can boost overconfident errors. In contrast, **MaxSup** applies a uniform smoothing penalty to the model’s top prediction—regardless of correctness—preserving richer per-sample information and improving both classification performance and downstream transfer. --- ## Table of Contents 1. [Overview](#overview) 2. [Methodology: MaxSup vs. Label Smoothing](#methodology-maxsup-vs-label-smoothing) 3. [Enhanced Feature Representation](#enhanced-feature-representation) - [Qualitative Evaluation](#qualitative-evaluation) - [Quantitative Evaluation](#quantitative-evaluation) 4. [Training Vision Transformers with MaxSup](#training-vision-transformers-with-maxsup) - [Accelerated Data Loading via Caching (Optional)](#accelerated-data-loading-via-caching-optional) - [Preparing Data and Annotations for Caching](#preparing-data-and-annotations-for-caching) 5. [Pretrained Weights](#pretrained-weights) 6. [Training ConvNets with MaxSup](#training-convnets-with-maxsup) 7. [Logit Characteristic Visualization](#logit-characteristic-visualization) 8. [Citation](#citation) 9. [References](#references) --- ## Overview Traditional Label Smoothing (LS) replaces one-hot labels with a smoothed version to reduce overconfidence. However, LS can over-tighten feature clusters within each class and may reinforce errors by making mispredictions overconfident. **MaxSup** tackles these issues by applying a smoothing penalty to the model's **top-1 logit** output regardless of whether the prediction is correct, thus preserving intra-class diversity and enhancing inter-class separation. The result is improved performance on both classification tasks and downstream applications such as linear transfer and image segmentation. --- ## Methodology: MaxSup vs. Label Smoothing Label Smoothing softens the target distribution by blending the one-hot vector with a uniform distribution. Although effective at reducing overconfidence, LS inadvertently introduces two effects: - A **regularization term** that...

Source context: 0 downloads · 0 likes · Pipeline image-feature-extraction · Repo zhouyuxuanyx/MaxSup-Regularized-ResNet50