3.3 KiB
Mapping Networks — Raw Archive
- Title: Mapping Networks
- Authors: Lord Sen, Shyamapada Mukherjee (National Institute of Technology Rourkela, India)
- arXiv ID: 2602.19134
- Category: cs.CV (Computer Vision)
- Date: 2026-02-22
- Pages: 10
Abstract
The escalating parameter counts in modern deep learning models pose a fundamental challenge to efficient training and resolution of overfitting. We address this by introducing the Mapping Networks which replace the high dimensional weight space by a compact, trainable latent vector based on the hypothesis that the trained parameters of large networks reside on smooth, low-dimensional manifolds. Henceforth, the Mapping Theorem enforced by a dedicated Mapping Loss, shows the existence of a mapping from this latent space to the target weight space both theoretically and in practice. Mapping Networks significantly reduce overfitting and achieve comparable to better performance than target network across complex vision and sequence tasks, including Image Classification, Deepfake Detection etc, with 99.5%, i.e., around 500× reduction in trainable parameters.
Key Contributions
- Mapping Theorem: Establishes existence of a smooth, low-dimensional parameterization capable of generating optimal high-dimensional weights for a target network with arbitrarily small bounded error.
- Mapping Network: Low-dimensional trainable latent vector + modulated Mapping Weights to efficiently produce target network parameters, decoupling training from target network.
- Mapping Loss: Jointly optimizes task performance and enforces geometric/analytic properties required by Mapping Theorem (Task + Stability + Smoothness + Alignment).
Core Mechanism
- Weight-Manifold Hypothesis: Neural network parameters during training do not explore full P-dimensional Euclidean space; they evolve along smooth, low-dimensional manifolds M_θ ⊂ R^P with dim(M_θ) ≪ P.
- Architecture: Trainable latent vector z ∈ R^d → fixed orthogonally initialized mapping network modulated by z (w_ij ← w_ij + α·z_i) → generates target network parameters θ̂ → target network does feed-forward only.
- Mapping Loss: L_map = L_task + λ·L_stab + λ·L_smooth + λ·L_align (all λ trainable)
- Two strategies: SLVT (single latent vector) and LWT (layer-wise training)
Results Summary
| Task | Baseline Params | Mapping Params | Reduction | Performance |
|---|---|---|---|---|
| MNIST (CNN1) | 537,994 | 1,024-4,078 | 260-525× | 98.78-99.67% (vs 99.32%) |
| FMNIST (CNN1) | 537,994 | 1,024-4,078 | 260-525× | 93.02-94.83% (vs 92.89%) |
| Deepfake (CNN2, Celeb-DF) | 108,618 | 1,024-2,688 | 40-106× | 78.83-89.98% (vs 79.03-83.13%) |
| Segmentation (CNN3) | 1,734,803 | 8,192-9,126 | 190-211× | 97.56-97.92% (vs 93.21%) |
| LSTM (Air Pollution) | 12,961 | 64-2,048 | 6-202× | MSE 0.00061-0.0019 (vs 0.0035) |
| Fine-tune ResNet50 | 25M | 1,024-2,048 | ~12,000× | 91.02-95.10% (vs 91.78-95.23%) |
Related Work
- Manifold Hypothesis [7, 19]
- HyperNetworks [13]
- Low-Rank Compression [14]
- Lottery Ticket Hypothesis [9]
- Intrinsic Dimension of loss landscapes [16]
- Training trajectory manifolds [18]
Extensions
Compatible with Low-Rank Decomposition, Pruning, and Quantization for further inference-time optimization.