Beyond Real Weights: Hypercomplex Representations for Stable Quantization

Vision language models (VLMs) demand immense parameter capacity to align high-dimensional visual features with linguistic representations, making them highly sensitive to quantization. We introduce a hypercomplex quantization framework that encodes model weights in complex space, $\mathbb{C}^n$ rather than $\mathbb{R}^n$, where a single complex weight simultaneously represents coupled real and imaginary components. Formally, we view quantization as an isomorphism $\varphi: \mathbb{R}^2 \to \mathbb{C}$, allowing each quantized parameter to preserve both magnitude and angular phase information under constrained bit-widths. This coupling reduces representational redundancy while maintaining alignment fidelity between modalities. In practice, replacing large feed-forward projections with hypercomplex operators yields a parameterization that is half the size in storage but twice as expressive per weight, stabilizing training dynamics even under aggressive quantization. Beyond compression, hypercomplex quantization provides a natural inductive bias for multimodal fusion, since visual embeddings are inherently spatial phase-rich and thus more faithfully preserved in hypercomplex form. Our framework enables VLMs to sustain high cross-modal alignment accuracy while operating with significantly compressed memory footprints, offering a principled path toward efficient yet stable multimodal intelligence.