RemEdit: Efficient Diffusion Editing with Riemannian Geometry
Abstract
Controllable image generation is fundamental to the success of modern generative AI, yet it faces a critical trade-off between semantic fidelity and inference speed. The RemEdit diffusion-based framework addresses this trade-off, avoiding the compromise between geometric precision and inference speed from which existing methods suffer; RemEdit overcomes this with two synergistic innovations. First, for editing fidelity, we navigate the latent space as a Riemannian manifold. A Mamba-based module efficiently learns the manifold's structure via Christoffel symbols, enabling direct and accurate geodesic path computation for smooth semantic edits. This control is further refined by a dual-SLERP blending technique and a goal-aware prompt enrichment pass from a Vision-Language Model. Second, for additional acceleration, we introduce a novel task-specific attention pruning mechanism. A lightweight pruning head learns to identify and retain only tokens essential to the edit, enabling effective optimization without the semantic degradation common in content-agnostic approaches. RemEdit surpasses prior SOTA editing frameworks while maintaining real-time performance under 50% pruning. Consequently, RemEdit establishes a new benchmark for practical and powerful image editing. RemEdit source code will be released upon publication.