DiRe: Diversity-promoting Regularization for Dataset Condensation

In dataset condensation, given an original training dataset, the goal is to synthesize a small dataset that replicates the training utility of the original dataset, when used to train neural networks. Existing condensation methods synthesize datasets that contain significant redundancy, leading to their inefficiency. Thus, there is a dire need to ensure diversity in the synthesized datasets. In this work, we propose an intuitive Diversity Regularizer (DiRe) composed of cosine similarity and Euclidean distance. Most importantly, the proposed regularizer can be applied off-the-shelf to various state-of-the-art optimization-driven condensation methods. Through extensive experimentation, we demonstrate that our approach improves state-of-the-art condensation methods on various benchmark datasets from CIFAR-10 to ImageNet-1K regarding generalization and diversity metrics.