March 15, 2025

ikayaniaamirshahzad@gmail.com

[2411.00113] A Geometric Framework for Understanding Memorization in Generative Models


View a PDF of the paper titled A Geometric Framework for Understanding Memorization in Generative Models, by Brendan Leigh Ross and 7 other authors

View PDF
HTML (experimental)

Abstract:As deep generative models have progressed, recent work has shown them to be capable of memorizing and reproducing training datapoints when deployed. These findings call into question the usability of generative models, especially in light of the legal and privacy risks brought about by memorization. To better understand this phenomenon, we propose the manifold memorization hypothesis (MMH), a geometric framework which leverages the manifold hypothesis into a clear language in which to reason about memorization. We propose to analyze memorization in terms of the relationship between the dimensionalities of (i) the ground truth data manifold and (ii) the manifold learned by the model. This framework provides a formal standard for “how memorized” a datapoint is and systematically categorizes memorized data into two types: memorization driven by overfitting and memorization driven by the underlying data distribution. By analyzing prior work in the context of the MMH, we explain and unify assorted observations in the literature. We empirically validate the MMH using synthetic data and image datasets up to the scale of Stable Diffusion, developing new tools for detecting and preventing generation of memorized samples in the process.

Submission history

From: Brendan Ross [view email]
[v1]
Thu, 31 Oct 2024 18:09:01 UTC (24,964 KB)
[v2]
Wed, 12 Mar 2025 18:00:00 UTC (26,530 KB)



Source link

Leave a Comment