Scaling behavior of goldfish loss at very large model sizes

Determine how the memorization-mitigation effects of the goldfish loss scale when training large language models with tens to hundreds of billions of parameters, given evidence that larger models memorize more of their training data.

Background

The paper introduces the goldfish loss, a modified next-token prediction objective that excludes a pseudo-random subset of tokens from the loss to mitigate verbatim memorization while preserving overall model utility.

Experiments demonstrate reduced extractable memorization on models up to 7B parameters and a 1.1B pretraining setting, with minimal impact on downstream performance. Prior work indicates that larger models tend to memorize more, motivating investigation into whether the mitigation benefits of goldfish loss persist or change at substantially larger scales (tens to hundreds of billions of parameters).

References

Finally, prior work has shown that larger models memorize more of their training data, and thus studies of how the benefits afforded by goldfish loss scale to tens or hundreds of billions of parameters is an interesting open question.

— Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs  (2406.10209 - Hans et al., 2024) in Section 6.3 (Limitations: Don't Mistake Fish Oil for Snake Oil)