Measuring the Depth of LLM Unlearning via Activation Patching
Quick summary
arXiv:2605.24614v2 Announce Type: replace-cross Abstract: Large language model (LLM) unlearning has emerged as a crucial post-hoc mechanism for privacy protection and AI safety, yet auditing whether target knowledge is truly erased remains challenging. Existing output-level metrics fail to detect when this knowledge remains recoverable from internal representations. Recent white-box studies reveal such residual knowledge but often rely on auxiliary training or dataset-specific adaptations, leaving no generalizable metric. We close this gap with the Unlearning Depth Score (UDS), a metric that q
Key takeaways
- arXiv:2605.24614v2 Announce Type: replace-cross Abstract: Large language model (LLM) unlearning has emerged as a crucial post-hoc mechanism for privacy protection and AI safety, yet auditing whether target knowledge is truly erased remains challenging.
- Existing output-level metrics fail to detect when this knowledge remains recoverable from internal representations.
- Recent white-box studies reveal such residual knowledge but often rely on auxiliary training or dataset-specific adaptations, leaving no generalizable metric.
Why it matters
The significance is not only the legal text but how it changes product design. Decisions around “Measuring the Depth of LLM Unlearning via Activation Patching” may reshape data collection, model training, output accountability and market access.
Member comments