Federated Unlearning: Concept & Challenges

Summary:
In preparation for a lab meeting, I studied the concept of Federated Unlearning, which extends the idea of “machine unlearning” to federated learning environments. While federated learning protects raw data by keeping it on clients, requests such as the “Right to be Forgotten” raise a crucial question: How can we safely remove the influence of specific data or clients from a trained federated model? This summary is based on the PEPR ’24 talk Learning and Unlearning Your Data in Federated Settings (USENIX).

🔑 Research Question:

• Can federated learning systems support safe and efficient data deletion without full retraining?

⚙️ Conceptual Approaches:

• Passive Unlearning:
  • Server-only (leveraging stored updates)
  • Client-aided (clients assist with gradient/history)
• Active Unlearning:
  • Server and clients collaboratively remove the influence of target data.
• Level of Unlearning:
  • Record-level, class-level, or client-level unlearning

📊 Key Insightes:

• Retraining is reliable but computationally prohibitive.
• Approximate unlearning can preserve accuracy but may weaken guarantees.
• Privacy, consistency, and efficiency must be carefully balanced.
• Lack of formal proof of unlearning remains a major challenge.

⚠️ Limitations & Open Challenges:

• Verifiability: proving that unlearning actually occurred.
• Dynamic participation: handling clients joining or leaving.
• Fairness and explainability remain underexplored.
• New privacy risks may arise during the unlearning process itself.

🎥 Reference:

• PEPR ‘24 - Learning and Unlearning Your Data in Federated Settings

Slides:
PDF (Korean) Download