Machine Unlearning via Adaptive Gradient Reweighting and Multi-stage Objective Optimization

Machine Unlearning (MU) focuses on removing the influence of training samples from pre-trained models without retraining the model entirely. Existing MU methods have made several efforts to enable complete forgetting while preserving the model’s performance on remaining data. However, they typically apply equal weights across different data, overlooking the ambiguous decision boundaries between similar samples or approximate classes. This leads to unnecessary consumption of shallowly memorized samples and significant performance degradation for approximate retention classes. Additionally, the inherent inconsistency between forgetting and retention objectives results in gradient conflict and domination problems during training, hindering model convergence and degrading overall performance. To address these, we introduce a novel adaptive gradient reweighting that assigns importance weights to individual forget samples or vulnerable retention classes, thereby enabling more efficient unlearning and preserving the performance of approximate classes. Subsequently, we propose a multi-stage objective optimization strategy, which comprises three optimization stages: Direction Rectification, Temporal Stabilization, and Adaptive Objective Combination. This strategy rectifies the direction of conflicting gradients and prevents one task (forgetting or retention) from dominating the model update. Comprehensive analyses and extensive experiments on multiple public datasets demonstrate that our method achieves considerable performance improvements in various tasks and scenarios.