Learning in the Null Space: Small Singular Values for Continual Learning
This paper introduces NESS, a continual learning method that mitigates catastrophic forgetting by constraining task-specific updates to an approximate null space derived from the smallest singular values of input representations, thereby enabling efficient adaptation while preserving performance on previous tasks.