Self distillation can decrease the correct answers of an AI model
The authors add Representation Geometry Regularization, which keeps the geometry of hidden representations and usually stops the decrease.
Claimed, not confirmed
Sources
Posted
What matters in AI.
SubscribeThe authors add Representation Geometry Regularization, which keeps the geometry of hidden representations and usually stops the decrease.
Claimed, not confirmed
Posted
