‘Catastrophic overtraining’ could harm large language AI models that are trained on more data for the sake of training

Researchers from prime US universities warn extending pre-training could be detrimental to efficiency An excessive amount of pre-training can ship worse efficiency because of one thing akin to the butterfly impact The extra they’re pre-trained, the extra they grow to be delicate to small modifications that would disrupt the tip consequence

Researchers from Carnegie Mellon, Stanford, Harvard, and Princeton are difficult considered one of AI improvement’s accepted core beliefs – that the extra pre-training information the higher the efficiency.