A joint team from Google Research and DeepMind has developed a training method called SALT (Small model aided large model training) that cuts training time by up to 28 percent while improving performance. The key innovation? Using smaller language models as assistant teachers. The researchers also created an enhanced version called SALTDS that carefully selects training data, focusing on examples where the smaller model performs well. Source: Google finds new way to train AI models using smaller ‘teacher’ models