Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Paper
• 2203.05482 • Published
• 7
Papers about model merging
referenced in the mergekit repo: https://github.com/cg123/mergekit
Note Linear merging
Editing Models with Task Arithmetic
Paper • 2212.04089 • Published • 7Note Task Arithmetic
Resolving Interference When Merging Models
Paper • 2306.01708 • Published • 17Note TIES
Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch
Paper • 2311.03099 • Published • 30Note DARE