Scaling Language Models: Methods, Analysis & Insights from Training Gopher.
Jack W. RaeSebastian BorgeaudTrevor CaiKatie MillicanJordan HoffmannH. Francis SongJohn AslanidesSarah HendersonRoman RingSusannah YoungEliza RutherfordTom HenniganJacob MenickAlbin CassirerRichard PowellGeorge van den DriesscheLisa Anne HendricksMaribeth RauhPo-Sen HuangAmelia GlaeseJohannes WelblSumanth DathathriSaffron HuangJonathan UesatoJohn MellorIrina HigginsAntonia CreswellNat McAleeseAmy WuErich ElsenSiddhant M. JayakumarElena BuchatskayaDavid BuddenEsme SutherlandKaren SimonyanMichela PaganiniLaurent SifreLena MartensXiang Lorraine LiAdhiguna KuncoroAida NematzadehElena GribovskayaDomenic DonatoAngeliki LazaridouArthur MenschJean-Baptiste LespiauMaria TsimpoukelliNikolai GrigorevDoug FritzThibault SottiauxMantas PajarskasToby PohlenZhitao GongDaniel ToyamaCyprien de Masson d'AutumeYujia LiTayfun TerziVladimir MikulikIgor BabuschkinAidan ClarkDiego de Las CasasAurelia GuyChris JonesJames BradburyMatthew JohnsonBlake A. HechtmanLaura WeidingerIason GabrielWilliam S. IsaacEdward LockhartSimon OsinderoLaura RimellChris DyerOriol VinyalsKareem AyoubJeff StanwayLorrayne BennettDemis HassabisKoray KavukcuogluGeoffrey IrvingPublished in: CoRR (2021)