Eric Hallahan
4 Papers
Eric Hallahan is an academic researcher. The author has contributed to research in topics: Computer science. The author has an hindex of 2, co-authored 2 publications.
Chat about Author
Papers
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
Stella Biderman,Hailey Schoelkopf,Quentin Anthony,Herbie Bradley,Eric Hallahan,Mohammad Aflah Khan,Shivanshu Purohit,Usvsn Sai Prashanth,Edward Raff,Lintang A. Sutawika,Oskar van der Wal +10 more
TL;DR: Pythia as discussed by the authors ) is a suite of 16 language models trained on public data seen in the exact same order and ranging in size from 70M to 12B parameters, with 154 checkpoints for each one of the 16 models, alongside tools to download and reconstruct their exact training dataaloaders for further study.
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
Sid Black,Stella Biderman,Eric Hallahan,Quentin Anthony,Leo Gao,Laurence Golding,Horace He,Connor Leahy,Kyle McDonell,Jason Phang,Michael Pieler,Usvsn Sai Prashanth,Shivanshu Purohit,Laria Reynolds,J. S. Tow,Benqi Wang,Samuel Weinbach +16 more
- 14 Apr 2022
TL;DR: GPT-NeoX-20B is introduced, a 20 billion parameter autoregressive language model trained on the Pile, whose weights will be made freely and openly available to the public through a permissive license.
545
Structured Pruning for Large Language Models Using Coupled Components Elimination and Minor Fine-tuning
Sid Black,Stella Biderman,Eric Hallahan,Leo Anthony,Laurence Gao,Horace Golding,He,Wei-Lin Chiang,Zhuohan Li,Zi Lin,Ying Sheng,Christopher Clark,Kenton Lee,Ming-Wei Chang,Peter Clark,Isaac Cowhey,Oren Etzioni,Tushar Khot,Tri Dao,Dan Fu,Stefano Ermon,Atri Rudra,Edward J. Hu,Yelong Shen,Zeyuan Phillip Wallis,Eldar Kurtic,Daniel Campos,Tuan Nguyen,Elias Fran-728,Mark Kurtz,Benjamin Fineran,M. Ellen Goin,Hao Li,A.A. Kadav,Igor Durdanovic,Hanan Samet,Chen Liang,Simiao Zuo,Minshuo Chen,Xia-Ming Jiang,Pengcheng Liu,Tuo He,Zhao Chen,Mitchell P. Marcus,Beatrice Santorini,Mary Ann,Stephen Merity,Caiming Xiong,James Bradbury,Todor Mihaylov +49 more
TL;DR: Researchers propose a novel structured pruning algorithm for large language models, eliminating coupled components and preserving dependency relationships, achieving 20% parameter reduction with minimal performance loss and requiring only few epochs of fine-tuning.
1
BLESS: Benchmarking Large Language Models on Sentence Simplification
Tannon Kew,Alison Chi,Laura Vásquez-Rodríguez,Sweta Agrawal,Dennis Aumiller,Fernando Emilio Alva Manchego,Matthew Shardlow,Jason Baumgartner,Savvas Zannettou,Brian Keegan,Megan Squire,Jeremy Blackburn. 2020,Sid Black,Eric Hallahan,Quentin Anthony,Leo Gao,Laurence Golding,Horace He,Connor Leahy,Kyle McDonell,Jason Phang,Michael Pieler,Tom Brown,Benjamin Mann,Nick Ryder,Melanie Subbiah,Jared Kaplan,Prafulla Dhariwal,Arvind Neelakantan,Pranav Shyam,Girish Sastry,Li-Kuang Chen,Yi-Chen Chang,Xi Srinivasan Iyer,Victoria Lin,Ramakanth Pasunuru,Todor Mihaylov,Daniel Simig,Ping Yu,Kurt Shuster,Tianlu Wang,Punit Qing Liu,Singh Koura,Xian Li,Brian O'Horo,Gabriel Pereyra,Jeff Wang,Christopher Dewan,A. Celikyilmaz,Luke Zettlemoyer,Ves Stoyanov. 2023,Chao Jiang,Mounica Maddela,Wuwei Lan,Yang Zhong,Wei Xu,Neural,J. P. Kincaid,R. P. Fishburne,R. L. Rogers,Brad S. Chissom. 1975,Hugo Laurençon,Lucile Saulnier,Thomas Wang,Christopher Akiki,Albert Villanova,del Moral,Teven Le Scao,Leandro von Werra,Chenghao Mou,E. G. Ponferrada,Huu Nguyen,Mike Lewis,Yin Shi Liu,Naman Goyal,Marjan Ghazvininejad,Abdelrahman Mohamed,Omer Levy +77 more
TL;DR: The evaluation indicates that the best LLMs, despite not being trained on TS, perform comparably with state-of-the-art TS baselines, and certain LLMs demonstrate a greater range and diversity of edit operations.