The Llama Herd of Models: System 1, 2, 3 Go!

Modern artificial intelligence (AI) systems are powered by foundation models. These models enable AI systems to understand and produce language, to perceive and generate visual content, to recognize and produce speech, and to perform actions in digital environments. While foundation models initially resembled System 1 (thinking fast), they are starting to implement a type of System 2 (thinking slow) that enables them to reason through complex problems before producing an answer. In this talk, I will describe the development of the Llama family of foundational models. I will also argue that the next frontier in the development of foundation models is to equip them with a “System 3” that enables models to think together.