LARGE LANGUAGE MODELS: FOUNDATIONS AND E

Large Language Models (LLMs) such as GPT-3, ChatGPT, LLaMA are models that are trained on large amounts of data and are adaptable to a wide range of tasks. They are the basis of most state-of-the-art systems in Natural Language Processing. While the potential of these technologies for social good is large, the risks are also comparable. In this course, the students will learn the fundamentals about the modeling, theory and ethical aspects of LLMs and their applications, while gaining experience working with them. The course will be structured as a seminar, where one class is dedicated to instructor-led lecture and one to studentled discussion of papers around topics covered in the lecture. Each paper discussion will be structured as a panel of 3-4 students, each with an assigned role. Each panel role covers one aspect of critically assessing an academic/industry paper. Everyone in the class should participate by commenting and asking questions from the panel. The class is project-based, meaning there will be a semester-long project focused on evaluating LLMs and/or building LLMs around a topic/problem/task you care about, with an end of semester final paper. The projects will be done by groups of 3-4 students.

Prerequisite(s): COMS W3134 or W3136 or W3137 (or equivalent). Background in probability/statistics and linear algebra is also required and experience with Python programming is strongly encouraged. Some previous or concurrent exposure to NLP, AI or machine learning is beneficial, but not required.