Computer Science Seminar by Xiang Yue: Open Language Models that Reason Across Modalities
Speaker: , post-doctoral research associate, Neubig Lab, Language Technologies Institute, Carnegie Mellon University
Title: Open Language Models that Reason across Modalities
Abstract: Large language models (LLMs) have made remarkable progress, but critical challenges persist in enhancing their reasoning capabilities across modalities, particularly within the open-source community. In this talk, I will present our research on understanding and advancing the reasoning abilities of open language models across modalities. I will begin by introducing our work on developing rigorous, open-access reasoning benchmarks like MMMU, designed to uncover key limitations in current models while promoting transparency and collaboration through broad community accessibility. Next, I will discuss our approaches to improving open-source LLMs’ reasoning, focusing on scalable synthetic data generation techniques that leverage academia-level compute to produce millions of high-quality reasoning datasets, enabling the training of LLMs comparable to industry-leading models. Additionally, I will emphasize the importance of building responsible LMs, with a focus on safeguarding data privacy. The talk will conclude with a discussion of future directions for creating intelligent systems that dynamically learn and evolve through user interactions and feedback.
Bio: Xiang Yue is a Postdoctoral Fellow at Carnegie Mellon University. He received his PhD from The Ohio State University in 2023. His research focuses on understanding and enhancing the reasoning capabilities of large language models. He has been awarded a postdoctoral fellowship from the Carnegie Bosch Institute, two AI rising stars, two Best Paper Finalist or Honorable Mention at CVPR 2024 and ACL 2023. Xiang’s recent work on developing the MMMU evaluation benchmark has garnered attention beyond academia, being featured in the releases of OpenAI GPT-4o and Google Gemini.