Reasoning on Tabular and Multimodal Data

<<< All talks

Reasoning on Tabular and Multimodal Data

When	Tuesday, October 29, 2024, 12:30 PM – 1:30 PM (MT)
Where	WEB 1230

Abstract

In this talk, I’ll walk through some of the latest AI advancements that address the challenges of working with complex data, with a focus on improving reasoning for both tabular and multimodal data.

I’ll begin by introducing H-STAR, a hybrid algorithm that combines symbolic and semantic reasoning to enhance question answering for tabular data. By leveraging multi-view table extraction and adaptive reasoning, H-STAR has shown great potential in improving reasoning across tabular datasets.

Then, I’ll introduce MMTabQA, a dataset we developed to evaluate how AI systems manage multimodal tables that integrate structured text and images. Our research reveals where current models struggle to process these diverse data types, highlighting key areas for improvement.

To wrap up, I’ll discuss open challenges and future directions, including expanding reasoning capabilities to other complex data types—such as charts, maps, and flowcharts—and enhancing AI systems’ robustness in handling numerical, temporal, and visual reasoning, particularly with large (vision) language models.

Speaker

Tags: computer vision data management natural language processing visualization

---

Part of the Data Science Seminar. Something wrong on this page? Edit _data/talks/2024-10-29-vivek-gupta.toml.