09:00-09:30: Invited Talk

9:00am - 9:30am: A Constraint-Satisfaction Lens for Evaluating and Understanding Factual Errors of Language Models.

Besmira Nushi, Microsoft Research

Abstract: Many real-world information retrieval (IR) queries consist of specific requirements and constraints that users articulate in natural language (e.g., 'a list of ice cream shops in San Diego'). In the past, constraint satisfaction queries in IR were considered to be tasks that could only be solved via web-search or knowledge bases. More recently, large language models (LLMs) have demonstrated initial emergent abilities in this task, but major concerns remain related to information fabrication and factual errors. In this talk, we will discuss how using a constraint satisfaction lens to evaluate and understand capabilities of large language models (LLMs) can help with measuring and debugging failures of models in this space. The first part of this talk will focus on describing KITAB, a new dataset for measuring constraint satisfaction abilities of language models in the literature domain. Evaluation of state-of-the-art models in this dataset shows that current models still have major gaps in understanding and following constraints, regardless the availability of context. Then, the talk will deep dive into describing recent efforts in mechanistically understanding factual errors of models when they fail to satisfy constraints. Specifically, we discover a strong positive relation between the model's attention to constraint tokens and the factual accuracy of its responses. The work also proposes SAT Probe, a method probing self-attention patterns, that can predict constraint satisfaction and factual errors, and allows early error identification. The talk will conclude with summarizing more relevant work in this space and future avenues.
[ Slides ]

09:30-10:30: Session 1

9:30am - 9:50am: Constraint Acquisition by Transformer. S. Chandra, S. D. Prestwich, G. Provan, School of Computer Science and Information Technology, University College Cork, Ireland
[ Paper | Slides ]

9:50am - 10:10am: One Model, Any CSP: Graph Neural Networks as Fast Global Search Heuristics for Constraint Satisfaction. Jan Tonshoff, Berke Kisin, Jakob Lindner, Martin Theisen and Martin Grohe, RWTH Aachen University, Aachen, Germany
[ Poster | Slides ]

10:10am - 10:30am: Learning User Preferences in Interactive Constraint Programming. Mohamed Siala, LAAS-CNRS, Université de Toulouse, CNRS, INSA, Toulouse, France
[ Paper | Slides ]

10:30-11:00: Break (Light refreshments available near session rooms)

11:00-11:30: Invited Presentation

11:00am - 11:30am: Chatbots and LLMs for Constraint Programming: Opportunities and Challenges.

Dimos Tsouros, DTAI, KU Leuven, Belgium, Serdar Kadioğlu, AI Center of Excellence, Fidelity, USA, Dept. of Computer Science, Brown, USA

Abstract: Twenty-seven years ago, E. Freuder highlighted that "Constraint programming represents one of the closest approaches computer science has yet made to the Holy Grail of programming: the user states the problem, the computer solves it". Nowadays, CP users have great modeling tools available (like Minizinc and CPMpy), allowing them to formulate the problem and then let a solver do the rest of the job, getting closer to the stated goal. However, this still requires the CP user to know the formalism and respect it. Another significant challenge lies in the expertise required to effectively model combinatorial problems. All this limits the wider adoption of CP. In this discussion, we investigate how to leverage NLP approaches to model constraint problems from textual description. We discuss bottom-up and top-up approaches, by either building each required component (e.g., variables, constraints, objective, etc.) and then combining them together, or using pre-trained Large Language Models to directly extract the models. We will present early results with both.
[ Slides ]

11:30-12:30: Session 2

11:30am - 11:50am: Mab2Rec: Contextual Multi-Armed Bandits for Recommender Systems. Serdar Kadioğlu, AI Center of Excellence, Fidelity Investments, Boston, USA, Department of Computer Science, Brown University, Providence, USA
[ Paper | Slides ]

11:50am - 12:10pm: Integrating Reasoning and Learning for Design Generation & Scientific Discovery. Maxwell J. Jacobson, Nan Jiang, Md Nasim, Yexiang Xue, Purdue University, Department of Computer Science
[ Paper | Slides ]

12:10pm - 12:30pm: CP for ILP. Andrew Cropper and Céline Hocquette, University of Oxford, Oxford, UK
[ Paper | Slides ]

12:30-14:00: Lunch (on your own; no sponsored lunch provided)

14:00-15:30: Tutorial/Lab

14:00-15:30: Machine Learning for Better Constraint Solving: Portfolios and Automated Tuning.

Lars Kotthoff, Department of Electrical Engineering and Computer Science, School of Computing, University of Wyoming, Laramie, WY, USA

Abstract: You've modeled your constraint problem, now all you need to do is solve it. But wait, your solver has been running for a week and still hasn't found the solution! You're back to square one -- what to do? In this tutorial/lab, we'll have a look at machine learning methods for improving solving performance, without having to implement your own solver or fancy constraints. Using portfolios of solvers and automatically tuning their parameters, you can often get substantial performance improvements. > We'll cover some of the basic methods (which are useful not only in Constraint Programming) and how to implement them in practice, with exercises that you can re-use for the constraints application that you're interested in. The methods are mostly agnostic to the particular constraint solvers and problems they are applied to and can be adapted quickly to other applications. We'll finish by looking at some open questions.
[ Slides ]

15:30-16:00: Break (Light refreshments available near session rooms)

16:00-17:00: Benchmark Problems

16:00-17:00: Scheduling Examples for Constraint Acquisition. Helmut Simonis. Insight SFI Centre for Data Analytics, School of Computer Science and Information Technology, University College Cork, Cork, Ireland
[ Paper | Slides ]