Mathematics

Current LLM alignment techniques use pairwise human preferences at a sample level, and as such, they do not imply an alignment on the distributional level. We propose in this paper Alignment via Optimal Transport (AOT), a novel method for distributional preference alignment of LLMs. AOT aligns LLMs on unpaired preference data by making the reward distribution of the positive samples stochastically dominant in the first order on the distribution of negative samples. We introduce a convex relaxation of this first-order stochastic dominance and cast it as an optimal transport problem with a smooth and convex cost. Thanks to the one-dimensional nature of the resulting optimal transport problem and the convexity of the cost, it has a closed-form solution via sorting on empirical measures. We fine-tune LLMs with this AOT objective, which enables alignment by penalizing the violation of the stochastic dominance of the reward distribution of the positive samples on the reward distribution of the negative samples. We analyze the sample complexity of AOT by considering the dual of the OT problem and show that it converges at the parametric rate. Empirically, we show on a diverse set of alignment datasets and LLMs that AOT leads to state-of-the-art models in the 7B family of models when evaluated with Open LLM Benchmarks and AlpacaEval. We will cover how these ideas extend to multivariate stochastic dominance, that is crucial for covering the multi-reward setting in the context of LLMs.

• https://arxiv.org/pdf/2406.05882v1
• https://arxiv.org/abs/2406.06425

In practice, L-functions appear as generating functions encapsulating information about various objects, such as Galois representations, elliptic curves, arithmetic functions, modular forms, Maass forms, etc. Studying L-functions is therefore of utmost importance in number theory at large. Two of their attached data carry critical information: their zeros, which govern the distributional behavior of underlying objects; and their central values, which are related to invariants such as the class number of a field extension. We discuss a connection between low-lying zeros and central values of L-functions, in particular showing that results about the distribution of low-lying zeros (towards the density conjecture of Katz-Sarnak) implies results about the distribution of the central values (towards the normal distribution conjecture of Keating-Snaith). Even though we discuss this principle in general, we instanciate it in the case of modular forms in the level aspect to give a statement and explain the arguments of the proof.

Over the years, there have been several open problems involving polynomials that I would love to tell others about. This opportunity to speak at my “home ground” seems the perfect time to do so. More specifically, I will discuss the following:

- A conjecture of Ruzsa for integers and a related problem in a joint work with Bell for polynomials over finite fields.
- A conjectural lower bound for the degree of irreducible factors of certain polynomials from a joint work with DeMarco, Ghioca, Krieger, Tucker, and Ye.
- The irreducibility of certain Gleason polynomials.

The quadratically regularized optimal transport problem (QOT) has emerged in the literature as a sparse alternative to entropic regularization (EOT). Unlike EOT, whose solutions always have full support—even for small regularization parameters—QOT solutions, or QOT plans, tend to approximate the support of the unregularized transport problem. This raises natural questions: Do the supports decrease monotonically? At what rate does this support reduction occur? How quickly does the QOT cost converge to the optimal transport cost? In this talk, we will review recent theoretical results addressing these questions.

Bounds on the logarithmic derivative and the reciprocal of the Riemann zeta function are studied as they have a wide range of applications, such as computing bounds for Mertens function. In this talk, we are mainly concerned with explicit bounds. Obtaining decent bounds are tricky, as they are only valid in a zero-free region, and the constants involved tend to blow up as one approaches the edge of the region, and a potential zero. We will discuss such bounds, their uses, and the computational and analytic techniques involved. Finally, we also show how to obtain a power savings in the case of the reciprocal of zeta.