Scientific

We study a class of linear-quadratic stochastic differential games in which each player interacts directly only with its nearest neighbors in a given graph. We find a semi-explicit Markovian equilibrium for any transitive graph, in terms of the empirical eigenvalue distribution of the graph’s normalized Laplacian matrix. This facilitates large-population asymptotics for various graph sequences, with several sparse and dense examples discussed in detail. In particular, the mean field game is the correct limit only in the dense graph case, i.e., when the degrees diverge in a suitable sense. Even though equilibrium strategies are nonlocal, depending on the behavior of all players, we use a correlation decay estimate to prove a propagation of chaos result in both the dense and sparse regimes, with the sparse case owing to the large distances between typical vertices. Without assuming the graphs are transitive, we show also that the mean field game solution can be used to construct decentralized approximate equilibria on any sufficiently dense graph sequence. This is joint work with Daniel Lacker.

The complexity of a dense graph increases combinatorically as its size increases. One approach to alleviate this complexity is to use graphon analysis to find an approximation of a very large graph’s adjacency matrix. Standard graphons are defined as functions on the unit square, but mapping nodes of a graph onto the unit interval may entail the loss of information. To account for this, a type of random graph is introduced called a featured graph which is a graph where each vertex has meaningful attributes determining connectivity. Featured graphons also provide an approach to the problems arising with graphs embedded in higher dimensional spaces. It is shown that in an appropriate norm the adjacency matrix operator converges to the associated featured graphon. Convergence is illustrated numerically with an SIR epidemic model generalized to multiple communities.

A two-stage enrichment design is a type of adaptive design, which extends a stratified design with a futility analysis on the marker negative cohort at the first stage, and the second stage can be either a targeted design with only the marker positive stratum, or still the stratified design with both marker strata, depending on the result of the interim futility analysis.

In this talk we consider the situation where the marker assay and the classification rule are possibly subject to error. We derive the sequential tests for the global hypothesis as well as the component tests for the overall cohort and the marker-positive cohort. We discuss the power analysis with the control of the type-I error rate and show the adverse impact of the misclassification on the powers. We also show the enhanced power of the two-stage enrichment over the one-stage design, and illustrate with examples of the recent successful development of immunotherapy in non-small-cell lung cancer.​

The Graphon Mean Field Game equations consist of a collection of parameterized Hamilton-Jacobi-Bellman equations, and a collection of parameterized Fokker-Planck-Kolmogorov equations coupled through a given graphon. In this talk, we will discuss the sensitivity of the gradient of HJB solutions with respect to the coefficients, which can be used for the solvability of Graphon Mean Field Game equation. It's based on the joint work with Peter Caines, Daniel Ho, Minyi Huang, and Jiamin Jian, see https://arxiv.org/pdf/2009.12144.pdf.

Large-scale systems comprising of multiple subsystems connected over a network arise in a number of applications including power systems, traffic networks, communication networks, and some economic systems. A common feature of such systems is that the state evolutions and costs are coupled, i.e., the state evolution and local cost of one subsystems depend not only on its own state and control action, but also on the state and control actions of other subsystems in the network.

We consider the problem of designing control strategies for such systems when some of the parameters of the system model are not known. Due to the unknown parameters, the control problem is also a learning problem. Directly using existing reinforcement learning algorithms on such network coupled subsystems would incur $O(n^{1.5} \sqrt{T})$ regret over a horizon $T$, where the $n$ is the number of subsystems. This superlinear dependence on $n$ is prohibitive in large scale networked systems because the regret per subsystem (which is $O(\sqrt{nT})$ grows with the size of the network.

We consider networks where the dynamics coupling may be represented by a symmetric matrix (e.g., the adjacency or Laplacian matrix corresponding to a undirected weighted graph) and the cost coupling matrix have the same eigenvalues as the dynamics coupling. We use spectral decomposition of the coupling matrices to decompose the system into (L + 1) systems which are only coupled through the noise, where L is the rank of the coupling matrix. We show that, when the system model is known, the optimal control input at each subsystem can be computing by solving (L+1) decoupled Riccati equations.

Using this structure of the planning solution, we propose a new Thompson sampling algorithm and show that its regret is bounded by $O(n\sqrt{T})$, which increases linearly in the size of the network. We present numerical simulations to illustrate our results on mean-field control and general low-rank networks.

Joint work with Shuang Gao, Sagar Sudhakara Ashutosh Nayyar, and Ouyang Yi