Research

My research focuses on learning dynamics in multi-agent systems.

Multi-agent learning of reactive strategies in iterated games

When everyone acts in their own interest, cooperation can be hard to achieve. we look at populations of agents that use gradient ascent to improve their own payoffs in the iterated prisoner’s dilemma.

Differences in learning rates can change the collective outcome. Heterogeneous learners reach a cooperative outcome, whereas learners with the same learning rate don't.

Dynamical systems · Evolutionary game theory · Multi-agent learning

Read the preprint

Learning dynamics with heterogeneous learning rates. Each point represents an agent’s reactive strategy: p is the probability of cooperating after cooperation, and q after defection. Color indicates learning rate η. After all the agents reach an unforgiving (low q) strategy, they eventually reach a cooperative (high p) outcome.

Competitive exploration in structured environments

Foragers face a tradeoff: stay with a known resource, or search for something better. The optimal strategy of a forager depends on their environment. How far away is the reward, and how much more valuable is it? When foragers compete, the best balance also depends on what the other forager does.

I study how individually beneficial exploration can lead to outcomes that are worse for the pair. We find examples of suboptimal Nash equilibria and other equilibria reachable through local strategy updating across parameter regimes.

A simple signal that a richer resource has been found can resolve many of these social dilemmas.

Reinforcement learning · Markov chains · Game theory

Read the preprint

Payoff landscape and local best-response dynamics of a two-person foraging game, as the reward ratio of the far reward (with unknown location) and near reward changes. Each player's strategy is their exploration rate (a higher exploration rate means they reach the richer resource faster, but cannot exploit it as well). The figure on the left is a fine-grid payoff matrix. The figure on the right is the result of a local best-response algorithm, where each agent adjusts their strategy to whichever neighboring strategy maximizes their payoff. The best-response curves, local equilibria, and Nash equilibria shift as the ratio of the two rewards changes.