Advancing Strategic Decision Science Since 2014
Many real-world strategic interactions are not one-shot events but unfold over time in an environment that changes stochastically. To analyze these situations, researchers at the Nevada Institute of Game Theory employ stochastic games (also called Markov games) and their single-agent counterpart, Markov Decision Processes (MDPs). A stochastic game is defined by a set of states, a set of players, actions available in each state, a transition probability function (which gives the probability of moving to a new state given the current state and the players' actions), and a payoff function for each player. This framework is extraordinarily powerful for modeling everything from repeated business competition in a fluctuating market to long-term environmental management under ecological uncertainty to multi-agent reinforcement learning in AI.
The solution concept for stochastic games is more complex than for static games. Researchers study Markov Perfect Equilibrium (MPE), where each player's strategy depends only on the current state of the game, not on the entire history. Finding an MPE involves solving a system of functional equations, often requiring sophisticated computational methods. NIGT mathematicians and computer scientists develop algorithms to compute equilibria for classes of stochastic games with special structures (e.g., zero-sum, switching control, or games with a low-dimensional state space). They also study the properties of these equilibria, such as their efficiency (price of anarchy in dynamic settings) and how they evolve as players become more patient (i.e., as the discount factor approaches 1, placing more weight on future payoffs).
A prime application area is the management of renewable resources like fisheries, forests, or groundwater aquifers shared by multiple exploiters. This is modeled as a stochastic common-pool resource game, where the state is the current stock level, which grows according to a natural process and is depleted by harvesting. Players choose how much to harvest each period. The game-theoretic analysis reveals the tension between individual short-term gain and collective long-term sustainability. It shows how strategic uncertainty about others' harvests can lead to over-extraction even when all parties understand the dynamics, a phenomenon known as the 'dynamic tragedy of the commons.' NIGT researchers use these models to design regulatory mechanisms, such as dynamic quotas or transferable harvesting rights, that align individual incentives with sustainable management.
In finance, stochastic games model strategic trading in illiquid markets, where a large trader's actions affect prices and trigger responses from others. In industrial organization, they model R&D races, where firms invest in research under technological uncertainty, and the first to achieve a breakthrough wins a patent. The state might represent the current knowledge level or cost position of each firm. These models can predict the pace of innovation, the level of over-investment in R&D due to competition, and the welfare effects. Similarly, models of dynamic price competition with switching costs or network effects are inherently stochastic, as customer bases change over time. NIGT's work helps firms in such industries understand the long-term implications of their strategic choices.
The field of reinforcement learning (RL) in AI is built upon MDPs. When multiple learning agents interact, the problem becomes a multi-agent reinforcement learning (MARL) problem, which is essentially the problem of learning in a stochastic game. NIGT researchers are at the forefront of this intersection. They study the convergence properties of MARL algorithms—do they learn to play an equilibrium? They also use game theory to analyze the robustness of learned policies to the presence of other strategic agents. Conversely, they employ RL techniques as computational tools to approximate equilibria in large, complex stochastic games that are analytically intractable. This symbiotic relationship is driving advances in both AI and game theory, enabling the analysis and design of intelligent systems that must operate strategically in complex, dynamic, and uncertain worlds—a core mission of the Nevada Institute.