38 lines
2.2 KiB
Markdown
38 lines
2.2 KiB
Markdown
---
|
|
title: "Minimax-Optimal Policy Regret in Partially Observable Markov Games"
|
|
source: "arXiv:2606.02363v1"
|
|
authors: "Raman Arora"
|
|
affiliation: "Johns Hopkins University"
|
|
year: 2026
|
|
category: "cs.LG, stat.ML"
|
|
published: "2026-06-01"
|
|
venue: "ICML 2026"
|
|
---
|
|
|
|
# Minimax-Optimal Policy Regret in Partially Observable Markov Games
|
|
|
|
**Author**: Raman Arora (Johns Hopkins University)
|
|
**arXiv**: 2606.02363v1 [cs.LG, stat.ML]
|
|
**Venue**: ICML 2026, Seoul
|
|
**Published**: 2026-06-01
|
|
|
|
## Abstract
|
|
|
|
We study sequential decision-making in partially observable environments against strategic, adaptive opponents, modeled as partially observable Markov games (POMGs). The central challenge is to learn latent dynamics from partial observations while facing an adversary whose behavior depends on the learner's strategy, making standard regret notions inadequate.
|
|
|
|
We prove that an epoch-based optimistic maximum-likelihood algorithm achieves O~(sqrt(T)) policy regret, with explicit dependence on the horizon, adversary memory, confidence radius, and the aggregate Eluder dimension of the observable-operator class. A matching lower bound confirms minimax optimality. Extensions include horizon-adaptive guarantees and adversaries with geometric fading memory.
|
|
|
|
## Key Concepts
|
|
- [[partially-observable-markov-game|POMG]] — core model: partial observability + strategic adversary
|
|
- [[policy-regret|Policy Regret]] — counterfactual regret against adaptive opponents
|
|
- [[eluder-dimension|Eluder Dimension]] — sequential complexity measure
|
|
- [[observable-operator-model|OOM]] — operator-based representation of POMG dynamics
|
|
- [[posterior-lipschitz-adversary|Posterior-Lipschitz Adversary]] — smoothness assumption
|
|
- [[weak-revealing-condition|Weak Revealing]] — observation informativeness condition
|
|
- [[causal-decomposition-pomg|Causal Decomposition]] — separating world from adversary
|
|
- [[epoch-based-optimistic-mle|Epoch-based Optimistic MLE]] — the algorithm
|
|
- [[minimax-optimality|Minimax Optimality]] — matching upper and lower bounds
|
|
- [[pomdp|POMDP]] — single-agent precursor
|
|
- [[adaptive-adversary|Adaptive Adversary]] — strategic opponent model
|
|
- [[fading-memory|Fading Memory]] — adversary memory extension
|