The Center for Education and Research in Information Assurance and Security (CERIAS)

The Center for Education and Research in
Information Assurance and Security (CERIAS)

Robust and Adversarially Resilient Reinforcement Learning

Principal Investigator: Vaneet Aggarwal

A decision policy learned in a simulator, or from yesterday's data, can fail when the real world is different. The same is true when some of the participants feeding it data are faulty or acting in bad faith. This project develops reinforcement learning (RL) methods that keep working in these situations, with mathematical guarantees rather than hope.

Our outcomes include:

  • Robust RL: methods that optimize for the worst case over a range of plausible environments, so a policy stays good even if the model was somewhat wrong. We gave the first efficient learning methods with performance guarantees for long-running systems in this setting, along with sharper analyses of how much data robust learning needs.
  • Generalizing across tasks: curriculum training that prepares agents for a whole family of tasks, not just one.
  • Learning with unreliable partners: federated RL, where many agents learn together without sharing raw data, that still converges when some agents send corrupted updates or their updates arrive late.
  • Multi-agent behavior: a method to detect when agents in a game are secretly colluding, and methods that infer the goals and sub-goals behind observed expert behavior (adversarial inverse RL).

Representative Publications

  • Trevor Bonjour, Vaneet Aggarwal, and Bharat Bhargava, "Information-Theoretic Approach to Detect Collusion in Multi-Agent Games," in Proc. UAI, Aug 2022.

  • Jiayu Chen, Dipesh Tamboli, Tian Lan, and Vaneet Aggarwal, "Multi-task Hierarchical Adversarial Inverse Reinforcement Learning," in Proc. ICML, Jul 2023.

  • Jiayu Chen, Tian Lan, Vaneet Aggarwal, "Hierarchical Adversarial Inverse Reinforcement Learning," IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 12, pp. 17549-17558, Dec. 2024.

  • Guangchen Lan, Dong-Jun Han, Abolfazl Hashemi, Vaneet Aggarwal, Christopher G. Brinton, "Asynchronous Federated Reinforcement Learning with Policy Gradient Updates: Algorithm Design and Convergence Analysis," in Proc. ICLR, Apr 2025.

  • Anirudh Satheesh, Keenan Powell, and Vaneet Aggarwal, "Distributionally Robust Self Paced Curriculum Reinforcement Learning," Reinforcement Learning Journal, Aug 2026.

  • Swetha Ganesh, Jiayu Chen, Gugan Thoppe, and Vaneet Aggarwal, "Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries," Transactions on Machine Learning Research, Oct 2024.

  • Yang Xu, Washim Uddin Mondal, and Vaneet Aggarwal, "Finite-Sample Analysis of Policy Evaluation for Robust Average Reward Reinforcement Learning," in Proc. NeurIPS, Dec 2025.

  • Yang Xu, Swetha Ganesh, and Vaneet Aggarwal, "Efficient Q-Learning and Actor-Critic Methods for Robust Average Reward Reinforcement Learning," in Proc. UAI, Aug 2026.

  • Swetha Ganesh and Vaneet Aggarwal, "A Refined Sample-Complexity Analysis of Robust Policy Optimization under Decaying Actor Stepsizes," in Proc. NeurIPS, Dec 2026.

Keywords: adversarial inverse reinforcement learning, adversarial resilience, Byzantine agents, collusion detection, distributional robustness, federated reinforcement learning, robust reinforcement learning