Improving Non-Playable-Characters (NPC) in Games Using Reinforcement Learning

Abstract

Non-Player Characters (NPCs) play a critical role in shaping player experience in modern digital games, yet traditional NPC behaviour is largely driven by static, rule-based logic that lacks adaptability and realism. Such approaches often result in predictable and exploitable gameplay, limiting long-term player engagement. This project presents Neo NPC, a reinforcement learning–based framework for developing adaptive NPC behaviour in a 2D fighting game environment. The proposed system leverages offline reinforcement learning to train intelligent NPC agents that exhibit progressively sophisticated combat strategies across predefined difficulty levels, namely Easy, Medium, and Hard. The study focuses on designing a structured training and deployment pipeline that decouples model learning from real-time gameplay execution. Gameplay environments are modelled to generate state-action-reward trajectories, which are used to train NPC policies using the Proximal Policy Optimization (PPO) algorithm. Trained policies are periodically evaluated, versioned, and categorized into difficulty tiers based on quantitative performance metrics such as win rate, damage efficiency, and survival time. These validated models are then integrated into the game engine through a modular inference layer, enabling real-time decision-making without modifying core game logic. Experimental results demonstrate clear behavioural differentiation across difficulty levels, with higher-tier models exhibiting improved defensive responses, reduced vulnerability to repetitive player strategies, and increased action diversity. Human playtesting further confirms that the adaptive NPCs provide a more challenging and engaging gameplay experience compared to traditional scripted opponents. The proposed approach highlights the effectiveness of reinforcement learning in producing scalable, reusable, and intelligent NPC behaviour. Future extensions of this work include online adaptation, multi-agent self- play, and transfer of trained models to more complex game environments.

Introduction

Neo NPC – Interact, Adapt, Evolve is a project that seeks to transform Non-Playable Characters (NPCs) in video games by making them intelligent, adaptive, and context-aware. Traditionally, NPCs operate on rigid, scripted rules, resulting in predictable and emotionally flat interactions that reduce player immersion. Neo NPC introduces NPCs driven by pre-trained reinforcement learning (RL) models, allowing them to exhibit progressively complex behaviors (Easy, Medium, Hard) without relying on real-time self-evolution. The goal is to make NPCs appear lifelike, responsive, and contextually aware, enhancing the overall gameplay experience.

Objectives:

1. Develop a modular NPC framework integrating RL, neural networks, and behavior-driven AI.

2. Enable NPCs to act based on learned models rather than scripted logic, ensuring immersive and varied interactions.

Literature Survey Highlights:

• Adaptive and context-aware NPCs: AI chatbots and hybrid AI frameworks enhance storytelling, dialogue, and emotional responsiveness but face computational and coherence challenges ([Khan, 2023]).

• Deep Reinforcement Learning (DRL): Surveys highlight DRL algorithms like DQN, PPO, and A3C, emphasizing transfer learning, multi-agent coordination, and scalability issues ([Shao, 2022]).

• Case Studies:

  • Asterion in MIR5 demonstrates cloud-based learning for real-time adaptive NPCs but requires proprietary hardware ([Wemade Next, 2025]).

  • RL bots for Super Mario show emergent behavior through reward shaping, with hierarchical RL suggested for complex tasks ([Kadam, 2020]).

  • FSM-based NPCs offer predictable behavior but lack adaptability and learning ([Carneiro, 2021]).

• Multiplayer and competitive games:

  • RL and hybrid models improve FPS and fighting game agents, balancing aggression, defense, and player-aligned feedback ([Almeida, 2024; Wang, 2025; Zhang, 2024]).

  • Two-tier systems combining DRL and LLM-based agents enhance player experience through adaptive matchups but increase complexity ([Wang, 2025]).

• Novel input methods: Using LIDAR and Transformers for motion-based gameplay improves temporal understanding but introduces higher computational demands ([Luptáková, 2024]).

Key Takeaways:
Neo NPC builds on prior research in RL, DRL, hybrid AI, and context-aware systems to develop NPCs that are immersive, adaptive, and capable of learned behaviors. The project addresses limitations of traditional scripted NPCs, focusing on scalability, realism, and dynamic interaction in increasingly complex game environments.

Conclusion

The Neo NPC system successfully demonstrates how reinforcement learning can be used to create adaptive, realistic, and scalable NPC behaviour in game environments. By training the agent through repeated interaction and evaluating multiple checkpoints, the project delivers three balanced difficulty levels—Easy, Medium, and Hard—without relying on scripted patterns or fixed decision trees. The modular architecture allows trained models to be integrated seamlessly into the game engine, enabling NPCs to react intelligently to player actions and exhibit more dynamic combat strategies. The results show clear improvements in responsiveness, adaptiveness, and player challenge compared to traditional rule-based NPCs, validating the effectiveness of the proposed design and implementation. The project also demonstrates the advantages of using a modular pipeline, where training, evaluation, and deployment remain independent yet well-connected stages. This separation ensures that updated models or improved training methods can be integrated without altering the core game logic. As a result, Neo NPC remains flexible for future enhancements and scalable across different game genres and mechanics Furthermore, the comparative results between rule-based NPCs and reinforcement- learning-driven NPCs show a clear improvement in challenge, engagement, and unpredictability. The system proves that even with limited inputs and simplified environments, reinforcement learning can generate believable behaviours that significantly elevate the overall gameplay experience. Neo NPC, therefore, acts as a strong foundation for more advanced AI-driven game designs.

References

[1] Natalia Khan, N. Sabahat, “AI Chatbots in Gaming Experience Transformation,” International Journal of Scientific Research in Computer Science, Engineering and Information Technology, Vol. 9, Issue 1, 2023. [2] Kun Shao, Yufeng Yuan, Jie Zhang, Dongbin Zhao, “A Survey of Deep Reinforcement Learning in Video Games,” Computational Intelligence and Neuroscience, Vol. 2022, Article ID 7061722, 22 pages. [3] WeMade, NVIDIA, “AI-Powered NPCs in MMORPG: Asterion Case Study,” NVIDIA Technical Report, 2025. [4] Tejal Kadam, Archisha Chandel, Akanksha Dubey, Rahul Patil, “Reinforcement Learning Bot for Mario,” International Journal of Scientific Research in Engineering and Management, Vol. 4, Issue 9, 2020. [5] Natalia Curado Carneiro, “FSM-Based Behaviour Modeling for Mario,” International Journal of Advanced Computer Science and Applications, Vol. 12, No. 10, 2021. [6] Pedro Almeida, Vítor Carvalho, Alberto Simões, “Reinforcement Learning as an Approach to Train Multiplayer First-Person Shooter Game Agents,” Applied Sciences, Vol. 14, Issue 4, 2024. [7] Shouren Wang, Zehua Jiang, Fernando Sliva, Sam Earle, Julian Togelius, “Enhancing Player Enjoyment with a Two Tier DRL and LLM Based Agent System for Fighting Games,” arXiv preprint, arXiv:2504.07425, April 2025. [8] Chen Zhang, Qiang He, Yuan Zhou, Elvis S. Liu, Hong Wang, Jian Zhao, Yang Wang, “Advancing DRL Agents in Commercial Fighting Games: Training, Integration, and Agent-Human Alignment,” IEEE Transactions on Games, Early Access, 2024. [9] Iveta Dirgová Luptáková, Martin Kubov?ík, Ji?í Pospíchal, “Playing Flappy Bird Based on Motion Recognition Using a Transformer Model and LIDAR Sensor,” Sensors, Vol. 24, No. 2, 2024. [10] Maciej ?wiechowski, Konrad Godlewski, Bartosz Sawicki, Jacek Ma?dziuk, “Monte Carlo Tree Search: A Review of Recent Modifications and Applications,” Artificial Intelligence Review, Vol. 56, 2023, pp. 841–907. [11] Muhammad Bambang Firdaus, Alan Zulfikar Waksito, Andi Tejawati, Medi Taruk, M. Khairul Anam, Akhmad Irsyad, “FSM for Retro Arcade Fighting Game,” Journal of Physics: Conference Series, Vol. 2632, 2023. [12] Natalia Curado Carneiro, Tejal Kadam, Archisha Chandel, Rahul Patil, “Integrating FSM and Reinforcement Learning for Hybrid NPCs,” International Journal of Interactive Multimedia and Artificial Intelligence, Vol. 7, No. 3, 2022. [13] Afzal Hussain, Abdul Rauf, Mian Rizwan, “A Survey on Unity Game Engine: Architecture, Features, and Its Applications,” Indonesian Journal of Electrical Engineering and Computer Science, Vol. 29, No. 2, 2023.

Copyright

Copyright © 2025 Suraj S J, Dr. Sankhya Nayak, Shreyas G Hegde, Sujay Bharadwaj K, Ullas L M. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.