Neo NPC Using Deep Learning: Literature Review

Abstract

This paper presents a novel approach to Non-Playable Character (NPC) design, exploring the integration of artificial intelligence to create dynamic, responsive, and evolving behaviors in virtual environments. We examine how Reinforcement Learning (RL) and Neural Networks (NNs) can be leveraged to develop intelligent NPCs capable of adapting to player interactions, retaining memory of past encounters, and evolving strategies over time. It details the implementation of a playable prototype featuring an AI-powered boss character, whose behavior dynamically adjusts based on player actions to offer a unique experience in every session. This paper aims to provide researchers and practitioners with a comprehensive understanding of how machine learning can transform traditional NPC systems. We review current literature on AI in gaming, compare various techniques for adaptive NPC behavior, and outline our system\'s design, emphasizing its ability to produce lifelike, interactive, and emotionally resonant characters

Introduction

The text examines the evolution of NPC (Non-Player Character) behavior in video games, highlighting the shift from traditional scripted logic and finite state machines to modern machine learning–driven systems. Conventional NPCs rely on static rules, making them predictable and non-adaptive. With advances in neural networks and reinforcement learning (RL), developers can now design NPCs that learn from player interactions, remember past events, and evolve strategies dynamically. The paper proposes a modular framework for creating lifelike, context-aware NPCs applicable to games, simulations, and interactive training environments.

The literature review surveys research on deep reinforcement learning, AI-driven dialog systems, hybrid frameworks combining FSM and RL, and industry case studies. Collectively, the studies show a clear trend: from rigid, rule-based systems toward adaptive, data-driven NPCs capable of context awareness, emotional responsiveness, and real-time learning. Several works explore DRL algorithms (DQN, PPO, A3C), hierarchical and curriculum learning, hybrid FSM-RL architectures, and applications in platformers, fighting games, and multiplayer FPS environments. Industrial examples illustrate cloud-based learning and advanced AI engines that personalize NPC strategies for players.

Despite advances, challenges remain—including computational power requirements, training instability, limited generalization, reliance on proprietary systems, and the difficulty of balancing human-aligned behavior. Surveys also note the lack of standard evaluation environments and reproducibility issues across gaming studies. Traditional FSM and MCTS methods remain relevant for deterministic or planning-heavy tasks but lack scalability for open-ended, adaptive behavior.

Conclusion

The literature reviewed highlights the progressive shift in NPC behavior modeling from rule-based systems such as Finite State Machines to adaptive learning techniques including Deep Reinforcement Learning and hybrid AI architectures. While FSMs remain useful for deterministic control and simplicity, they fall short in dynamic or player-responsive scenarios. Reinforcement learning methods have demonstrated strong potential in developing agents capable of emergent, human-like behaviors, though they often face issues related to training complexity, generalization, and computational cost. Hybrid approaches—combining FSM structure with neural adaptability or integrating LLMs for contextual reasoning—represent a promising direction for developing scalable and intelligent NPCs. Commercial implementations such as NVIDIA\'s Asterion and HELT-trained agents in Naruto Mobile provide evidence that adaptive NPC systems are no longer just academic exercises but viable in large-scale game environments. Overall, the surveyed works establish a foundation for modular, memory-augmented, and context-aware NPC frameworks, positioning them as central to the future of game AI.

References

REFERENCES [1] Natalia Khan, N. Sabahat, “AI Chatbots in Gaming Experience Transformation,” International Journal of Scientific Research in Computer Science, Engineering and Information Technology, Vol. 9, Issue 1, 2023. [2] Kun Shao, Yufeng Yuan, Jie Zhang, Dongbin Zhao, “A Survey of Deep Reinforcement Learning in Video Games,” Computational Intelligence and Neuroscience, Vol. 2022, Article ID 7061722, 22 pages. [3] WeMade, NVIDIA, “AI-Powered NPCs in MMORPG: Asterion Case Study,” NVIDIA Technical Report, 2025. [4] Tejal Kadam, Archisha Chandel, Akanksha Dubey, Rahul Patil, “Reinforcement Learning Bot for Mario,” International Journal of Scientific Research in Engineering and Management, Vol. 4, Issue 9, 2020. [5] Natalia Curado Carneiro, “FSM-Based Behavior Modeling for Mario,” International Journal of Advanced Computer Science and Applications, Vol. 12, No. 10, 2021. [6] Pedro Almeida, Vítor Carvalho, Alberto Simões, “Reinforcement Learning as an Approach to Train Multiplayer First-Person Shooter Game Agents,” Applied Sciences, Vol. 14, Issue 4, 2024. [7] Shouren Wang, Zehua Jiang, Fernando Sliva, Sam Earle, Julian Togelius, “Enhancing Player Enjoyment with a Two Tier DRL and LLM Based Agent System for Fighting Games,” arXiv preprint, arXiv:2504.07425, April 2025. [8] Chen Zhang, Qiang He, Yuan Zhou, Elvis S. Liu, Hong Wang, Jian Zhao, Yang Wang, “Advancing DRL Agents in Commercial Fighting Games: Training, Integration, and Agent-Human Alignment,” IEEE Transactions on Games, Early Access, 2024. [9] Iveta Dirgová Luptáková, Martin Kubov?ík, Ji?í Pospíchal, “Playing Flappy Bird Based on Motion Recognition Using a Transformer Model and LIDAR Sensor,” Sensors, Vol. 24, No. 2, 2024. [10] Maciej ?wiechowski, Konrad Godlewski, Bartosz Sawicki, Jacek Ma?dziuk, “Monte Carlo Tree Search: A Review of Recent Modifications and Applications,” Artificial Intelligence Review, Vol. 56, 2023, pp. 841–907.

Copyright

Copyright © 2025 Suraj S J, Dr. Sankhya Nayak, Shreyas G Hegde, Sujay Bharadwaj K, Ullas L M. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.