AI Agent-Driven Datacenter Optimizer: A Guardrailed, Explainable LLM Orchestrator for Cloud Platform Management

Abstract

Modern datacenters must balance three objectives that often conflict: keeping hardware within safe thermal limits, meeting latency expectations for high-priority work, and reducing the cost and carbon footprint of electricity. Traditional controllers handle these with fixed rules or heavily engineered optimization pipelines. This paper presents Grid Control, an autonomous datacenter orchestration system in which a large language model (LLM) acts as the decision-making agent. A simulated multi-region datacenter (5 regions, 15 servers, 60 chipsets) streams telemetry every second. Every decision interval the agent receives a frozen telemetry window and selects exactly one action from a small, closed vocabulary: shift_within_server, shift_within_region, migrate_region, or pass. A prioritized policy ranks overheating first, latency and load balance second, and cost and carbon third. Safety is enforced outside the model by a guardrail layer that routes high-criticality or administrator-prohibited actions to a human approval queue. A feedback loop returns measured outcomes of past actions to the prompt as in-context learning signal, and a scope-restricted explainability module answers operator questions about the agent\'s behavior. Over 20 paired simulation runs, a deterministic stand-in for the agent\'s policy reduced time spent above the 68 °C threshold by about 24% and carbon-weighted load by about 9% relative to a no-agent baseline, with about 3% lower electricity cost. LLM-in-the-loop evaluation is left to future work. The contribution is an architecture and a reproducible testbed for studying safe, explainable, LLM-driven infrastructure control.

Introduction

Datacenters are essential to the digital economy but consume increasing amounts of electricity due to growing computational demands, particularly from Artificial Intelligence workloads. Traditional approaches to datacenter management, including carbon-aware scheduling, machine learning-based cooling, and cluster management systems, can improve energy efficiency but often require specialized models, forecasting systems, and extensive engineering. This study proposes an LLM-based datacenter control framework called Grid Control to optimize workload placement by balancing thermal safety, latency, resource utilization, electricity costs, and carbon emissions.

The proposed framework uses a Large Language Model (LLM) to analyze structured telemetry data and select appropriate actions according to a predefined priority policy. Its architecture follows a six-component operational loop: sensing the datacenter state, making decisions, applying safety checks, executing permitted actions, observing outcomes, and providing feedback. The LLM operates within a simulated environment containing five geographically distributed regions, 15 servers, and 60 chipsets. These regions differ in electricity prices, carbon intensity, ambient temperature, and computing capacity, creating realistic trade-offs for workload placement.

The decision-making process follows three priorities:

1. Thermal safety: Move eligible tasks away from chipsets exceeding the 68°C temperature threshold.

2. Latency and load balancing: Move high-priority tasks to less-loaded servers when utilization differences exceed the specified threshold.

3. Cost and carbon optimization: Migrate tasks to regions offering lower electricity costs or carbon emissions when the projected benefits exceed migration costs.

The framework incorporates deterministic guardrails to ensure safe execution. The LLM cannot directly modify infrastructure; its proposed actions must pass JSON validation, policy checks, and environment-side precondition checks. Low-risk actions may execute automatically, while prohibited or high-criticality actions require administrative review. Invalid outputs, unrecognized actions, and provider errors trigger fail-safe handling rather than unsafe execution.

An in-context feedback mechanism records the effects of executed actions on temperature, utilization, electricity prices, and carbon intensity. These results are included in subsequent prompts, allowing the model to adjust future decisions without retraining its parameters. Additionally, an explainability module enables administrators to ask questions about recent decisions, blocked actions, and active policies. Its read-only design, restricted context, and access protections improve transparency while limiting operational and security risks.

The study addresses a research gap by combining LLM-based reasoning, a bounded action space, deterministic safety controls, human oversight, and feedback-driven adaptation in a reproducible simulated datacenter environment. Unlike conventional controllers that may require retraining or specialized models for new objectives, Grid Control allows its priorities to be adjusted through policy prompts while maintaining execution safeguards.

In conclusion, Grid Control presents a promising approach to adaptive, explainable, and energy-aware datacenter management. Its combination of LLM reasoning, deterministic guardrails, human approval, and outcome-based feedback could support more efficient workload placement while reducing thermal risks, electricity costs, and carbon emissions. However, its practical effectiveness must be established through systematic experiments measuring energy consumption, latency, thermal safety, carbon emissions, decision quality, and reliability under different workloads and operating conditions.

Conclusion

This paper presented an AI agent-driven datacenter optimizer that combines an LLM decision-maker with a bounded action vocabulary, deterministic guardrails, human approval for risky actions, in-context outcome feedback, and a scoped explainability interface, all on an open simulated testbed. The architecture\'s central idea is that an LLM is most useful and least dangerous when it is treated as an untrusted proposer: its output is schema-validated, policy-gated, and re-checked against live state before any effect. Offline experiments on the testbed show that following the prescribed priority order reduces thermal excursions (?24% time above threshold) and carbon-weighted load (?9%) with modest cost savings (?3%) relative to naive placement, while almost never producing invalid actions. The framework provides a practical starting point for studying safe, transparent, natural-language-configurable infrastructure control.

References

[1] Masanet, E., Shehabi, A., Lei, N., Smith, S., & Koomey, J. (2020). Recalibrating global data center energy-use estimates. Science, 367(6481), 984-986. [2] Radovanovi?, A., Koningstein, R., Schneider, I., Chen, B., Duarte, A., Roy, B., ... & Cirne, W. (2022). Carbon-aware computing for datacenters. IEEE Transactions on Power Systems, 38(2), 1270-1280. [3] Wiesner, P., Behnke, I., Scheinert, D., Gontarska, K., & Thamsen, L. (2021, December). Let\'s wait awhile: How temporal workload shifting can reduce carbon emissions in the cloud. In Proceedings of the 22nd International Middleware Conference (pp. 260-272). [4] Lazic, N., Boutilier, C., Lu, T., Wong, E., Roy, B., Ryu, M. K., & Imwalle, G. (2018). Data center cooling using model-predictive control. Advances in Neural Information Processing Systems, 31. [5] Mao, H., Alizadeh, M., Menache, I., & Kandula, S. (2016, November). Resource management with deep reinforcement learning. In Proceedings of the 15th ACM workshop on hot topics in networks (pp. 50-56). [6] Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., & Wilkes, J. (2015, April). Large-scale cluster management at Google with Borg. In Proceedings of the tenth european conference on computer systems (pp. 1-17). [7] Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022). React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629. [8] Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., & Yao, S. (2023). Reflexion: Language agents with verbal reinforcement learning. Advances in neural information processing systems, 36, 8634-8652. [9] Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016). Concrete problems in AI safety. arXiv preprint arXiv:1606.06565. [10] Rebedea, T., Dinu, R., Sreedhar, M. N., Parisien, C., & Cohen, J. (2023, December). Nemo guardrails: A toolkit for controllable and safe llm applications with programmable rails. In Proceedings of the 2023 conference on empirical methods in natural language processing: system demonstrations (pp. 431-445).

Copyright

Copyright © 2026 Jay Dighore, Vedant Gaidhane, Ayush Dumbhare, Kunal Kanchankar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.