AI-Driven Self-Healing Automated UI Testing Framework with Visual Proof

Abstract

Automateduser-interface (UI)testingisessentialfor continuoussoftware delivery, yetitremains fragile in the face of frequent front-end changes. Conventional automation frame- works (Selenium, Cypress, Playwright) depend on static locators and basic assertion primitives that are prone to breakage and produce flaky results. Recent research and commercial developments propose AI-enabled strategies to reduce fragility: self- healing locators, perceptual visual checks, and intelligent reporting.This review synthesizes literature from 2018– 2025, identifies key deficiencies in existing techniques, and articulates a unified architecture that integrates(a) hybrid AI-driven locator healing, (b) semantic Visual Proof Validation (VPV) inspired by Vision Transformer architectures, and (c) dual-format AI reporting with automated rerun orchestration.We present comparative analysis, identify research gaps, and discuss prototype results demonstrating improved healing accuracy and actionable test intelligence.

Introduction

Automated User Interface (UI) testing is a critical component of modern software development, particularly within Agile and CI/CD pipelines, ensuring rapid releases without compromising functional and visual integrity. However, test fragility caused by evolving front-end frameworks, dynamic DOM structures, and CSS changes results in high maintenance costs and unstable test suites. Traditional automation tools like Selenium and Cypress provide robust scripting but lack adaptability against such dynamic changes.

Recent advancements in Artificial Intelligence (AI), Deep Learning (DL), and Computer Vision (CV) have introduced self-healing capabilities that automatically repair broken locators and perform semantic-level visual verification. Vision Transformers (ViT) and contrastive learning now enable context-aware visual comparisons, surpassing traditional pixel-diff approaches. Similarly, Large Language Models (LLMs) are being explored for automated test generation and failure explanation, although their integration with runtime healing and visual proofing remains limited.

The literature from 2018–2025 shows a transition from rule-based heuristics to hybrid AI-driven frameworks combining deep embeddings, semantic validation, and adaptive learning. Despite this progress, major research gaps persist:

1. Limited integration of semantic visual verification with DOM-based healing.

2. Lack of continuous learning pipelines to adapt models as interfaces evolve.

3. Poor integration between test generation, runtime healing, and visual validation.

4. Deficient explainability and auditability of AI-driven healing decisions.

5. Challenges in cross-platform generalization across web and mobile UIs.

6. High computational costs hindering scalability in CI/CD pipelines.

7. Unaddressed security and privacy concerns related to captured UI data.

Future research emphasizes hybrid multimodal frameworks that unify DOM, vision, and language cues; online and active learning pipelines; and explainable, resource-efficient, privacy-preserving architectures.

The proposed AI-driven framework aims to integrate locator self-healing with semantic Visual Proof Validation (VPV) through a six-stage loop—covering test execution, healing, visual verification, and reporting. This closed-loop architecture enhances resilience, transparency, and automation reliability, paving the way for intelligent, self-adaptive, and human-comprehensible UI testing systems capable of supporting modern continuous delivery environments.

Conclusion

This review and prototype study argues that the next generation of UI test automation should transcend brittle locatorsandnaivevisualdiffsbyintegratinghybridAIhealing,semanticvisualproof,andactionableAIreporting.The combined approach yields substantial improvements in healing accuracy and maintenance efficiency while providing the explainability necessary for human trust and governance.

References

[1] D.S.Battina,“Artificialintelligenceinsoftwaretestautomation:Asystematicliteraturereview,”Int.J.Emerg.Technol.Innov.Res.,vol. 6, no. 5, pp. 120–127, 2019. [2] R.Khankhoje,“AnIn-DepthReviewofTest AutomationFrameworks: TypesandTrade-offs,”IJARSCT,vol.3, no.1,pp.55–64,2023 [3] K.Pelluru,“AI-DrivenDevOpsOrchestrationinCloudEnvironments,”Int.J.Sci.Technol., vol.1,no.6, pp.1–15,2024. [4] A.Jime´nez-Ram´?rezetal.,“Automatedtestinginroboticprocessautomationprojects,”J.Softw.:Evol.Process,vol.35,no.3,2023. [5] Z.Liuetal.,“ChattingwithGPT-3for zero-shotmobileautomatedGUItesting,”arXivpreprintarXiv:2305.09434,2023. [6] M.Scha¨fer etal.,“AnempiricalevaluationofLLMs forautomatedunittestgeneration,”IEEETrans.Softw.Eng.,2023. [7] R.Feldtetal.,“Towardsautonomoustestingagentsvia conversationalLLMs,” inProc.ASEConf.,2023. [8] K.Lietal.,“Utilizingdeeplearningtooptimizesoftwaredevelopmentprocesses,” arXivpreprintarXiv:2404.13630,2024. [9] S.Lukasczyketal.,“Automatedunittestgeneration forPython,”EmpiricalSoftw.Eng.,vol.28,no.2,2023. [10] Q.Guoetal.,“ExploringChatGPTinautomatedcoderefinement,”inProc.ICSE,2024. [11] N.Alshahwanetal.,“AutomatedunittestimprovementusingLLMsatMeta,”inProc.FSE,2024. [12] R.Patel andA.Saha,“Reviewonvisualregressiontestingframeworks,”ACMSIGSOFT Notes,vol.50,no.1,2025. [13] Y.Chenetal.,“Transformer-basedvisual validationformobileUItesting,”SpringerLNCS,vol.14532,pp.310–322,2025. [14] P.Singh,M.Gupta,andA.Mehra,“Adaptiveself-healinglocators withLLM-driventestrepair,”IEEE Access,vol.13,pp.56789–56803,A.2025. [15] J.Zhang,R.Lin,andP.Bose,“IntegratingVisionTransformers withDOMcontextforUIanomalydetection,”IEEETrans.Neural [16] Netw.Learn.Syst.,vol.36,no.9,pp.12045–12059,2025. [17] M.H.Leeand A.Fernandez,“LLM-assistedself-healing frameworksfor end-to-endtesting,”EmpiricalSoftw.Eng.,vol.30,no.4,A.2025. [18] V.SharmaandK.Rao,“Surveyonmultimodal learningfor softwarequalityassurance,”ACMComput.Surv.,vol.56,no.8,2024. [19] C.D.Nguyen,“RobustUIlocatorrecoveryusingcontrastivepretraining,” Softw.Test.Verif.Reliab.,vol.34,no.1,2025. [20] T.Oliveira etal.,“Self-healingtestcasesvia contextualattention mechanisms,”inProc.IEEEICSEWorkshops,2024. [21] D.Patel,“EvaluatingexplainableAItechniquesforvisualvalidation,”IEEEAccess,vol. 2025

Copyright

Copyright © 2025 Payal Chede, P. A. Tijare. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.