AgroAI: An AI-Powered Intelligent Smart Agriculture Assistant Using Machine Learning and Multimodal Interaction

Abstract

The productivity of agriculture in developing economies like India is not yet advanced owing to the disjointed access to advice, soil erosion, unpredictable weather conditions, and continuous absence of professional advice in the rural areas. The case described in this paper is AgroAI that is a multimodal AI-based smart farming assistant comprising of a YOLOv8-CLS [11] convolutional neural network (CNN) to detect crop diseases, supervised machine learning models to recommend crops and fertilizers based on soil, and a large language model (LLM) chatbot to interact with the region using voice recognition. The module of disease detection was trained on the dataset of PlantVillage [12] that covered 15 classes of diseases and healthy plants. The model obtained a total classification accuracy of 98.8 using transfer learning, data augmentation, and AdamW optimiser with automatic mixed precision with a macro-averaged precision, recall, and F1-score of 0.99. The crop recommendation module uses parameter data of the soil nutrients, weather history, and market price feeds to produce explainable profit-conscious advisory decisions. Progressive Web App (PWA) architecture provides offline capabilities to rural areas with low-connectivity conditions, caching services workers and fallback logic based on rules. The confusion matrix analysis test proves that there is little non-inter-class misclassification mainly between the similar tomato pathologies visually. These results make AgroAI a comprehensive, all-inclusive and deployable platform in support of Indian smallholder farmers.

Introduction

Indian agriculture faces significant challenges due to the gap between advanced agronomic research and its practical use by small farmers. Many farmers still rely on traditional knowledge instead of scientific data, leading to low productivity, resource wastage, and financial instability. Key barriers include limited extension services, language diversity, and poor internet connectivity.

Although technologies like machine learning, deep learning, and NLP have shown strong potential—such as disease detection, crop recommendation, and multilingual advisory—most existing solutions are isolated, require internet access, and lack explainability.

To address these issues, the paper proposes AgroAI, an intelligent, offline-capable agriculture assistant that integrates multiple technologies into a single platform. It combines CNN-based disease detection (YOLOv8), ML-based crop and fertilizer recommendations, a multilingual chatbot (supporting text, voice, and images), and a Progressive Web App (PWA) with offline functionality.

The system uses diverse data sources such as soil data, weather information, market prices, and crop images. It processes both structured (NPK, weather) and unstructured (images, text) data to provide accurate, explainable, and localized recommendations. It also includes offline fallback mechanisms using rule-based advisory for areas with poor connectivity.

Overall, AgroAI offers a unified, scalable, and farmer-friendly solution that improves decision-making, enhances productivity, and bridges the gap between agricultural research and real-world farming practices in India.

Conclusion

The paper has introduced the AgroAI: Intelligent Smart Agriculture Assistant, a multimodal AI-advised assistant platform that will provide solutions to three related gaps in the agricultural advisory ecosystem in India: limited coverage of tools, reliance on the internet, and a non-transparent method of recommendation. The system incorporates a YOLOv8-CLS convolutional neural network to diagnose crop disease, supervised ML models to explain the recommendations of crops and fertilisers based on the soil, an LLM-based multilingual chatbot with voice and image support, and a Progressive Web App architecture with an offline defugalty architecture. The disease detection module is experimentally evaluated on a 15-class test set of 4,138 images on the PlantVillage creating a total of 98.8 per cent with macro-averaged precision, recall, and F1-score all equal to 0.99. These are the first results to be obtained in augmentation conditions to simulate real-world imaging variability, a material improvement over the previous systems which could only perform highly under controlled conditions or, by compromising performance to enable offline operation. The analysis of the confusion matrix proves the existence of residual misclassification that is clinically explainable and concentrated in the pathology pairs that are visually proximate. In addition to the performance of the detection, AgroAI has made its contribution in the form of structural consistency: the first system in the literature reviewed included CNN disease detection, soil-ML recommendation, integration of market prices, multilingual LLM ad dialogue, and offline PWA deployment in a common platform specifically created in Indian smallholder context. The explainability layer, the feature that converts the importance of ML features into plain-language advisory reasoning, directly goes to the barrier of trust that has restricted the application of AI advisory tools in rural settings. There are still significant drawbacks. Next steps are field-level verification in the environment of various Indian imaging conditions, longitudinal analyses of the effects of yield and income, and chatbots with dialect-specific evaluation. The prototype that is now being developed is about 25-30% of the entire system implementation, and final module integration, live field pilots, and admin analytics dashboards will be developed during the next step in the project. When these elements reach a sufficient level of maturity, AgroAI can emerge as a core output upon which millions of Indian farmers will be able to plan their agronomic decisions with the concentration that evidence-based advice can provide.

References

[1] Bansal, R., & Singla, R. (2025). AI-driven agricultural decision support system using weather and soil data. International Journal of Smart Farming Technologies, 12(4), 45–57. [2] Kaggle. “PlantVillage Dataset – Plant Disease Images.” [Online]. Available: https://www.kaggle.com/datasets/emmarex/plantdisease [3] Kamduri, R., & Gupta, A. (2025). Smart advisor for precision agriculture using low-infrastructure AI. Journal of Emerging Agricultural Technologies, 9(1), 12–23. [4] Kamilaris, A., & Prenafeta-Boldú, F. X. (2018). Deep learning in agriculture: A survey. Computers and Electronics in Agriculture, 147, 70–90. [5] Liakos, K. G., Busato, P., Moshou, D., Pearson, S., & Bochtis, D. (2018). Machine learning in agriculture: A review. Sensors, 18(8), 2674. [Online]. Available: https://www.mdpi.com/1424-8220/18/8/2674 [6] Mohanty, S. P., Hughes, D. P., & Salathé, M. (2016). Using deep learning for image-based plant disease detection. Frontiers in Plant Science, 7, 1419. [Online]. Available: https://www.frontiersin.org/articles/10.3389/fpls.2016.01419/full [7] Pantazi, X. E., Moshou, D., & Tamouridou, A. A. (2019). Automated leaf disease detection through image feature analysis and one-class classifiers. Computers and Electronics in Agriculture, 156, 96–104. [8] Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., ... & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. [9] Rajanala, K., Aditya, R., & Prasad, N. (2025). ACVA: Agricultural chatbot voice assistant using MLP and NLP. International Conference on Smart Rural Technologies, 88–95. [10] Sardeshmukh, M., Gupta, A., & Sahu, P. (2025). AI-based smart crop recommendation framework using soil, weather and market data. International Journal of Agricultural Data Science, 7(2), 112–126. [11] Ultralytics. (2023). YOLOv8: A new state-of-the-art computer vision model. [Online]. Available: https://github.com/ultralytics/ultralytics [12] PlantVillage. “Labeled Crop Disease Image Repository.” [Online]. Available: https://plantvillage.psu.edu [13] Agmarknet. Government of India – Agricultural Marketing Portal. [Online]. Available: https://agmarknet.gov.in/

Copyright

Copyright © 2026 Shivam Sharma, Swayam Jaiswal, Ujjwal Ahlawat. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.