AgroAI: An AI-Powered Intelligent Smart Agriculture Assistant Using Artificial Intelligence and Multimodal Interaction

Abstract

The productivity of agriculture in developing economies like India is not yet advanced owing to the disjointed access to advice, soil erosion, unpredictable weather conditions, and continuous absence of professional advice in the rural areas. The case described in this paper is AgroAI that is a multimodal AI-based smart farming assistant comprising of a YOLOv8-CLS convolutional neural network (CNN) to detect crop diseases, supervised machine learning models to recommend crops and fertilizers based on soil, and a large language model (LLM) chatbot to interact with the region using voice recognition. The module of disease detection was trained on the dataset of PlantVillage that covered 15 classes of diseases and healthy plants. The model obtained a total classification accuracy of 98.8 using transfer learning, data augmentation, and AdamW optimiser with automatic mixed precision with a macro-averaged precision, recall, and F1-score of 0.99. The crop recommendation module uses parameter data of the soil nutrients, weather history, and market price feeds to produce explainable profit-conscious advisory decisions. Progressive Web App (PWA) architecture provides offline capabilities to rural areas with low-connectivity conditions, caching services workers and fallback logic based on rules. The confusion matrix analysis test proves that there is little non-inter-class misclassification mainly between the similar tomato pathologies visually. These results make AgroAI a comprehensive, all-inclusive and deployable platform in support of Indian smallholder farmers.

Introduction

The text introduces AgroAI: Intelligent Smart Agriculture Assistant, an AI-powered platform designed to help Indian farmers make informed agricultural decisions by integrating crop disease detection, crop recommendation, multilingual communication, and offline functionality into a single system.

Problem Statement

Agriculture supports the livelihoods of nearly 600 million Indians, yet many farmers still rely on traditional experience-based practices for crop selection, fertilizer use, irrigation, and pest management. Limited access to scientific advisory services, language barriers, and poor internet connectivity often result in:

• Lower crop yields
• Inefficient use of resources
• Post-harvest losses
• Financial instability among farmers

Although technologies such as machine learning, deep learning, and natural language processing have shown promise in agriculture, existing solutions are typically fragmented, internet-dependent, and difficult for farmers to trust due to their lack of explainability.

Proposed Solution: AgroAI

AgroAI addresses these limitations by combining multiple AI technologies into one unified platform:

1. Disease Detection
   • Uses a YOLOv8-CLS convolutional neural network trained on the PlantVillage dataset.
   • Diagnoses crop diseases from leaf images in real time.
2. Agricultural Recommendations
   • Employs machine learning models such as Random Forests, Decision Trees, and K-Nearest Neighbors.
   • Provides recommendations for crops, fertilizers, and irrigation based on:
     • Soil nutrients (N, P, K)
     • Soil pH
     • Weather conditions
     • Historical yield data
     • Market prices
3. Multilingual Chatbot
   • Supports text, voice, and image inputs.
   • Uses NLP, speech-to-text, and text-to-speech technologies.
   • Communicates in regional Indian languages such as Hindi, Marathi, Tamil, Telugu, and Bengali.
4. Offline Capability
   • Implemented as a Progressive Web App (PWA).
   • Stores essential data locally and provides rule-based recommendations even without internet access.

Literature Review Findings

Previous research has successfully developed:

• Crop recommendation systems
• CNN-based disease detection models
• Agricultural chatbots
• Precision irrigation systems

However, existing systems generally:

• Focus on only one agricultural task.
• Depend heavily on cloud computing and internet connectivity.
• Lack explainable recommendations.
• Do not combine disease diagnosis, crop advisory, chatbot interaction, market intelligence, and offline support into a single platform.

AgroAI aims to fill this gap by providing an integrated, explainable, and farmer-friendly solution.

Methodology

Data Collection

Data was collected from multiple sources:

• PlantVillage dataset for disease images
• ICAR and Kaggle datasets for soil and crop information
• OpenWeatherMap API for weather data
• Agmarknet portal for market prices

Data Preprocessing

• Soil data was cleaned and normalized.
• Crop images were resized and enhanced using data augmentation techniques such as:
  • Rotation
  • Cropping
  • Brightness and contrast adjustments
  • Horizontal flipping

Crop Recommendation Module

• Trained using soil nutrients, pH, weather history, and yield records.
• Market prices were incorporated to recommend profitable crops.

Disease Detection Module

• Fine-tuned a pretrained YOLOv8 Nano Classification model.
• Used transfer learning, AdamW optimization, early stopping, and stratified sampling.
• Dataset split:
  • 70% training
  • 15% validation
  • 15% testing

Chatbot Development

• Integrated:
  • Whisper speech-to-text
  • Google text-to-speech
  • Large Language Models (LLMs)
  • Dialogflow intent classification
• Supports both simple and complex agricultural queries.

Offline Advisory System

• Uses service workers and IndexedDB to store data locally.
• Provides recommendations through a rule-based engine when internet access is unavailable.

Evaluation

The system was evaluated using:

• Accuracy
• Precision
• Recall
• F1-score
• Confusion matrix
• System Usability Scale (SUS)

Dataset Description

The disease detection model used the PlantVillage dataset, containing approximately 54,000 labeled leaf images from crops such as:

• Tomato
• Potato
• Bell Pepper

The dataset includes 15 classes representing healthy and diseased plants. Despite class imbalance, techniques such as stratified sampling and data augmentation helped maintain strong performance.

Model Architecture

The disease classification component uses YOLOv8-CLS, a classification variant of YOLOv8. It consists of:

• A CNN backbone for feature extraction
• Global average pooling
• A fully connected softmax classifier

The model learns visual patterns from crop images and predicts disease categories with high accuracy.

Conclusion

The paper has introduced the AgroAI: Intelligent Smart Agriculture Assistant, a multimodal AI-advised assistant platform that will provide solutions to three related gaps in the agricultural advisory ecosystem in India: limited coverage of tools, reliance on the internet, and a non-transparent method of recommendation. The system incorporates a YOLOv8-CLS convolutional neural network to diagnose crop disease, supervised ML models to explain the recommendations of crops and fertilisers based on the soil, an LLM-based multilingual chatbot with voice and image support, and a Progressive Web App architecture with an offline defugalty architecture. The disease detection module is experimentally evaluated on a 15-class test set of 4,138 images on the PlantVillage creating a total of 98.8 per cent with macro-averaged precision, recall, and F1-score all equal to 0.99. These are the first results to be obtained in augmentation conditions to simulate real-world imaging variability, a material improvement over the previous systems which could only perform highly under controlled conditions or, by compromising performance to enable offline operation. The analysis of the confusion matrix proves the existence of residual misclassification that is clinically explainable and concentrated in the pathology pairs that are visually proximate. In addition to the performance of the detection, AgroAI has made its contribution in the form of structural consistency: the first system in the literature reviewed included CNN disease detection, soil-ML recommendation, integration of market prices, multilingual LLM ad dialogue, and offline PWA deployment in a common platform specifically created in Indian smallholder context. The explainability layer, the feature that converts the importance of ML features into plain-language advisory reasoning, directly goes to the barrier of trust that has restricted the application of AI advisory tools in rural settings. There are still significant drawbacks. Next steps are field-level verification in the environment of various Indian imaging conditions, longitudinal analyses of the effects of yield and income, and chatbots with dialect-specific evaluation. The prototype that is now being developed is about 25-30% of the entire system implementation, and final module integration, live field pilots, and admin analytics dashboards will be developed during the next step in the project. When these elements reach a sufficient level of maturity, AgroAI can emerge as a core output upon which millions of Indian farmers will be able to plan their agronomic decisions with the concentration that evidence-based advice can provide.

References

[1] Bansal, R., & Singla, R. (2025). AI-driven agricultural decision support system using weather and soil data. International Journal of Smart Farming Technologies, 12(4), 45–57. [2] Kaggle. “PlantVillage Dataset – Plant Disease Images.” [Online]. Available: https://www.kaggle.com/datasets/emmarex/plantdisease [3] Kamduri, R., & Gupta, A. (2025). Smart advisor for precision agriculture using low-infrastructure AI. Journal of Emerging Agricultural Technologies, 9(1), 12–23. [4] Kamilaris, A., & Prenafeta-Boldú, F. X. (2018). Deep learning in agriculture: A survey. Computers and Electronics in Agriculture, 147, 70–90. [5] Liakos, K. G., Busato, P., Moshou, D., Pearson, S., & Bochtis, D. (2018). Machine learning in agriculture: A review. Sensors, 18(8), 2674. [Online]. Available: https://www.mdpi.com/1424-8220/18/8/2674 [6] Mohanty, S. P., Hughes, D. P., & Salathé, M. (2016). Using deep learning for image-based plant disease detection. Frontiers in Plant Science, 7, 1419. [Online]. Available: https://www.frontiersin.org/articles/10.3389/fpls.2016.01419/full [7] Pantazi, X. E., Moshou, D., & Tamouridou, A. A. (2019). Automated leaf disease detection through image feature analysis and one-class classifiers. Computers and Electronics in Agriculture, 156, 96–104. [8] Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., ... & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. [9] Rajanala, K., Aditya, R., & Prasad, N. (2025). ACVA: Agricultural chatbot voice assistant using MLP and NLP. International Conference on Smart Rural Technologies, 88–95. [10] Sardeshmukh, M., Gupta, A., & Sahu, P. (2025). AI-based smart crop recommendation framework using soil, weather and market data. International Journal of Agricultural Data Science, 7(2), 112–126. [11] Ultralytics. (2023). YOLOv8: A new state-of-the-art computer vision model. [Online]. Available: https://github.com/ultralytics/ultralytics [12] PlantVillage. “Labeled Crop Disease Image Repository.” [Online]. Available: https://plantvillage.psu.edu [13] Agmarknet. Government of India – Agricultural Marketing Portal. [Online]. Available: https://agmarknet.gov.in/

Copyright

Copyright © 2026 Shivam Sharma, Swayam Jaiswal, Ujjwal Ahlawat, Manoj Kumar Dixit. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.