Text to SQL Query

Abstract

With the exponential growth of data in modern organizations, the ability to extract meaningful insights from databases has become crucial. However, interacting with structured databases often requires knowledge of SQL (Structured Query Language), which presents a barrier for non-technical users. To address this challenge, this project proposes a Text to SQL system that enables users to retrieve data from relational databases by simply expressing their queries in natural language. The goal is to bridge the gap between human language and machine-readable SQL commands through the use of Natural Language Processing (NLP) and machine learning techniques. The system is designed to accept a natural language input, process and understand its intent, and then convert it into an equivalent SQL query that can be executed on a target database. The core methodology involves text preprocessing, tokenization, semantic parsing, and SQL query generation. Modern NLP models, including transformer-based architectures, are explored to improve the understanding of context and the mapping between language and database schema.

Introduction

Overview:

This project presents a Text-to-SQL system that enables non-technical users to query databases using natural language instead of SQL. It aims to democratize data access in sectors like business, education, healthcare, and research by removing the dependency on technical teams for database interactions.

Problem Statement:

• SQL expertise is required to extract meaningful data, limiting access for non-technical professionals.

• Existing NLP-based systems struggle with:

  • Ambiguity in natural language

  • Schema adaptation across databases

  • Complex SQL generation (e.g., joins, aggregations)

Objectives:

1. Design a user-friendly interface that accepts natural language and outputs accurate SQL queries.

2. Ensure syntactic correctness and semantic accuracy of generated queries.

3. Maintain compatibility across multiple database schemas.

4. Improve accessibility through a web-based application (Streamlit) with immediate data feedback.

Methodology:

• Frontend: Built using Streamlit for real-time user interaction.

• Backend & Logic:

  • Natural language queries are processed and passed to the Gemini Pro LLM (Google Generative AI API).

  • The LLM returns SQL queries, which are executed on a SQLite database.

  • Results are displayed in table format.

• Uses NLP techniques like:

  • Intent recognition

  • Entity extraction

  • Context understanding

System Architecture:

1. Frontend (Streamlit) – User interface for input and output.

2. Application Logic – Converts input to structured prompts and handles query flow.

3. LLM Integration – Uses Gemini Pro to convert text to SQL.

4. Database Backend – Executes queries and retrieves results.

Results:

• Successfully implemented a functional home page and text-to-SQL conversion.

• Accurate SQL generation for straightforward queries.

Future Scope:

• Complex Query Support: Extend capabilities to handle joins, subqueries, and aggregation.

• Schema Auto-Extraction: Dynamically adapt to new databases.

• Query Validation: Add syntax and logic checks before execution.

• User Feedback Integration: Allow users to correct queries and provide feedback.

• Multilingual Input: Accept non-English queries.

• Result Visualization: Support charts, downloadable CSVs, and analytics dashboards.

Requirements:

Software:

• Python

• Streamlit

• Google Gemini API (GenAI)

• Visual Studio Code

Hardware:

• Minimum 2-core CPU, 4GB RAM, 128GB storage

Conclusion

1) This project successfully demonstrates the integration of a large language model (LLM) — Google’s Gemini Pro — into a natural language interface that translates user queries into executable SQL commands. 2) The key objective was to simplify data retrieval from structured databases using everyday English, making it accessible to users without technical or SQL knowledge. Using Streamlit for the front-end and SQLite as the backend database, we built a fully functional prototype that responds accurately to a wide range of natural language queries. 3) The application interface is intuitive, the processing time is minimal, and the architecture is scalable for larger and more complex datasets with minor modifications. Furthermore, the use of environment variables ensured secure API access, demonstrating good development practices in deploying LLM-based applications. 4) Overall, this project confirms that LLMs can bridge the gap between non-technical users and relational databases by removing the requirement for SQL expertise, offering a natural, conversational method of interacting with data.

References

[1] Google Generative AI (Gemini Pro) API Documentation https://ai.google.dev/docs [2] Streamlit Documentation – An open-source app framework for Machine Learning and Data Scienceprojects. https://docs.streamlit.io/ [3] SQLite Documentation – Lightweight, disk-based database used in this project https://www.sqlite.org/docs.html [4] Rajkumar, S., & Liang, P. (2022). Evaluating Large Language Models for SQL Query Generation.arXiv preprint [arXiv:2204.00498]. https://arxiv.org/abs/2204.00498 [5] Finegan-Dollak, C., et al. (2018). Improving Text-to-SQL Evaluation Methodology. Proceedings ofthe Annual Meeting of the Association for Computational Linguistics (ACL). https://aclanthology.org/P18-1033/ [6] OpenAI Cookbook – Guides and code samples for building with LLMs https://github.com/openai/openai-cookbook [7] Python Official Documentation – Programming language used to build the applicationhttps://docs.python.org/3/ [8] Vaswani, A., et al. (2017). Attention is All You Need. Advances in Neural Information ProcessingSystems. https://papers.nips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf [9] .env (dotenv) Python Library Documentation – For managing environment variables securely https://pypi.org/project/python-dotenv/ [10] Yu, T., et al. (2018). Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. https://arxiv.org/abs/1809.08887

Copyright

Copyright © 2025 Kartik Sharma, Shubham Choudhary, Vaibhav Katyan, Assistant Prof. Gosiya Kaleem. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.