LLM Powered MongoDB Query System using Generative AI

Introduction

This project aims to simplify database interaction by enabling users to query MongoDB using natural language instead of writing database queries manually. It leverages an open-source Large Language Model to understand user requirements, generate structured MongoDB queries, and retrieve the relevant data automatically. By combining natural language processing, schema-aware prompt engineering, and NoSQL database execution, the system provides an efficient and accessible approach to data retrieval while maintaining an offline processing workflow.

Technologies Used

• Python
• Machine Learning
• Generative AI
• PyTorch
• Large Language Models (LLMs)
• Hugging Face Transformers
• MongoDB
• Pandas

Installation

Clone the Repository

git clone https://github.com/archna13/LLM-Powered-MongoDB-Query-System-using-Generative-AI.git

Create a Virtual Environment

python -m venv .venv

Activate the Environment

.venv\Scripts\activate     # Windows
source .venv/bin/activate  # Linux/Mac

Install Dependencies

pip install -r requirements.txt

Run the Application

streamlit run app.py

Features

• Data Ingestion & Database Setup: The sample_data.csv file is loaded using Pandas and converted into JSON documents containing product information such as category, price, rating, stock, discount, and brand. These records are inserted into the local MongoDB ProductDB.ProductCollection using PyMongo.

• Offline LLM Loading: The CodeLlama-7B-Instruct model is loaded locally using Hugging Face Transformers and PyTorch with float16 precision for GPU memory optimization. Model weights are cached in the local directory to support offline execution.

• Natural Language Query Processing: Users provide questions in natural language through the command-line interface. A contextual prompt is constructed by combining the user's question with the valid MongoDB schema fields to guide the LLM in generating database queries.

• MongoDB Query Generation: CodeLlama processes the constructed prompt and generates a MongoDB query based on the user's requirements. Regular expression-based parsing is used to extract the query from the generated model output and remove unnecessary formatting.

• Query Sanitization & Conversion: The generated query is cleaned using regular expressions to produce valid dictionary syntax. ast.literal_eval() converts the cleaned query into a Python dictionary that can be executed through PyMongo.

• Query Execution & Data Export: The generated query is executed against MongoDB using PyMongo, and the matching documents are converted into a Pandas DataFrame. Results can be displayed in the console and exported as CSV files for further use.

• Query Logging & Validation: User prompts and generated MongoDB queries are recorded in Queries_generated.txt for auditing and tracking. Test cases validate the generated queries, with successful cases exporting filtered results to CSV files and unsupported query structures being identified through execution errors.

Conclusion

This project demonstrates an end-to-end approach for converting natural language questions into executable MongoDB queries using an Large Language Model. By combining CodeLlama, prompt engineering, query sanitization, and PyMongo, the system enables users to interact with structured database information through natural language. The tested workflows demonstrate successful query generation and execution while also highlighting areas for improving complex query handling.