Merge pull request #1608 from siiddhantt/feat/api-tool

feat: API Tool

.github/FUNDING.yml

@@ -0,0 +1,3 @@
+# These are supported funding model platforms
+
+github: arc53

.github/dependabot.yml

@@ -8,12 +8,12 @@ updates:
   - package-ecosystem: "pip" # See documentation for possible values
     directory: "/application" # Location of package manifests
     schedule:
-      interval: "weekly"
+      interval: "daily"
   - package-ecosystem: "npm" # See documentation for possible values
     directory: "/frontend" # Location of package manifests
     schedule:
-      interval: "weekly"
+      interval: "daily"
   - package-ecosystem: "github-actions"
     directory: "/"
     schedule:
-      interval: "weekly"
+      interval: "daily"

.github/workflows/pytest.yml

@@ -6,7 +6,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ["3.11"]
+        python-version: ["3.12"]
     steps:
       - uses: actions/checkout@v4
       - name: Set up Python ${{ matrix.python-version }}
@@ -23,8 +23,8 @@ jobs:
         run: |
           python -m pytest --cov=application --cov-report=xml
       - name: Upload coverage reports to Codecov
-        if: github.event_name == 'pull_request' && matrix.python-version == '3.11'
-        uses: codecov/codecov-action@v4
+        if: github.event_name == 'pull_request' && matrix.python-version == '3.12'
+        uses: codecov/codecov-action@v5
         env:
           CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
 

.vscode/launch.json

@@ -11,6 +11,44 @@
       "skipFiles": [
         "<node_internals>/**"
       ]
+    },
+    {
+        "name": "Flask Debugger",
+        "type": "debugpy",
+        "request": "launch",
+        "module": "flask",
+        "env": {
+            "FLASK_APP": "application/app.py",
+            "PYTHONPATH": "${workspaceFolder}",
+            "FLASK_ENV": "development",
+            "FLASK_DEBUG": "1",
+            "FLASK_RUN_PORT": "7091",
+            "FLASK_RUN_HOST": "0.0.0.0"
+
+        },
+        "args": [
+            "run",
+            "--no-debugger"
+        ],
+        "cwd": "${workspaceFolder}",
+    },
+    {
+      "name": "Celery Debugger",
+      "type": "debugpy",
+      "request": "launch",
+      "module": "celery",
+      "env": {
+        "PYTHONPATH": "${workspaceFolder}",
+      },
+      "args": [
+        "-A",
+        "application.app.celery",
+        "worker",
+        "-l",
+        "INFO",
+        "--pool=solo"
+      ],
+      "cwd": "${workspaceFolder}"
     }
   ]
 }

CONTRIBUTING.md

@@ -27,6 +27,7 @@ Before creating issues, please check out how the latest version of our app looks
 
 ### 👨‍💻 If you're interested in contributing code, here are some important things to know:
 
+For instructions on setting up a development environment, please refer to our [Development Deployment Guide](https://docs.docsgpt.cloud/Deploying/Development-Environment).
 
 Tech Stack Overview:
 
@@ -36,15 +37,14 @@ Tech Stack Overview:
 
 ### 🌐 If you are looking to contribute to frontend (⚛️React, Vite):
 
-- The current frontend is being migrated from [`/application`](https://github.com/arc53/DocsGPT/tree/main/application) to [`/frontend`](https://github.com/arc53/DocsGPT/tree/main/frontend) with a new design, so please contribute to the new one.
-- Check out this [milestone](https://github.com/arc53/DocsGPT/milestone/1) and its issues.
+
 - The updated Figma design can be found [here](https://www.figma.com/file/OXLtrl1EAy885to6S69554/DocsGPT?node-id=0%3A1&t=hjWVuxRg9yi5YkJ9-1).
 
 Please try to follow the guidelines.
 
 ### 🖥 If you are looking to contribute to Backend (🐍 Python):
 
-- Review our issues and contribute to [`/application`](https://github.com/arc53/DocsGPT/tree/main/application) or [`/scripts`](https://github.com/arc53/DocsGPT/tree/main/scripts) (please disregard old [`ingest_rst.py`](https://github.com/arc53/DocsGPT/blob/main/scripts/old/ingest_rst.py) [`ingest_rst_sphinx.py`](https://github.com/arc53/DocsGPT/blob/main/scripts/old/ingest_rst_sphinx.py) files; these will be deprecated soon).
+- Review our issues and contribute to [`/application`](https://github.com/arc53/DocsGPT/tree/main/application) 
 - All new code should be covered with unit tests ([pytest](https://github.com/pytest-dev/pytest)). Please find tests under [`/tests`](https://github.com/arc53/DocsGPT/tree/main/tests) folder.
 - Before submitting your Pull Request, ensure it can be queried after ingesting some test data.
   

HACKTOBERFEST.md

@@ -1,41 +0,0 @@
-# **🎉 Join the Hacktoberfest with DocsGPT and win a Free T-shirt and other prizes! 🎉**
-
-Welcome, contributors! We're excited to announce that DocsGPT is participating in Hacktoberfest. Get involved by submitting meaningful pull requests.
-
-All contributors with accepted PRs will receive a cool Holopin! 🤩 (Watch out for a reply in your PR to collect it).
-
-### 🏆 Top 50 contributors will receive a special T-shirt
-
-### 🏆 [LLM Document analysis by LexEU competition](https://github.com/arc53/DocsGPT/blob/main/lexeu-competition.md): 
-A separate competition is available for those who submit new retrieval / workflow method that will analyze a Document using EU laws.
-With 200$, 100$, 50$ prize for 1st, 2nd and 3rd place respectively.
-You can find more information [here](https://github.com/arc53/DocsGPT/blob/main/lexeu-competition.md)
-
-## 📜 Here's How to Contribute:
-```text
-🛠️ Code: This is the golden ticket! Make meaningful contributions through PRs.
-
-🧩 API extension: Build an app utilising DocsGPT API. We prefer submissions that showcase original ideas and turn the API into an AI agent.
-They can be a completely separate repos. 
-For example: 
-https://github.com/arc53/tg-bot-docsgpt-extenstion or 
-https://github.com/arc53/DocsGPT-cli
-
-Non-Code Contributions:
-
-📚 Wiki: Improve our documentation, create a guide or change existing documentation.
-
-🖥️ Design: Improve the UI/UX or design a new feature.
-
-📝 Blogging or Content Creation: Write articles or create videos to showcase DocsGPT or highlight your contributions!
-```
-
-### 📝 Guidelines for Pull Requests:
-- Familiarize yourself with the current contributions and our [Roadmap](https://github.com/orgs/arc53/projects/2).
-- Before contributing we highly advise that you check existing [issues](https://github.com/arc53/DocsGPT/issues) or [create](https://github.com/arc53/DocsGPT/issues/new/choose) an issue and wait to get assigned.
-- Once you are finished with your contribution, please fill in this [form](https://airtable.com/appikMaJwdHhC1SDP/pagoblCJ9W29wf6Hf/form).
-- Refer to the [Documentation](https://docs.docsgpt.cloud/).
-- Feel free to join our [Discord](https://discord.gg/n5BX8dh8rU) server. We're here to help newcomers, so don't hesitate to jump in! Join us [here](https://discord.gg/n5BX8dh8rU).
-  
-Thank you very much for considering contributing to DocsGPT during Hacktoberfest! 🙏 Your contributions (not just simple typos) could earn you a stylish new t-shirt and other prizes as a token of our appreciation. 🎁 Join us, and let's code together! 🚀
-

README.md

@@ -3,13 +3,11 @@
 </h1>
 
 <p align="center">
-  <strong>Open-Source Documentation Assistant</strong>
+  <strong>Open-Source RAG Assistant</strong>
 </p>
 
 <p align="left">
-  <strong><a href="https://www.docsgpt.cloud/">DocsGPT</a></strong> is a cutting-edge open-source solution that streamlines the process of finding information in the project documentation. With its integration of the powerful <strong>GPT</strong> models, developers can easily ask questions about a project and receive accurate answers.
-  
-Say goodbye to time-consuming manual searches, and let <strong><a href="https://www.docsgpt.cloud/">DocsGPT</a></strong> help you quickly find the information you need. Try it out and see how it revolutionizes your project documentation experience. Contribute to its development and be a part of the future of AI-powered assistance.
+  <strong><a href="https://www.docsgpt.cloud/">DocsGPT</a></strong> is an open-source genAI tool that helps users get reliable answers from any knowledge source, while avoiding hallucinations. It enables quick and reliable information retrieval, with tooling and agentic system capability built in.
 </p>
 
 <div align="center">
@@ -20,175 +18,122 @@ Say goodbye to time-consuming manual searches, and let <strong><a href="https://
   <a href="https://discord.gg/n5BX8dh8rU">![link to discord](https://img.shields.io/discord/1070046503302877216)</a>
   <a href="https://twitter.com/docsgptai">![X (formerly Twitter) URL](https://img.shields.io/twitter/follow/docsgptai)</a>
 
- 
+  <br>
+
+  [☁️ Cloud Version](https://app.docsgpt.cloud/) • [💬 Discord](https://discord.gg/n5BX8dh8rU) • [📖 Guides](https://docs.docsgpt.cloud/)
+  <br>
+  [👫 Contribute](https://github.com/arc53/DocsGPT/blob/main/CONTRIBUTING.md) • [🏠 Self-host](https://docs.docsgpt.cloud/Guides/How-to-use-different-LLM) • [⚡️ Quickstart](https://github.com/arc53/DocsGPT#quickstart) 
+
 </div>
+<div align="center">
+<img src="https://d3dg1063dc54p9.cloudfront.net/videos/demov7.gif" alt="video-example-of-docs-gpt" width="800" height="450">
+</div>
+<h3 align="left">
+  <strong>Key Features:</strong>
+</h3>
+<ul align="left">
+    <li><strong>🗂️ Wide Format Support:</strong> Reads PDF, DOCX, CSV, XLSX, EPUB, MD, RST, HTML, MDX, JSON, PPTX, and images.</li>
+    <li><strong>🌐 Web & Data Integration:</strong> Ingests from URLs, sitemaps, Reddit, GitHub and web crawlers.</li>
+    <li><strong>✅ Reliable Answers:</strong> Get accurate, hallucination-free responses with source citations viewable in a clean UI.</li>
+    <li><strong>🔗 Actionable Tooling:</strong> Connect to APIs, tools, and other services to enable LLM actions.</li>
+    <li><strong>🧩 Pre-built Integrations:</strong> Use readily available HTML/React chat widgets, search tools, Discord/Telegram bots, and more.</li>
+    <li><strong>🔌 Flexible Deployment:</strong> Works with major LLMs (OpenAI, Google, Anthropic) and local models (Ollama, llama_cpp).</li>
+    <li><strong>🏢 Secure & Scalable:</strong> Run privately and securely with Kubernetes support, designed for enterprise-grade reliability.</li>
+</ul>
+
+## Roadmap
+
+- [x] Full GoogleAI compatibility (Jan 2025)
+- [x] Add tools (Jan 2025)
+- [ ] Anthropic Tool compatibility
+- [ ] Add triggerable actions / tools (webhook)
+- [ ] Add OAuth 2.0 authentication for tools and sources
+- [ ] Manually updating chunks in the app UI
+- [ ] Devcontainer for easy development
+- [ ] Chatbots menu re-design to handle tools, scheduling, and more
+
+You can find our full roadmap [here](https://github.com/orgs/arc53/projects/2). Please don't hesitate to contribute or create issues, it helps us improve DocsGPT!
 
 ### Production Support / Help for Companies:
 
 We're eager to provide personalized assistance when deploying your DocsGPT to a live environment.
 
-[Book a Meeting :wave:](https://cal.com/arc53/docsgpt-demo-b2b)⁠
+[Get a Demo :wave:](https://www.docsgpt.cloud/contact)⁠
 
-[Send Email :email:](mailto:contact@arc53.com?subject=DocsGPT%20support%2Fsolutions)
+[Send Email :email:](mailto:support@docsgpt.cloud?subject=DocsGPT%20support%2Fsolutions)
 
 
-<img src="https://github.com/user-attachments/assets/9a1f21de-7a15-4e42-9424-70d22ba5a913" alt="video-example-of-docs-gpt" width="1000" height="500">
-
-## Roadmap
-
-You can find our roadmap [here](https://github.com/orgs/arc53/projects/2). Please don't hesitate to contribute or create issues, it helps us improve DocsGPT!
-
-## Our Open-Source Models Optimized for DocsGPT:
-
-| Name                                                                  | Base Model  | Requirements (or similar) |
-| --------------------------------------------------------------------- | ----------- | ------------------------- |
-| [Docsgpt-7b-mistral](https://huggingface.co/Arc53/docsgpt-7b-mistral)   | Mistral-7b   | 1xA10G gpu                |
-| [Docsgpt-14b](https://huggingface.co/Arc53/docsgpt-14b)               | llama-2-14b | 2xA10 gpu's               |
-| [Docsgpt-40b-falcon](https://huggingface.co/Arc53/docsgpt-40b-falcon) | falcon-40b  | 8xA10G gpu's              |
-
-If you don't have enough resources to run it, you can use bitsnbytes to quantize.
-
-## End to End AI Framework for Information Retrieval
-
-![Architecture chart](https://github.com/user-attachments/assets/fc6a7841-ddfc-45e6-b5a0-d05fe648cbe2)
-
-## Useful Links
-
-- :mag: :fire: [Cloud Version](https://app.docsgpt.cloud/)
-
-- :speech_balloon: :tada: [Join our Discord](https://discord.gg/n5BX8dh8rU)
-
-- :books: :sunglasses: [Guides](https://docs.docsgpt.cloud/)
-
-- :couple: [Interested in contributing?](https://github.com/arc53/DocsGPT/blob/main/CONTRIBUTING.md)
-
-- :file_folder: :rocket: [How to use any other documentation](https://docs.docsgpt.cloud/Guides/How-to-train-on-other-documentation)
-
-- :house: :closed_lock_with_key: [How to host it locally (so all data will stay on-premises)](https://docs.docsgpt.cloud/Guides/How-to-use-different-LLM)
-
-## Project Structure
-
-- Application - Flask app (main application).
-
-- Extensions - Chrome extension.
-
-- Scripts - Script that creates similarity search index for other libraries.
-
-- Frontend - Frontend uses <a href="https://vitejs.dev/">Vite</a> and <a href="https://react.dev/">React</a>.
-
 ## QuickStart
 
 > [!Note]
 > Make sure you have [Docker](https://docs.docker.com/engine/install/) installed
 
+
+1. Clone the repository and run the following command:
+  ```bash
+  git clone https://github.com/arc53/DocsGPT.git
+  cd DocsGPT
+  ```
+
 On Mac OS or Linux, write:
 
-`./setup.sh`
+
+2. Run the following command:
+  ```bash
+  ./setup.sh
+  ```
 
 It will install all the dependencies and allow you to download the local model, use OpenAI or use our LLM API.
 
 Otherwise, refer to this Guide for Windows:
 
-1. Download and open this repository with `git clone https://github.com/arc53/DocsGPT.git`
-2. Create a `.env` file in your root directory and set the env variables and `VITE_API_STREAMING` to true or false, depending on whether you want streaming answers or not.
+On windows:
+
+2. Create a `.env` file in your root directory and set the env variables.
    It should look like this inside:
 
    ```
    LLM_NAME=[docsgpt or openai or others] 
-   VITE_API_STREAMING=true
    API_KEY=[if LLM_NAME is openai]
    ```
 
-   See optional environment variables in the [/.env-template](https://github.com/arc53/DocsGPT/blob/main/.env-template) and [/application/.env_sample](https://github.com/arc53/DocsGPT/blob/main/application/.env_sample) files.
+   See optional environment variables in the [/application/.env_sample](https://github.com/arc53/DocsGPT/blob/main/application/.env_sample) file.
 
-3. Run [./run-with-docker-compose.sh](https://github.com/arc53/DocsGPT/blob/main/run-with-docker-compose.sh).
+3. Run the following command:
+
+   ```bash
+   docker compose up --build
+   ```
 4. Navigate to http://localhost:5173/.
 
 To stop, just run `Ctrl + C`.
 
-## Development Environments
-
-### Spin up Mongo and Redis
-
-For development, only two containers are used from [docker-compose.yaml](https://github.com/arc53/DocsGPT/blob/main/docker-compose.yaml) (by deleting all services except for Redis and Mongo).
-See file [docker-compose-dev.yaml](./docker-compose-dev.yaml).
-
-Run
-
-```
-docker compose -f docker-compose-dev.yaml build
-docker compose -f docker-compose-dev.yaml up -d
-```
-
-### Run the Backend
-
 > [!Note]
-> Make sure you have Python 3.10 or 3.11 installed.
-
-1. Export required environment variables or prepare a `.env` file in the project folder:
-   - Copy [.env-template](https://github.com/arc53/DocsGPT/blob/main/application/.env-template) and create `.env`.
-
-(check out [`application/core/settings.py`](application/core/settings.py) if you want to see more config options.)
-
-2. (optional) Create a Python virtual environment:
-   You can follow the [Python official documentation](https://docs.python.org/3/tutorial/venv.html) for virtual environments.
-
-a) On Mac OS and Linux
-
-```commandline
-python -m venv venv
-. venv/bin/activate
-```
-
-b) On Windows
-
-```commandline
-python -m venv venv
- venv/Scripts/activate
-```
-
-3. Download embedding model and save it in the `model/` folder:
-You can use the script below, or download it manually from [here](https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip), unzip it and save it in the `model/` folder.
-
-```commandline
-wget https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip
-unzip mpnet-base-v2.zip -d model
-rm mpnet-base-v2.zip
-```
-
-4. Install dependencies for the backend:
-
-```commandline
-pip install -r application/requirements.txt
-```
-
-5. Run the app using `flask --app application/app.py run --host=0.0.0.0 --port=7091`.
-6. Start worker with `celery -A application.app.celery worker -l INFO`.
-
-### Start Frontend
-
-> [!Note]
-> Make sure you have Node version 16 or higher.
-
-1. Navigate to the [/frontend](https://github.com/arc53/DocsGPT/tree/main/frontend) folder.
-2. Install the required packages `husky` and `vite` (ignore if already installed).
-
-```commandline
-npm install husky -g
-npm install vite -g
-```
-
-3. Install dependencies by running `npm install --include=dev`.
-4. Run the app using `npm run dev`.
+> For development environment setup instructions, please refer to the [Development Environment Guide](https://docs.docsgpt.cloud/Deploying/Development-Environment).
 
 ## Contributing
 
 Please refer to the [CONTRIBUTING.md](CONTRIBUTING.md) file for information about how to get involved. We welcome issues, questions, and pull requests.
 
+## Architecture
+
+![Architecture chart](https://github.com/user-attachments/assets/fc6a7841-ddfc-45e6-b5a0-d05fe648cbe2)
+
+## Project Structure
+
+- Application - Flask app (main application).
+
+- Extensions - Extensions, like react widget or discord bot.
+
+- Frontend - Frontend uses <a href="https://vitejs.dev/">Vite</a> and <a href="https://react.dev/">React</a>.
+
+- Scripts - Miscellaneous scripts.
+
 ## Code Of Conduct
 
 We as members, contributors, and leaders, pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation. Please refer to the [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) file for more information about contributing.
 
+
 ## Many Thanks To Our Contributors⚡
 
 <a href="https://github.com/arc53/DocsGPT/graphs/contributors" alt="View Contributors">

application/Dockerfile

@@ -8,14 +8,14 @@ RUN apt-get update && \
     add-apt-repository ppa:deadsnakes/ppa && \
 # Install necessary packages and Python
     apt-get update && \
-    apt-get install -y --no-install-recommends gcc wget unzip libc6-dev python3.11 python3.11-distutils python3.11-venv && \
+    apt-get install -y --no-install-recommends gcc wget unzip libc6-dev python3.12 python3.12-venv && \
     rm -rf /var/lib/apt/lists/* 
 
 # Verify Python installation and setup symlink
-RUN if [ -f /usr/bin/python3.11 ]; then \
-        ln -s /usr/bin/python3.11 /usr/bin/python; \
+RUN if [ -f /usr/bin/python3.12 ]; then \
+        ln -s /usr/bin/python3.12 /usr/bin/python; \
     else \
-        echo "Python 3.11 not found"; exit 1; \
+        echo "Python 3.12 not found"; exit 1; \
     fi
 
 # Download and unzip the model
@@ -33,7 +33,7 @@ RUN apt-get remove --purge -y wget unzip && apt-get autoremove -y && rm -rf /var
 COPY requirements.txt .
 
 # Setup Python virtual environment
-RUN python3.11 -m venv /venv
+RUN python3.12 -m venv /venv
 
 # Activate virtual environment and install Python packages
 ENV PATH="/venv/bin:$PATH"
@@ -50,8 +50,8 @@ RUN apt-get update && \
     apt-get install -y software-properties-common && \
     add-apt-repository ppa:deadsnakes/ppa && \
 # Install Python
-    apt-get update && apt-get install -y --no-install-recommends python3.11 && \
-    ln -s /usr/bin/python3.11 /usr/bin/python && \
+    apt-get update && apt-get install -y --no-install-recommends python3.12 && \
+    ln -s /usr/bin/python3.12 /usr/bin/python && \
     rm -rf /var/lib/apt/lists/*
 
 # Set working directory

application/api/answer/routes.py

@@ -18,7 +18,7 @@ from application.error import bad_request
 from application.extensions import api
 from application.llm.llm_creator import LLMCreator
 from application.retriever.retriever_creator import RetrieverCreator
-from application.utils import check_required_fields
+from application.utils import check_required_fields, limit_chat_history
 
 logger = logging.getLogger(__name__)
 
@@ -37,7 +37,7 @@ api.add_namespace(answer_ns)
 gpt_model = ""
 # to have some kind of default behaviour
 if settings.LLM_NAME == "openai":
-    gpt_model = "gpt-3.5-turbo"
+    gpt_model = "gpt-4o-mini"
 elif settings.LLM_NAME == "anthropic":
     gpt_model = "claude-2"
 elif settings.LLM_NAME == "groq":
@@ -118,8 +118,31 @@ def is_azure_configured():
     )
 
 
-def save_conversation(conversation_id, question, response, source_log_docs, llm):
-    if conversation_id is not None and conversation_id != "None":
+def save_conversation(conversation_id, question, response, source_log_docs, llm,index=None):
+    if conversation_id is not None and index is not None:
+        conversations_collection.update_one(
+            {"_id": ObjectId(conversation_id), f"queries.{index}": {"$exists": True}},
+            {
+                "$set": {
+                    f"queries.{index}.prompt": question,
+                    f"queries.{index}.response": response,
+                    f"queries.{index}.sources": source_log_docs,
+                }
+            }
+        )
+        ##remove following queries from the array
+        conversations_collection.update_one(
+            {"_id": ObjectId(conversation_id), f"queries.{index}": {"$exists": True}},
+            {
+                "$push":{
+                    "queries":{
+                        "$each":[],
+                        "$slice":index+1
+                    }
+                }
+            }
+        )
+    elif conversation_id is not None and conversation_id != "None":
         conversations_collection.update_one(
             {"_id": ObjectId(conversation_id)},
             {
@@ -186,7 +209,7 @@ def get_prompt(prompt_id):
 
 
 def complete_stream(
-    question, retriever, conversation_id, user_api_key, isNoneDoc=False
+    question, retriever, conversation_id, user_api_key, isNoneDoc=False,index=None
 ):
 
     try:
@@ -217,7 +240,7 @@ def complete_stream(
         )
         if user_api_key is None:
             conversation_id = save_conversation(
-                conversation_id, question, response_full, source_log_docs, llm
+                conversation_id, question, response_full, source_log_docs, llm,index
             )
             # send data.type = "end" to indicate that the stream has ended as json
             data = json.dumps({"type": "id", "id": str(conversation_id)})
@@ -282,6 +305,9 @@ class Stream(Resource):
             "isNoneDoc": fields.Boolean(
                 required=False, description="Flag indicating if no document is used"
             ),
+            "index":fields.Integer(
+                required=False, description="The position where query is to be updated"
+            ),
         },
     )
 
@@ -290,23 +316,23 @@ class Stream(Resource):
     def post(self):
         data = request.get_json()
         required_fields = ["question"]
-
+        if "index" in data:
+            required_fields = ["question","conversation_id"]
         missing_fields = check_required_fields(data, required_fields)
         if missing_fields:
             return missing_fields
 
         try:
             question = data["question"]
-            history = data.get("history", [])
-            history = json.loads(history)
+            history = limit_chat_history(json.loads(data.get("history", [])), gpt_model=gpt_model)
             conversation_id = data.get("conversation_id")
             prompt_id = data.get("prompt_id", "default")
             
-
+            index=data.get("index",None)
             chunks = int(data.get("chunks", 2))
             token_limit = data.get("token_limit", settings.DEFAULT_MAX_HISTORY)
             retriever_name = data.get("retriever", "classic")
-
+            
             if "api_key" in data:
                 data_key = get_data_from_api_key(data["api_key"])
                 chunks = int(data_key.get("chunks", 2))
@@ -343,7 +369,7 @@ class Stream(Resource):
                 gpt_model=gpt_model,
                 user_api_key=user_api_key,
             )
-
+            
             return Response(
                 complete_stream(
                     question=question,
@@ -351,6 +377,7 @@ class Stream(Resource):
                     conversation_id=conversation_id,
                     user_api_key=user_api_key,
                     isNoneDoc=data.get("isNoneDoc"),
+                    index=index,
                 ),
                 mimetype="text/event-stream",
             )
@@ -428,7 +455,7 @@ class Answer(Resource):
 
         try:
             question = data["question"]
-            history = data.get("history", [])
+            history = limit_chat_history(json.loads(data.get("history", [])), gpt_model=gpt_model)
             conversation_id = data.get("conversation_id")
             prompt_id = data.get("prompt_id", "default")
             chunks = int(data.get("chunks", 2))

application/api/user/routes.py

@@ -1,14 +1,14 @@
 import datetime
+import math
 import os
 import shutil
 import uuid
-import math
 
 from bson.binary import Binary, UuidRepresentation
 from bson.dbref import DBRef
 from bson.objectid import ObjectId
-from flask import Blueprint, jsonify, make_response, request, redirect
-from flask_restx import inputs, fields, Namespace, Resource
+from flask import Blueprint, jsonify, make_response, redirect, request
+from flask_restx import fields, inputs, Namespace, Resource
 from werkzeug.utils import secure_filename
 
 from application.api.user.tasks import ingest, ingest_remote
@@ -16,9 +16,10 @@ from application.api.user.tasks import ingest, ingest_remote
 from application.core.mongo_db import MongoDB
 from application.core.settings import settings
 from application.extensions import api
+from application.tools.tool_manager import ToolManager
+from application.tts.google_tts import GoogleTTS
 from application.utils import check_required_fields
 from application.vectorstore.vector_creator import VectorCreator
-from application.tts.google_tts import GoogleTTS
 
 mongo = MongoDB.get_client()
 db = mongo["docsgpt"]
@@ -30,6 +31,7 @@ api_key_collection = db["api_keys"]
 token_usage_collection = db["token_usage"]
 shared_conversations_collections = db["shared_conversations"]
 user_logs_collection = db["user_logs"]
+user_tools_collection = db["user_tools"]
 
 user = Blueprint("user", __name__)
 user_ns = Namespace("user", description="User related operations", path="/")
@@ -39,6 +41,9 @@ current_dir = os.path.dirname(
     os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 )
 
+tool_config = {}
+tool_manager = ToolManager(config=tool_config)
+
 
 def generate_minute_range(start_date, end_date):
     return {
@@ -176,10 +181,17 @@ class SubmitFeedback(Resource):
             "FeedbackModel",
             {
                 "question": fields.String(
-                    required=True, description="The user question"
+                    required=False, description="The user question"
                 ),
-                "answer": fields.String(required=True, description="The AI answer"),
+                "answer": fields.String(required=False, description="The AI answer"),
                 "feedback": fields.String(required=True, description="User feedback"),
+                "question_index": fields.Integer(
+                    required=True,
+                    description="The question number in that particular conversation",
+                ),
+                "conversation_id": fields.String(
+                    required=True, description="id of the particular conversation"
+                ),
                 "api_key": fields.String(description="Optional API key"),
             },
         )
@@ -189,23 +201,24 @@ class SubmitFeedback(Resource):
     )
     def post(self):
         data = request.get_json()
-        required_fields = ["question", "answer", "feedback"]
+        required_fields = ["feedback", "conversation_id", "question_index"]
         missing_fields = check_required_fields(data, required_fields)
         if missing_fields:
             return missing_fields
 
-        new_doc = {
-            "question": data["question"],
-            "answer": data["answer"],
-            "feedback": data["feedback"],
-            "timestamp": datetime.datetime.now(datetime.timezone.utc),
-        }
-
-        if "api_key" in data:
-            new_doc["api_key"] = data["api_key"]
-
         try:
-            feedback_collection.insert_one(new_doc)
+            conversations_collection.update_one(
+                {
+                    "_id": ObjectId(data["conversation_id"]),
+                    f"queries.{data['question_index']}": {"$exists": True},
+                },
+                {
+                    "$set": {
+                        f"queries.{data['question_index']}.feedback": data["feedback"]
+                    }
+                },
+            )
+
         except Exception as err:
             return make_response(jsonify({"success": False, "error": str(err)}), 400)
 
@@ -248,13 +261,10 @@ class DeleteOldIndexes(Resource):
                 jsonify({"success": False, "message": "Missing required fields"}), 400
             )
 
+        doc = sources_collection.find_one({"_id": ObjectId(source_id), "user": "local"})
+        if not doc:
+            return make_response(jsonify({"status": "not found"}), 404)
         try:
-            doc = sources_collection.find_one(
-                {"_id": ObjectId(source_id), "user": "local"}
-            )
-            if not doc:
-                return make_response(jsonify({"status": "not found"}), 404)
-
             if settings.VECTOR_STORE == "faiss":
                 shutil.rmtree(os.path.join(current_dir, "indexes", str(doc["_id"])))
             else:
@@ -263,12 +273,12 @@ class DeleteOldIndexes(Resource):
                 )
                 vectorstore.delete_index()
 
-            sources_collection.delete_one({"_id": ObjectId(source_id)})
         except FileNotFoundError:
             pass
         except Exception as err:
             return make_response(jsonify({"success": False, "error": str(err)}), 400)
 
+        sources_collection.delete_one({"_id": ObjectId(source_id)})
         return make_response(jsonify({"success": True}), 200)
 
 
@@ -339,6 +349,9 @@ class UploadFile(Resource):
                         ".json",
                         ".xlsx",
                         ".pptx",
+                        ".png",
+                        ".jpg",
+                        ".jpeg",
                     ],
                     job_name,
                     final_filename,
@@ -365,6 +378,9 @@ class UploadFile(Resource):
                         ".json",
                         ".xlsx",
                         ".pptx",
+                        ".png",
+                        ".jpg",
+                        ".jpeg",
                     ],
                     job_name,
                     final_filename,
@@ -473,11 +489,24 @@ class PaginatedSources(Resource):
         sort_order = request.args.get("order", "desc")  # Default to 'desc'
         page = int(request.args.get("page", 1))  # Default to 1
         rows_per_page = int(request.args.get("rows", 10))  # Default to 10
+        # add .strip() to remove leading and trailing whitespaces
+        search_term = request.args.get(
+            "search", ""
+        ).strip()  # add search for filter documents
 
-        # Prepare
+        # Prepare query for filtering
         query = {"user": user}
+        if search_term:
+            query["name"] = {
+                "$regex": search_term,
+                "$options": "i",  # using case-insensitive search
+            }
+
         total_documents = sources_collection.count_documents(query)
         total_pages = max(1, math.ceil(total_documents / rows_per_page))
+        page = min(
+            max(1, page), total_pages
+        )  # add this to make sure page inbound is within the range
         sort_order = 1 if sort_order == "asc" else -1
         skip = (page - 1) * rows_per_page
 
@@ -522,7 +551,7 @@ class CombinedJson(Resource):
         user = "local"
         data = [
             {
-                "name": "default",
+                "name": "Default",
                 "date": "default",
                 "model": settings.EMBEDDINGS_NAME,
                 "location": "remote",
@@ -1449,90 +1478,17 @@ class GetFeedbackAnalytics(Resource):
             )
         except Exception as err:
             return make_response(jsonify({"success": False, "error": str(err)}), 400)
+        
         end_date = datetime.datetime.now(datetime.timezone.utc)
 
         if filter_option == "last_hour":
             start_date = end_date - datetime.timedelta(hours=1)
             group_format = "%Y-%m-%d %H:%M:00"
-            group_stage_1 = {
-                "$group": {
-                    "_id": {
-                        "minute": {
-                            "$dateToString": {
-                                "format": group_format,
-                                "date": "$timestamp",
-                            }
-                        },
-                        "feedback": "$feedback",
-                    },
-                    "count": {"$sum": 1},
-                }
-            }
-            group_stage_2 = {
-                "$group": {
-                    "_id": "$_id.minute",
-                    "likes": {
-                        "$sum": {
-                            "$cond": [
-                                {"$eq": ["$_id.feedback", "LIKE"]},
-                                "$count",
-                                0,
-                            ]
-                        }
-                    },
-                    "dislikes": {
-                        "$sum": {
-                            "$cond": [
-                                {"$eq": ["$_id.feedback", "DISLIKE"]},
-                                "$count",
-                                0,
-                            ]
-                        }
-                    },
-                }
-            }
-
+            date_field = {"$dateToString": {"format": group_format, "date": "$date"}}
         elif filter_option == "last_24_hour":
             start_date = end_date - datetime.timedelta(hours=24)
             group_format = "%Y-%m-%d %H:00"
-            group_stage_1 = {
-                "$group": {
-                    "_id": {
-                        "hour": {
-                            "$dateToString": {
-                                "format": group_format,
-                                "date": "$timestamp",
-                            }
-                        },
-                        "feedback": "$feedback",
-                    },
-                    "count": {"$sum": 1},
-                }
-            }
-            group_stage_2 = {
-                "$group": {
-                    "_id": "$_id.hour",
-                    "likes": {
-                        "$sum": {
-                            "$cond": [
-                                {"$eq": ["$_id.feedback", "LIKE"]},
-                                "$count",
-                                0,
-                            ]
-                        }
-                    },
-                    "dislikes": {
-                        "$sum": {
-                            "$cond": [
-                                {"$eq": ["$_id.feedback", "DISLIKE"]},
-                                "$count",
-                                0,
-                            ]
-                        }
-                    },
-                }
-            }
-
+            date_field = {"$dateToString": {"format": group_format, "date": "$date"}}
         else:
             if filter_option in ["last_7_days", "last_15_days", "last_30_days"]:
                 filter_days = (
@@ -1550,61 +1506,59 @@ class GetFeedbackAnalytics(Resource):
                 hour=23, minute=59, second=59, microsecond=999999
             )
             group_format = "%Y-%m-%d"
-            group_stage_1 = {
-                "$group": {
-                    "_id": {
-                        "day": {
-                            "$dateToString": {
-                                "format": group_format,
-                                "date": "$timestamp",
-                            }
-                        },
-                        "feedback": "$feedback",
-                    },
-                    "count": {"$sum": 1},
-                }
-            }
-            group_stage_2 = {
-                "$group": {
-                    "_id": "$_id.day",
-                    "likes": {
-                        "$sum": {
-                            "$cond": [
-                                {"$eq": ["$_id.feedback", "LIKE"]},
-                                "$count",
-                                0,
-                            ]
-                        }
-                    },
-                    "dislikes": {
-                        "$sum": {
-                            "$cond": [
-                                {"$eq": ["$_id.feedback", "DISLIKE"]},
-                                "$count",
-                                0,
-                            ]
-                        }
-                    },
-                }
-            }
+            date_field = {"$dateToString": {"format": group_format, "date": "$date"}}
 
         try:
             match_stage = {
                 "$match": {
-                    "timestamp": {"$gte": start_date, "$lte": end_date},
+                    "date": {"$gte": start_date, "$lte": end_date},
+                    "queries": {"$exists": True, "$ne": []},
                 }
             }
             if api_key:
                 match_stage["$match"]["api_key"] = api_key
 
-            feedback_data = feedback_collection.aggregate(
-                [
-                    match_stage,
-                    group_stage_1,
-                    group_stage_2,
-                    {"$sort": {"_id": 1}},
-                ]
-            )
+            # Unwind the queries array to process each query separately
+            pipeline = [
+                match_stage,
+                {"$unwind": "$queries"},
+                {"$match": {"queries.feedback": {"$exists": True}}},
+                {
+                    "$group": {
+                        "_id": {
+                            "time": date_field,
+                            "feedback": "$queries.feedback"
+                        },
+                        "count": {"$sum": 1}
+                    }
+                },
+                {
+                    "$group": {
+                        "_id": "$_id.time",
+                        "positive": {
+                            "$sum": {
+                                "$cond": [
+                                    {"$eq": ["$_id.feedback", "LIKE"]},
+                                    "$count",
+                                    0
+                                ]
+                            }
+                        },
+                        "negative": {
+                            "$sum": {
+                                "$cond": [
+                                    {"$eq": ["$_id.feedback", "DISLIKE"]},
+                                    "$count",
+                                    0
+                                ]
+                            }
+                        }
+                    }
+                },
+                {"$sort": {"_id": 1}}
+            ]
+
+            feedback_data = conversations_collection.aggregate(pipeline)
 
             if filter_option == "last_hour":
                 intervals = generate_minute_range(start_date, end_date)
@@ -1619,8 +1573,8 @@ class GetFeedbackAnalytics(Resource):
 
             for entry in feedback_data:
                 daily_feedback[entry["_id"]] = {
-                    "positive": entry["likes"],
-                    "negative": entry["dislikes"],
+                    "positive": entry["positive"],
+                    "negative": entry["negative"]
                 }
 
         except Exception as err:
@@ -1786,3 +1740,294 @@ class TextToSpeech(Resource):
             )
         except Exception as err:
             return make_response(jsonify({"success": False, "error": str(err)}), 400)
+
+
+@user_ns.route("/api/available_tools")
+class AvailableTools(Resource):
+    @api.doc(description="Get available tools for a user")
+    def get(self):
+        try:
+            tools_metadata = []
+            for tool_name, tool_instance in tool_manager.tools.items():
+                doc = tool_instance.__doc__.strip()
+                lines = doc.split("\n", 1)
+                name = lines[0].strip()
+                description = lines[1].strip() if len(lines) > 1 else ""
+                tools_metadata.append(
+                    {
+                        "name": tool_name,
+                        "displayName": name,
+                        "description": description,
+                        "configRequirements": tool_instance.get_config_requirements(),
+                        "actions": tool_instance.get_actions_metadata(),
+                    }
+                )
+        except Exception as err:
+            return make_response(jsonify({"success": False, "error": str(err)}), 400)
+
+        return make_response(jsonify({"success": True, "data": tools_metadata}), 200)
+
+
+@user_ns.route("/api/get_tools")
+class GetTools(Resource):
+    @api.doc(description="Get tools created by a user")
+    def get(self):
+        try:
+            user = "local"
+            tools = user_tools_collection.find({"user": user})
+            user_tools = []
+            for tool in tools:
+                tool["id"] = str(tool["_id"])
+                tool.pop("_id")
+                user_tools.append(tool)
+        except Exception as err:
+            return make_response(jsonify({"success": False, "error": str(err)}), 400)
+
+        return make_response(jsonify({"success": True, "tools": user_tools}), 200)
+
+
+@user_ns.route("/api/create_tool")
+class CreateTool(Resource):
+    @api.expect(
+        api.model(
+            "CreateToolModel",
+            {
+                "name": fields.String(required=True, description="Name of the tool"),
+                "displayName": fields.String(
+                    required=True, description="Display name for the tool"
+                ),
+                "description": fields.String(
+                    required=True, description="Tool description"
+                ),
+                "config": fields.Raw(
+                    required=True, description="Configuration of the tool"
+                ),
+                "actions": fields.List(
+                    fields.Raw,
+                    required=True,
+                    description="Actions the tool can perform",
+                ),
+                "status": fields.Boolean(
+                    required=True, description="Status of the tool"
+                ),
+            },
+        )
+    )
+    @api.doc(description="Create a new tool")
+    def post(self):
+        data = request.get_json()
+        required_fields = [
+            "name",
+            "displayName",
+            "description",
+            "actions",
+            "config",
+            "status",
+        ]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+
+        user = "local"
+        transformed_actions = []
+        for action in data["actions"]:
+            action["active"] = True
+            if "parameters" in action:
+                if "properties" in action["parameters"]:
+                    for param_name, param_details in action["parameters"][
+                        "properties"
+                    ].items():
+                        param_details["filled_by_llm"] = True
+                        param_details["value"] = ""
+            transformed_actions.append(action)
+        try:
+            new_tool = {
+                "user": user,
+                "name": data["name"],
+                "displayName": data["displayName"],
+                "description": data["description"],
+                "actions": transformed_actions,
+                "config": data["config"],
+                "status": data["status"],
+            }
+            resp = user_tools_collection.insert_one(new_tool)
+            new_id = str(resp.inserted_id)
+        except Exception as err:
+            return make_response(jsonify({"success": False, "error": str(err)}), 400)
+
+        return make_response(jsonify({"id": new_id}), 200)
+
+
+@user_ns.route("/api/update_tool")
+class UpdateTool(Resource):
+    @api.expect(
+        api.model(
+            "UpdateToolModel",
+            {
+                "id": fields.String(required=True, description="Tool ID"),
+                "name": fields.String(description="Name of the tool"),
+                "displayName": fields.String(description="Display name for the tool"),
+                "description": fields.String(description="Tool description"),
+                "config": fields.Raw(description="Configuration of the tool"),
+                "actions": fields.List(
+                    fields.Raw, description="Actions the tool can perform"
+                ),
+                "status": fields.Boolean(description="Status of the tool"),
+            },
+        )
+    )
+    @api.doc(description="Update a tool by ID")
+    def post(self):
+        data = request.get_json()
+        required_fields = ["id"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+
+        try:
+            update_data = {}
+            if "name" in data:
+                update_data["name"] = data["name"]
+            if "displayName" in data:
+                update_data["displayName"] = data["displayName"]
+            if "description" in data:
+                update_data["description"] = data["description"]
+            if "actions" in data:
+                update_data["actions"] = data["actions"]
+            if "config" in data:
+                update_data["config"] = data["config"]
+            if "status" in data:
+                update_data["status"] = data["status"]
+
+            user_tools_collection.update_one(
+                {"_id": ObjectId(data["id"]), "user": "local"},
+                {"$set": update_data},
+            )
+        except Exception as err:
+            return make_response(jsonify({"success": False, "error": str(err)}), 400)
+
+        return make_response(jsonify({"success": True}), 200)
+
+
+@user_ns.route("/api/update_tool_config")
+class UpdateToolConfig(Resource):
+    @api.expect(
+        api.model(
+            "UpdateToolConfigModel",
+            {
+                "id": fields.String(required=True, description="Tool ID"),
+                "config": fields.Raw(
+                    required=True, description="Configuration of the tool"
+                ),
+            },
+        )
+    )
+    @api.doc(description="Update the configuration of a tool")
+    def post(self):
+        data = request.get_json()
+        required_fields = ["id", "config"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+
+        try:
+            user_tools_collection.update_one(
+                {"_id": ObjectId(data["id"])},
+                {"$set": {"config": data["config"]}},
+            )
+        except Exception as err:
+            return make_response(jsonify({"success": False, "error": str(err)}), 400)
+
+        return make_response(jsonify({"success": True}), 200)
+
+
+@user_ns.route("/api/update_tool_actions")
+class UpdateToolActions(Resource):
+    @api.expect(
+        api.model(
+            "UpdateToolActionsModel",
+            {
+                "id": fields.String(required=True, description="Tool ID"),
+                "actions": fields.List(
+                    fields.Raw,
+                    required=True,
+                    description="Actions the tool can perform",
+                ),
+            },
+        )
+    )
+    @api.doc(description="Update the actions of a tool")
+    def post(self):
+        data = request.get_json()
+        required_fields = ["id", "actions"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+
+        try:
+            user_tools_collection.update_one(
+                {"_id": ObjectId(data["id"])},
+                {"$set": {"actions": data["actions"]}},
+            )
+        except Exception as err:
+            return make_response(jsonify({"success": False, "error": str(err)}), 400)
+
+        return make_response(jsonify({"success": True}), 200)
+
+
+@user_ns.route("/api/update_tool_status")
+class UpdateToolStatus(Resource):
+    @api.expect(
+        api.model(
+            "UpdateToolStatusModel",
+            {
+                "id": fields.String(required=True, description="Tool ID"),
+                "status": fields.Boolean(
+                    required=True, description="Status of the tool"
+                ),
+            },
+        )
+    )
+    @api.doc(description="Update the status of a tool")
+    def post(self):
+        data = request.get_json()
+        required_fields = ["id", "status"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+
+        try:
+            user_tools_collection.update_one(
+                {"_id": ObjectId(data["id"])},
+                {"$set": {"status": data["status"]}},
+            )
+        except Exception as err:
+            return make_response(jsonify({"success": False, "error": str(err)}), 400)
+
+        return make_response(jsonify({"success": True}), 200)
+
+
+@user_ns.route("/api/delete_tool")
+class DeleteTool(Resource):
+    @api.expect(
+        api.model(
+            "DeleteToolModel",
+            {"id": fields.String(required=True, description="Tool ID")},
+        )
+    )
+    @api.doc(description="Delete a tool by ID")
+    def post(self):
+        data = request.get_json()
+        required_fields = ["id"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+
+        try:
+            result = user_tools_collection.delete_one({"_id": ObjectId(data["id"])})
+            if result.deleted_count == 0:
+                return {"success": False, "message": "Tool not found"}, 404
+        except Exception as err:
+            return {"success": False, "error": str(err)}, 400
+
+        return {"success": True}, 200

application/cache.py

@@ -1,8 +1,10 @@
-import redis
-import time
 import json
 import logging
+import time
 from threading import Lock
+
+import redis
+
 from application.core.settings import settings
 from application.utils import get_hash
 
@@ -11,41 +13,47 @@ logger = logging.getLogger(__name__)
 _redis_instance = None
 _instance_lock = Lock()
 
+
 def get_redis_instance():
     global _redis_instance
     if _redis_instance is None:
         with _instance_lock:
             if _redis_instance is None:
                 try:
-                    _redis_instance = redis.Redis.from_url(settings.CACHE_REDIS_URL, socket_connect_timeout=2)
+                    _redis_instance = redis.Redis.from_url(
+                        settings.CACHE_REDIS_URL, socket_connect_timeout=2
+                    )
                 except redis.ConnectionError as e:
                     logger.error(f"Redis connection error: {e}")
                     _redis_instance = None
     return _redis_instance
 
-def gen_cache_key(*messages, model="docgpt"):
+
+def gen_cache_key(messages, model="docgpt", tools=None):
     if not all(isinstance(msg, dict) for msg in messages):
         raise ValueError("All messages must be dictionaries.")
-    messages_str = json.dumps(list(messages), sort_keys=True)
-    combined = f"{model}_{messages_str}"
+    messages_str = json.dumps(messages)
+    tools_str = json.dumps(str(tools)) if tools else ""
+    combined = f"{model}_{messages_str}_{tools_str}"
     cache_key = get_hash(combined)
     return cache_key
 
+
 def gen_cache(func):
-    def wrapper(self, model, messages, *args, **kwargs):
+    def wrapper(self, model, messages, stream, tools=None, *args, **kwargs):
         try:
-            cache_key = gen_cache_key(*messages)
+            cache_key = gen_cache_key(messages, model, tools)
             redis_client = get_redis_instance()
             if redis_client:
                 try:
                     cached_response = redis_client.get(cache_key)
                     if cached_response:
-                        return cached_response.decode('utf-8')
+                        return cached_response.decode("utf-8")
                 except redis.ConnectionError as e:
                     logger.error(f"Redis connection error: {e}")
 
-            result = func(self, model, messages, *args, **kwargs)
-            if redis_client:
+            result = func(self, model, messages, stream, tools, *args, **kwargs)
+            if redis_client and isinstance(result, str):
                 try:
                     redis_client.set(cache_key, result, ex=1800)
                 except redis.ConnectionError as e:
@@ -55,20 +63,22 @@ def gen_cache(func):
         except ValueError as e:
             logger.error(e)
             return "Error: No user message found in the conversation to generate a cache key."
+
     return wrapper
 
+
 def stream_cache(func):
-    def wrapper(self, model, messages, stream, *args, **kwargs):
-        cache_key = gen_cache_key(*messages)
+    def wrapper(self, model, messages, stream, tools=None, *args, **kwargs):
+        cache_key = gen_cache_key(messages, model, tools)
         logger.info(f"Stream cache key: {cache_key}")
-        
+
         redis_client = get_redis_instance()
         if redis_client:
             try:
                 cached_response = redis_client.get(cache_key)
                 if cached_response:
                     logger.info(f"Cache hit for stream key: {cache_key}")
-                    cached_response = json.loads(cached_response.decode('utf-8'))
+                    cached_response = json.loads(cached_response.decode("utf-8"))
                     for chunk in cached_response:
                         yield chunk
                         time.sleep(0.03)
@@ -76,18 +86,18 @@ def stream_cache(func):
             except redis.ConnectionError as e:
                 logger.error(f"Redis connection error: {e}")
 
-        result = func(self, model, messages, stream, *args, **kwargs)
+        result = func(self, model, messages, stream, tools=tools, *args, **kwargs)
         stream_cache_data = []
-        
+
         for chunk in result:
             stream_cache_data.append(chunk)
             yield chunk
-        
+
         if redis_client:
             try:
                 redis_client.set(cache_key, json.dumps(stream_cache_data), ex=1800)
                 logger.info(f"Stream cache saved for key: {cache_key}")
             except redis.ConnectionError as e:
                 logger.error(f"Redis connection error: {e}")
-        
-    return wrapper
+
+    return wrapper

application/celery_init.py

@@ -2,14 +2,22 @@ from celery import Celery
 from application.core.settings import settings
 from celery.signals import setup_logging
 
+
 def make_celery(app_name=__name__):
-    celery = Celery(app_name, broker=settings.CELERY_BROKER_URL, backend=settings.CELERY_RESULT_BACKEND)
+    celery = Celery(
+        app_name,
+        broker=settings.CELERY_BROKER_URL,
+        backend=settings.CELERY_RESULT_BACKEND,
+    )
     celery.conf.update(settings)
     return celery
 
+
 @setup_logging.connect
 def config_loggers(*args, **kwargs):
     from application.core.logging_config import setup_logging
+
     setup_logging()
 
+
 celery = make_celery()

application/core/settings.py

@@ -16,8 +16,9 @@ class Settings(BaseSettings):
     MONGO_URI: str = "mongodb://localhost:27017/docsgpt"
     MODEL_PATH: str = os.path.join(current_dir, "models/docsgpt-7b-f16.gguf")
     DEFAULT_MAX_HISTORY: int = 150
-    MODEL_TOKEN_LIMITS: dict = {"gpt-3.5-turbo": 4096, "claude-2": 1e5}
+    MODEL_TOKEN_LIMITS: dict = {"gpt-4o-mini": 128000, "gpt-3.5-turbo": 4096, "claude-2": 1e5}
     UPLOAD_FOLDER: str = "inputs"
+    PARSE_PDF_AS_IMAGE: bool = False
     VECTOR_STORE: str = "faiss" #  "faiss" or "elasticsearch" or "qdrant" or "milvus" or "lancedb"
     RETRIEVERS_ENABLED: list = ["classic_rag", "duckduck_search"] # also brave_search
 

application/llm/anthropic.py

@@ -17,7 +17,7 @@ class AnthropicLLM(BaseLLM):
         self.AI_PROMPT = AI_PROMPT
 
     def _raw_gen(
-        self, baseself, model, messages, stream=False, max_tokens=300, **kwargs
+        self, baseself, model, messages, stream=False, tools=None, max_tokens=300, **kwargs
     ):
         context = messages[0]["content"]
         user_question = messages[-1]["content"]
@@ -34,7 +34,7 @@ class AnthropicLLM(BaseLLM):
         return completion.completion
 
     def _raw_gen_stream(
-        self, baseself, model, messages, stream=True, max_tokens=300, **kwargs
+        self, baseself, model, messages, stream=True, tools=None, max_tokens=300, **kwargs
     ):
         context = messages[0]["content"]
         user_question = messages[-1]["content"]

application/llm/base.py

@@ -1,6 +1,7 @@
 from abc import ABC, abstractmethod
+
+from application.cache import gen_cache, stream_cache
 from application.usage import gen_token_usage, stream_token_usage
-from application.cache import stream_cache, gen_cache
 
 
 class BaseLLM(ABC):
@@ -13,17 +14,43 @@ class BaseLLM(ABC):
         return method(self, *args, **kwargs)
 
     @abstractmethod
-    def _raw_gen(self, model, messages, stream, *args, **kwargs):
+    def _raw_gen(self, model, messages, stream, tools, *args, **kwargs):
         pass
 
-    def gen(self, model, messages, stream=False, *args, **kwargs):
+    def gen(self, model, messages, stream=False, tools=None, *args, **kwargs):
         decorators = [gen_token_usage, gen_cache]
-        return self._apply_decorator(self._raw_gen, decorators=decorators, model=model, messages=messages, stream=stream, *args, **kwargs)
+        return self._apply_decorator(
+            self._raw_gen,
+            decorators=decorators,
+            model=model,
+            messages=messages,
+            stream=stream,
+            tools=tools,
+            *args,
+            **kwargs
+        )
 
     @abstractmethod
     def _raw_gen_stream(self, model, messages, stream, *args, **kwargs):
         pass
 
-    def gen_stream(self, model, messages, stream=True, *args, **kwargs):
+    def gen_stream(self, model, messages, stream=True, tools=None, *args, **kwargs):
         decorators = [stream_cache, stream_token_usage]
-        return self._apply_decorator(self._raw_gen_stream, decorators=decorators, model=model, messages=messages, stream=stream, *args, **kwargs)
+        return self._apply_decorator(
+            self._raw_gen_stream,
+            decorators=decorators,
+            model=model,
+            messages=messages,
+            stream=stream,
+            tools=tools,
+            *args,
+            **kwargs
+        )
+
+    def supports_tools(self):
+        return hasattr(self, "_supports_tools") and callable(
+            getattr(self, "_supports_tools")
+        )
+
+    def _supports_tools(self):
+        raise NotImplementedError("Subclass must implement _supports_tools method")

application/llm/google_ai.py

@@ -1,21 +1,95 @@
+from google import genai
+from google.genai import types
+
 from application.llm.base import BaseLLM
 
+
 class GoogleLLM(BaseLLM):
-
     def __init__(self, api_key=None, user_api_key=None, *args, **kwargs):
-
         super().__init__(*args, **kwargs)
         self.api_key = api_key
         self.user_api_key = user_api_key
 
     def _clean_messages_google(self, messages):
-        return [
-            {
-                "role": "model" if message["role"] == "system" else message["role"],
-                "parts": [message["content"]],
-            }
-            for message in messages[1:]
-        ]
+        cleaned_messages = []
+        for message in messages:
+            role = message.get("role")
+            content = message.get("content")
+
+            if role == "assistant":
+                role = "model"
+
+            parts = []
+            if role and content is not None:
+                if isinstance(content, str):
+                    parts = [types.Part.from_text(content)]
+                elif isinstance(content, list):
+                    for item in content:
+                        if "text" in item:
+                            parts.append(types.Part.from_text(item["text"]))
+                        elif "function_call" in item:
+                            parts.append(
+                                types.Part.from_function_call(
+                                    name=item["function_call"]["name"],
+                                    args=item["function_call"]["args"],
+                                )
+                            )
+                        elif "function_response" in item:
+                            parts.append(
+                                types.Part.from_function_response(
+                                    name=item["function_response"]["name"],
+                                    response=item["function_response"]["response"],
+                                )
+                            )
+                        else:
+                            raise ValueError(
+                                f"Unexpected content dictionary format:{item}"
+                            )
+                else:
+                    raise ValueError(f"Unexpected content type: {type(content)}")
+
+                cleaned_messages.append(types.Content(role=role, parts=parts))
+
+        return cleaned_messages
+
+    def _clean_tools_format(self, tools_list):
+        genai_tools = []
+        for tool_data in tools_list:
+            if tool_data["type"] == "function":
+                function = tool_data["function"]
+                parameters = function["parameters"]
+                properties = parameters.get("properties", {})
+
+                if properties:
+                    genai_function = dict(
+                        name=function["name"],
+                        description=function["description"],
+                        parameters={
+                            "type": "OBJECT",
+                            "properties": {
+                                k: {
+                                    **v,
+                                    "type": v["type"].upper() if v["type"] else None,
+                                }
+                                for k, v in properties.items()
+                            },
+                            "required": (
+                                parameters["required"]
+                                if "required" in parameters
+                                else []
+                            ),
+                        },
+                    )
+                else:
+                    genai_function = dict(
+                        name=function["name"],
+                        description=function["description"],
+                    )
+
+                genai_tool = types.Tool(function_declarations=[genai_function])
+                genai_tools.append(genai_tool)
+
+        return genai_tools
 
     def _raw_gen(
         self,
@@ -23,13 +97,32 @@ class GoogleLLM(BaseLLM):
         model,
         messages,
         stream=False,
-        **kwargs
-    ):  
-        import google.generativeai as genai
-        genai.configure(api_key=self.api_key)
-        model = genai.GenerativeModel(model, system_instruction=messages[0]["content"])
-        response = model.generate_content(self._clean_messages_google(messages))
-        return response.text
+        tools=None,
+        formatting="openai",
+        **kwargs,
+    ):
+        client = genai.Client(api_key=self.api_key)
+        if formatting == "openai":
+            messages = self._clean_messages_google(messages)
+        config = types.GenerateContentConfig()
+        if messages[0].role == "system":
+            config.system_instruction = messages[0].parts[0].text
+            messages = messages[1:]
+
+        if tools:
+            cleaned_tools = self._clean_tools_format(tools)
+            config.tools = cleaned_tools
+            response = client.models.generate_content(
+                model=model,
+                contents=messages,
+                config=config,
+            )
+            return response
+        else:
+            response = client.models.generate_content(
+                model=model, contents=messages, config=config
+            )
+            return response.text
 
     def _raw_gen_stream(
         self,
@@ -37,12 +130,30 @@ class GoogleLLM(BaseLLM):
         model,
         messages,
         stream=True,
-        **kwargs
-    ):  
-        import google.generativeai as genai
-        genai.configure(api_key=self.api_key)
-        model = genai.GenerativeModel(model, system_instruction=messages[0]["content"])
-        response = model.generate_content(self._clean_messages_google(messages), stream=True)
-        for line in response:
-            if line.text is not None:
-                yield line.text
+        tools=None,
+        formatting="openai",
+        **kwargs,
+    ):
+        client = genai.Client(api_key=self.api_key)
+        if formatting == "openai":
+            messages = self._clean_messages_google(messages)
+        config = types.GenerateContentConfig()
+        if messages[0].role == "system":
+            config.system_instruction = messages[0].parts[0].text
+            messages = messages[1:]
+
+        if tools:
+            cleaned_tools = self._clean_tools_format(tools)
+            config.tools = cleaned_tools
+
+        response = client.models.generate_content_stream(
+            model=model,
+            contents=messages,
+            config=config,
+        )
+        for chunk in response:
+            if chunk.text is not None:
+                yield chunk.text
+
+    def _supports_tools(self):
+        return True

application/llm/groq.py

@@ -1,45 +1,32 @@
 from application.llm.base import BaseLLM
-
+from openai import OpenAI
 
 
 class GroqLLM(BaseLLM):
-
     def __init__(self, api_key=None, user_api_key=None, *args, **kwargs):
-        from openai import OpenAI
-
         super().__init__(*args, **kwargs)
         self.client = OpenAI(api_key=api_key, base_url="https://api.groq.com/openai/v1")
         self.api_key = api_key
         self.user_api_key = user_api_key
 
-    def _raw_gen(
-        self,
-        baseself,
-        model,
-        messages,
-        stream=False,
-        **kwargs
-    ):  
-        response = self.client.chat.completions.create(
-            model=model, messages=messages, stream=stream, **kwargs
-        )
-
-        return response.choices[0].message.content
+    def _raw_gen(self, baseself, model, messages, stream=False, tools=None, **kwargs):
+        if tools:
+            response = self.client.chat.completions.create(
+                model=model, messages=messages, stream=stream, tools=tools, **kwargs
+            )
+            return response.choices[0]
+        else:
+            response = self.client.chat.completions.create(
+                model=model, messages=messages, stream=stream, **kwargs
+            )
+            return response.choices[0].message.content
 
     def _raw_gen_stream(
-        self,
-        baseself,
-        model,
-        messages,
-        stream=True,
-        **kwargs
-    ):  
+        self, baseself, model, messages, stream=True, tools=None, **kwargs
+    ):
         response = self.client.chat.completions.create(
             model=model, messages=messages, stream=stream, **kwargs
         )
-
         for line in response:
-            # import sys
-            # print(line.choices[0].delta.content, file=sys.stderr)
             if line.choices[0].delta.content is not None:
                 yield line.choices[0].delta.content

application/llm/openai.py

@@ -1,6 +1,5 @@
-from application.llm.base import BaseLLM
 from application.core.settings import settings
-
+from application.llm.base import BaseLLM
 
 
 class OpenAILLM(BaseLLM):
@@ -10,10 +9,7 @@ class OpenAILLM(BaseLLM):
 
         super().__init__(*args, **kwargs)
         if settings.OPENAI_BASE_URL:
-            self.client = OpenAI(
-                api_key=api_key,
-                base_url=settings.OPENAI_BASE_URL
-            )
+            self.client = OpenAI(api_key=api_key, base_url=settings.OPENAI_BASE_URL)
         else:
             self.client = OpenAI(api_key=api_key)
         self.api_key = api_key
@@ -25,14 +21,20 @@ class OpenAILLM(BaseLLM):
         model,
         messages,
         stream=False,
+        tools=None,
         engine=settings.AZURE_DEPLOYMENT_NAME,
-        **kwargs
-    ):  
-        response = self.client.chat.completions.create(
-            model=model, messages=messages, stream=stream, **kwargs
-        )
-
-        return response.choices[0].message.content
+        **kwargs,
+    ):
+        if tools:
+            response = self.client.chat.completions.create(
+                model=model, messages=messages, stream=stream, tools=tools, **kwargs
+            )
+            return response.choices[0]
+        else:
+            response = self.client.chat.completions.create(
+                model=model, messages=messages, stream=stream, **kwargs
+            )
+            return response.choices[0].message.content
 
     def _raw_gen_stream(
         self,
@@ -40,19 +42,21 @@ class OpenAILLM(BaseLLM):
         model,
         messages,
         stream=True,
+        tools=None,
         engine=settings.AZURE_DEPLOYMENT_NAME,
-        **kwargs
-    ):  
+        **kwargs,
+    ):
         response = self.client.chat.completions.create(
             model=model, messages=messages, stream=stream, **kwargs
         )
 
         for line in response:
-            # import sys
-            # print(line.choices[0].delta.content, file=sys.stderr)
             if line.choices[0].delta.content is not None:
                 yield line.choices[0].delta.content
 
+    def _supports_tools(self):
+        return True
+
 
 class AzureOpenAILLM(OpenAILLM):
 

application/llm/sagemaker.py

@@ -76,7 +76,7 @@ class SagemakerAPILLM(BaseLLM):
         self.endpoint = settings.SAGEMAKER_ENDPOINT
         self.runtime = runtime
 
-    def _raw_gen(self, baseself, model, messages, stream=False, **kwargs):
+    def _raw_gen(self, baseself, model, messages, stream=False, tools=None, **kwargs):
         context = messages[0]["content"]
         user_question = messages[-1]["content"]
         prompt = f"### Instruction \n {user_question} \n ### Context \n {context} \n ### Answer \n"
@@ -105,7 +105,7 @@ class SagemakerAPILLM(BaseLLM):
         print(result[0]["generated_text"], file=sys.stderr)
         return result[0]["generated_text"][len(prompt) :]
 
-    def _raw_gen_stream(self, baseself, model, messages, stream=True, **kwargs):
+    def _raw_gen_stream(self, baseself, model, messages, stream=True, tools=None, **kwargs):
         context = messages[0]["content"]
         user_question = messages[-1]["content"]
         prompt = f"### Instruction \n {user_question} \n ### Context \n {context} \n ### Answer \n"

application/parser/chunking.py

@@ -1,5 +1,5 @@
 import re
-from typing import List, Tuple, Union
+from typing import List, Tuple
 import logging
 from application.parser.schema.base import Document
 from application.utils import get_encoding

application/parser/embedding_pipeline.py

@@ -61,7 +61,7 @@ def embed_and_store_documents(docs, folder_name, source_id, task_status):
 
     # Process and embed documents
     for idx, doc in tqdm(
-        docs,
+        enumerate(docs),
         desc="Embedding 🦖",
         unit="docs",
         total=total_docs,
@@ -69,7 +69,7 @@ def embed_and_store_documents(docs, folder_name, source_id, task_status):
     ):
         try:
             # Update task status for progress tracking
-            progress = int((idx / total_docs) * 100)
+            progress = int(((idx + 1) / total_docs) * 100)
             task_status.update_state(state="PROGRESS", meta={"current": progress})
 
             # Add document to vector store

application/parser/file/bulk.py

@@ -13,6 +13,7 @@ from application.parser.file.rst_parser import RstParser
 from application.parser.file.tabular_parser import PandasCSVParser,ExcelParser
 from application.parser.file.json_parser import JSONParser
 from application.parser.file.pptx_parser import PPTXParser
+from application.parser.file.image_parser import ImageParser
 from application.parser.schema.base import Document
 
 DEFAULT_FILE_EXTRACTOR: Dict[str, BaseParser] = {
@@ -27,6 +28,9 @@ DEFAULT_FILE_EXTRACTOR: Dict[str, BaseParser] = {
     ".mdx": MarkdownParser(),
     ".json":JSONParser(),
     ".pptx":PPTXParser(),
+    ".png": ImageParser(),
+    ".jpg": ImageParser(),
+    ".jpeg": ImageParser(),
 }
 
 

application/parser/file/docs_parser.py

@@ -7,7 +7,8 @@ from pathlib import Path
 from typing import Dict
 
 from application.parser.file.base_parser import BaseParser
-
+from application.core.settings import settings
+import requests
 
 class PDFParser(BaseParser):
     """PDF parser."""
@@ -18,6 +19,15 @@ class PDFParser(BaseParser):
 
     def parse_file(self, file: Path, errors: str = "ignore") -> str:
         """Parse file."""
+        if settings.PARSE_PDF_AS_IMAGE:
+            doc2md_service = "https://llm.arc53.com/doc2md"
+            # alternatively you can use local vision capable LLM
+            with open(file, "rb") as file_loaded:
+                files = {'file': file_loaded}
+                response = requests.post(doc2md_service, files=files)   
+                data = response.json()["markdown"] 
+            return data
+
         try:
             import PyPDF2
         except ImportError:

application/parser/file/image_parser.py

@@ -0,0 +1,27 @@
+"""Image parser.
+
+Contains parser for .png, .jpg, .jpeg files.
+
+"""
+from pathlib import Path
+import requests
+from typing import Dict, Union
+
+from application.parser.file.base_parser import BaseParser
+
+
+class ImageParser(BaseParser):
+    """Image parser."""
+
+    def _init_parser(self) -> Dict:
+        """Init parser."""
+        return {}
+
+    def parse_file(self, file: Path, errors: str = "ignore") -> Union[str, list[str]]:
+        doc2md_service = "https://llm.arc53.com/doc2md"
+        # alternatively you can use local vision capable LLM
+        with open(file, "rb") as file_loaded:
+            files = {'file': file_loaded}
+            response = requests.post(doc2md_service, files=files)   
+            data = response.json()["markdown"] 
+        return data

application/parser/file/rst_parser.py

@@ -91,6 +91,25 @@ class RstParser(BaseParser):
             ]
         return rst_tups
 
+    def chunk_by_token_count(self, text: str, max_tokens: int = 100) -> List[str]:
+        """Chunk text by token count."""
+    
+        avg_token_length = 5
+    
+        chunk_size = max_tokens * avg_token_length
+
+        chunks = []
+        for i in range(0, len(text), chunk_size):
+            chunk = text[i:i+chunk_size]
+            if i + chunk_size < len(text):
+                last_space = chunk.rfind(' ')
+                if last_space != -1:
+                    chunk = chunk[:last_space]
+            
+            chunks.append(chunk.strip())
+        
+        return chunks
+    
     def remove_images(self, content: str) -> str:
         pattern = r"\.\. image:: (.*)"
         content = re.sub(pattern, "", content)
@@ -136,7 +155,7 @@ class RstParser(BaseParser):
         return {}
 
     def parse_tups(
-            self, filepath: Path, errors: str = "ignore"
+            self, filepath: Path, errors: str = "ignore",max_tokens: Optional[int] = 1000
     ) -> List[Tuple[Optional[str], str]]:
         """Parse file into tuples."""
         with open(filepath, "r") as f:
@@ -156,6 +175,15 @@ class RstParser(BaseParser):
             rst_tups = self.remove_whitespaces_excess(rst_tups)
         if self._remove_characters_excess:
             rst_tups = self.remove_characters_excess(rst_tups)
+
+        # Apply chunking if max_tokens is provided
+        if max_tokens is not None:
+            chunked_tups = []
+            for header, text in rst_tups:
+                chunks = self.chunk_by_token_count(text, max_tokens)
+                for idx, chunk in enumerate(chunks):
+                    chunked_tups.append((f"{header} - Chunk {idx + 1}", chunk))
+            return chunked_tups    
         return rst_tups
 
     def parse_file(

application/parser/late_chunking.py

@@ -1,94 +0,0 @@
-from typing import List, Tuple, Union, Optional
-from transformers import AutoTokenizer, AutoModel
-from sentence_transformers import SentenceTransformer
-import torch
-import torch.nn as nn
-from application.parser.schema.base import Document
-
-
-class LateChunker:
-    def __init__(self, model_name: str, late_tokens: int = 1000, **model_kwargs):
-        """
-        Initialize the LateChunker with a model, tokenizer, and late_tokens limit.
-        Supports both transformers and sentence-transformers models.
-        """
-        self.late_tokens = late_tokens
-        self.model_name = model_name
-
-        # Load model based on type
-        if "sentence-transformers" in model_name:
-            self.model = SentenceTransformer(model_name, **model_kwargs)
-            self.tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
-            self.wrapper_type = "sentence_transformers"
-        else:
-            self.model = AutoModel.from_pretrained(model_name, trust_remote_code=True, **model_kwargs)
-            self.tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
-            self.wrapper_type = "transformers"
-
-    def tokenize_with_offsets(self, text: str):
-        """Tokenize text and return tokens with character offsets."""
-        tokens = self.tokenizer.encode_plus(
-            text, return_offsets_mapping=True, add_special_tokens=False
-        )
-        return tokens["input_ids"], tokens["offset_mapping"]
-
-    def late_chunk_with_embeddings(
-        self, documents: List[Document]
-    ) -> List[Tuple[str, List[Tuple[int, int]], List[float]]]:
-        """
-        Combines documents into 'super chunks' that fit within `late_tokens` limit.
-        Outputs each super chunk with span annotations and embeddings.
-        """
-        super_chunks = []
-        current_super_chunk_text = []
-        current_token_count = 0
-        span_annotations = []
-
-        for doc in documents:
-            doc_text = doc.text
-            input_ids, offsets = self.tokenize_with_offsets(doc_text)
-            doc_token_count = len(input_ids)
-
-            # Check if adding this document exceeds the late_tokens limit
-            if current_token_count + doc_token_count > self.late_tokens:
-                # Finalize the current super chunk
-                combined_text = " ".join(current_super_chunk_text)
-                embeddings = self.generate_embeddings(combined_text)
-
-                super_chunks.append((combined_text, span_annotations, embeddings))
-
-                # Reset for a new super chunk
-                current_super_chunk_text = []
-                span_annotations = []
-                current_token_count = 0
-
-            # Add document to the current super chunk
-            start_token = current_token_count
-            end_token = current_token_count + doc_token_count
-            span_annotations.append((start_token, end_token))
-            current_super_chunk_text.append(doc_text)
-            current_token_count = end_token
-
-        # Add the final super chunk if there are remaining documents
-        if current_super_chunk_text:
-            combined_text = " ".join(current_super_chunk_text)
-            embeddings = self.generate_embeddings(combined_text)
-            super_chunks.append((combined_text, span_annotations, embeddings))
-
-        return super_chunks
-
-    def generate_embeddings(self, text: str) -> List[float]:
-        """Generate embeddings for a given text using the loaded model."""
-        if self.wrapper_type == "sentence_transformers":
-            # Sentence-Transformers
-            embeddings = self.model.encode([text])
-            return embeddings[0].tolist()
-
-        elif self.wrapper_type == "transformers":
-            # Transformers models
-            inputs = self.tokenizer(text, return_tensors="pt")
-            model_output = self.model(**inputs)
-            return model_output.last_hidden_state.mean(dim=1).squeeze().tolist()
-
-        else:
-            raise ValueError("Unsupported model type for embedding generation.")

application/parser/open_ai_func.py

@@ -1,75 +0,0 @@
-import os
-
-from retry import retry
-
-from application.core.settings import settings
-
-from application.vectorstore.vector_creator import VectorCreator
-
-
-# from langchain_community.embeddings import HuggingFaceEmbeddings
-# from langchain_community.embeddings import HuggingFaceInstructEmbeddings
-# from langchain_community.embeddings import CohereEmbeddings
-
-
-@retry(tries=10, delay=60)
-def store_add_texts_with_retry(store, i, id):
-    # add source_id to the metadata
-    i.metadata["source_id"] = str(id)
-    store.add_texts([i.page_content], metadatas=[i.metadata])
-    # store_pine.add_texts([i.page_content], metadatas=[i.metadata])
-
-
-def call_openai_api(docs, folder_name, id, task_status):
-    # Function to create a vector store from the documents and save it to disk
-
-    if not os.path.exists(f"{folder_name}"):
-        os.makedirs(f"{folder_name}")
-
-    from tqdm import tqdm
-
-    c1 = 0
-    if settings.VECTOR_STORE == "faiss":
-        docs_init = [docs[0]]
-        docs.pop(0)
-
-        store = VectorCreator.create_vectorstore(
-            settings.VECTOR_STORE,
-            docs_init=docs_init,
-            source_id=f"{folder_name}",
-            embeddings_key=os.getenv("EMBEDDINGS_KEY"),
-        )
-    else:
-        store = VectorCreator.create_vectorstore(
-            settings.VECTOR_STORE,
-            source_id=str(id),
-            embeddings_key=os.getenv("EMBEDDINGS_KEY"),
-        )
-        store.delete_index()
-    # Uncomment for MPNet embeddings
-    # model_name = "sentence-transformers/all-mpnet-base-v2"
-    # hf = HuggingFaceEmbeddings(model_name=model_name)
-    # store = FAISS.from_documents(docs_test, hf)
-    s1 = len(docs)
-    for i in tqdm(
-        docs,
-        desc="Embedding 🦖",
-        unit="docs",
-        total=len(docs),
-        bar_format="{l_bar}{bar}| Time Left: {remaining}",
-    ):
-        try:
-            task_status.update_state(
-                state="PROGRESS", meta={"current": int((c1 / s1) * 100)}
-            )
-            store_add_texts_with_retry(store, i, id)
-        except Exception as e:
-            print(e)
-            print("Error on ", i)
-            print("Saving progress")
-            print(f"stopped at {c1} out of {len(docs)}")
-            store.save_local(f"{folder_name}")
-            break
-        c1 += 1
-    if settings.VECTOR_STORE == "faiss":
-        store.save_local(f"{folder_name}")

application/parser/remote/crawler_loader.py

@@ -2,16 +2,16 @@ import requests
 from urllib.parse import urlparse, urljoin
 from bs4 import BeautifulSoup
 from application.parser.remote.base import BaseRemote
+from application.parser.schema.base import Document
+from langchain_community.document_loaders import WebBaseLoader
 
 class CrawlerLoader(BaseRemote):
     def __init__(self, limit=10):
-        from langchain_community.document_loaders import WebBaseLoader
         self.loader = WebBaseLoader  # Initialize the document loader
         self.limit = limit  # Set the limit for the number of pages to scrape
 
     def load_data(self, inputs):
         url = inputs
-        # Check if the input is a list and if it is, use the first element
         if isinstance(url, list) and url:
             url = url[0]
 
@@ -19,24 +19,29 @@ class CrawlerLoader(BaseRemote):
         if not urlparse(url).scheme:
             url = "http://" + url
 
-        visited_urls = set()  # Keep track of URLs that have been visited
-        base_url = urlparse(url).scheme + "://" + urlparse(url).hostname  # Extract the base URL
-        urls_to_visit = [url]  # List of URLs to be visited, starting with the initial URL
-        loaded_content = []  # Store the loaded content from each URL
+        visited_urls = set()
+        base_url = urlparse(url).scheme + "://" + urlparse(url).hostname
+        urls_to_visit = [url]
+        loaded_content = []
 
-        # Continue crawling until there are no more URLs to visit
         while urls_to_visit:
-            current_url = urls_to_visit.pop(0)  # Get the next URL to visit
-            visited_urls.add(current_url)  # Mark the URL as visited
+            current_url = urls_to_visit.pop(0)
+            visited_urls.add(current_url)
 
-            # Try to load and process the content from the current URL
             try:
-                response = requests.get(current_url)  # Fetch the content of the current URL
-                response.raise_for_status()  # Raise an exception for HTTP errors
-                loader = self.loader([current_url])  # Initialize the document loader for the current URL
-                loaded_content.extend(loader.load())  # Load the content and add it to the loaded_content list
+                response = requests.get(current_url)
+                response.raise_for_status()
+                loader = self.loader([current_url])
+                docs = loader.load()
+                # Convert the loaded documents to your Document schema
+                for doc in docs:
+                    loaded_content.append(
+                        Document(
+                            doc.page_content,
+                            extra_info=doc.metadata
+                        )
+                    )
             except Exception as e:
-                # Print an error message if loading or processing fails and continue with the next URL
                 print(f"Error processing URL {current_url}: {e}")
                 continue
 
@@ -45,15 +50,15 @@ class CrawlerLoader(BaseRemote):
             all_links = [
                 urljoin(current_url, a['href'])
                 for a in soup.find_all('a', href=True)
-                if base_url in urljoin(current_url, a['href'])  # Ensure links are from the same domain
+                if base_url in urljoin(current_url, a['href'])
             ]
 
             # Add new links to the list of URLs to visit if they haven't been visited yet
             urls_to_visit.extend([link for link in all_links if link not in visited_urls])
-            urls_to_visit = list(set(urls_to_visit))  # Remove duplicate URLs
+            urls_to_visit = list(set(urls_to_visit))
 
             # Stop crawling if the limit of pages to scrape is reached
             if self.limit is not None and len(visited_urls) >= self.limit:
                 break
 
-        return loaded_content  # Return the loaded content from all visited URLs
+        return loaded_content

application/parser/remote/crawler_markdown.py

@@ -0,0 +1,139 @@
+import requests
+from urllib.parse import urlparse, urljoin
+from bs4 import BeautifulSoup
+from application.parser.remote.base import BaseRemote
+import re
+from markdownify import markdownify
+from application.parser.schema.base import Document
+import tldextract
+
+class CrawlerLoader(BaseRemote):
+    def __init__(self, limit=10, allow_subdomains=False):
+        """
+        Given a URL crawl web pages up to `self.limit`,
+        convert HTML content to Markdown, and returning a list of Document objects.
+
+        :param limit: The maximum number of pages to crawl.
+        :param allow_subdomains: If True, crawl pages on subdomains of the base domain.
+        """
+        self.limit = limit
+        self.allow_subdomains = allow_subdomains
+        self.session = requests.Session()
+
+    def load_data(self, inputs):
+        url = inputs
+        if isinstance(url, list) and url:
+            url = url[0]
+
+        # Ensure the URL has a scheme (if not, default to http)
+        if not urlparse(url).scheme:
+            url = "http://" + url
+
+        # Keep track of visited URLs to avoid revisiting the same page
+        visited_urls = set()
+
+        # Determine the base domain for link filtering using tldextract
+        base_domain = self._get_base_domain(url)
+        urls_to_visit = {url}
+        documents = []
+
+        while urls_to_visit:
+            current_url = urls_to_visit.pop()
+
+            # Skip if already visited
+            if current_url in visited_urls:
+                continue
+            visited_urls.add(current_url)
+
+            # Fetch the page content
+            html_content = self._fetch_page(current_url)
+            if html_content is None:
+                continue
+
+            # Convert the HTML to Markdown for cleaner text formatting
+            title, language, processed_markdown = self._process_html_to_markdown(html_content, current_url)
+            if processed_markdown:
+                # Create a Document for each visited page
+                documents.append(
+                    Document(
+                        processed_markdown,  # content
+                        None,  # doc_id
+                        None,  # embedding
+                        {"source": current_url, "title": title, "language": language} # extra_info
+                    )
+                )
+
+            # Extract links and filter them according to domain rules
+            new_links = self._extract_links(html_content, current_url)
+            filtered_links = self._filter_links(new_links, base_domain)
+
+            # Add any new, not-yet-visited links to the queue
+            urls_to_visit.update(link for link in filtered_links if link not in visited_urls)
+
+            # If we've reached the limit, stop crawling
+            if self.limit is not None and len(visited_urls) >= self.limit:
+                break
+
+        return documents
+
+    def _fetch_page(self, url):
+        try:
+            response = self.session.get(url, timeout=10)
+            response.raise_for_status()
+            return response.text
+        except requests.exceptions.RequestException as e:
+            print(f"Error fetching URL {url}: {e}")
+            return None
+
+    def _process_html_to_markdown(self, html_content, current_url):
+        soup = BeautifulSoup(html_content, 'html.parser')
+        title_tag = soup.find('title')
+        title = title_tag.text.strip() if title_tag else "No Title"
+
+        # Extract language
+        language_tag = soup.find('html')
+        language = language_tag.get('lang', 'en') if language_tag else "en"
+
+        markdownified = markdownify(html_content, heading_style="ATX", newline_style="BACKSLASH")
+        # Reduce sequences of more than two newlines to exactly three
+        markdownified = re.sub(r'\n{3,}', '\n\n\n', markdownified)
+        return title, language, markdownified
+
+    def _extract_links(self, html_content, current_url):
+        soup = BeautifulSoup(html_content, 'html.parser')
+        links = []
+        for a in soup.find_all('a', href=True):
+            full_url = urljoin(current_url, a['href'])
+            links.append((full_url, a.text.strip()))
+        return links
+
+    def _get_base_domain(self, url):
+        extracted = tldextract.extract(url)
+        # Reconstruct the domain as domain.suffix
+        base_domain = f"{extracted.domain}.{extracted.suffix}"
+        return base_domain
+
+    def _filter_links(self, links, base_domain):
+        """
+        Filter the extracted links to only include those that match the crawling criteria:
+        - If allow_subdomains is True, allow any link whose domain ends with the base_domain.
+        - If allow_subdomains is False, only allow exact matches of the base_domain.
+        """
+        filtered = []
+        for link, _ in links:
+            parsed_link = urlparse(link)
+            if not parsed_link.netloc:
+                continue
+
+            extracted = tldextract.extract(parsed_link.netloc)
+            link_base = f"{extracted.domain}.{extracted.suffix}"
+
+            if self.allow_subdomains:
+                # For subdomains: sub.example.com ends with example.com
+                if link_base == base_domain or link_base.endswith("." + base_domain):
+                    filtered.append(link)
+            else:
+                # Exact domain match
+                if link_base == base_domain:
+                    filtered.append(link)
+        return filtered

application/parser/remote/web_loader.py

@@ -1,5 +1,7 @@
 from application.parser.remote.base import BaseRemote
+from application.parser.schema.base import Document
 from langchain_community.document_loaders import WebBaseLoader
+from urllib.parse import urlparse
 
 headers = {
     "User-Agent": "Mozilla/5.0",
@@ -23,10 +25,20 @@ class WebLoader(BaseRemote):
             urls = [urls]
         documents = []
         for url in urls:
+            # Check if the URL scheme is provided, if not, assume http
+            if not urlparse(url).scheme:
+                url = "http://" + url
             try:
                 loader = self.loader([url], header_template=headers)
-                documents.extend(loader.load())
+                loaded_docs = loader.load()
+                for doc in loaded_docs:
+                    documents.append(
+                        Document(
+                            doc.page_content,
+                            extra_info=doc.metadata,
+                        )
+                    )
             except Exception as e:
                 print(f"Error processing URL {url}: {e}")
                 continue
-        return documents
+        return documents

application/parser/token_func.py

@@ -1,79 +0,0 @@
-import re
-from math import ceil
-from typing import List
-
-import tiktoken
-from application.parser.schema.base import Document
-
-
-def separate_header_and_body(text):
-    header_pattern = r"^(.*?\n){3}"
-    match = re.match(header_pattern, text)
-    header = match.group(0)
-    body = text[len(header):]
-    return header, body
-
-
-def group_documents(documents: List[Document], min_tokens: int, max_tokens: int) -> List[Document]:
-    docs = []
-    current_group = None
-
-    for doc in documents:
-        doc_len = len(tiktoken.get_encoding("cl100k_base").encode(doc.text))
-
-        # Check if current group is empty or if the document can be added based on token count and matching metadata
-        if (current_group is None or 
-            (len(tiktoken.get_encoding("cl100k_base").encode(current_group.text)) + doc_len < max_tokens and 
-             doc_len < min_tokens and 
-             current_group.extra_info == doc.extra_info)):
-            if current_group is None:
-                current_group = doc  # Use the document directly to retain its metadata
-            else:
-                current_group.text += " " + doc.text  # Append text to the current group
-        else:
-            docs.append(current_group)
-            current_group = doc  # Start a new group with the current document
-
-    if current_group is not None:
-        docs.append(current_group)
-
-    return docs
-
-
-def split_documents(documents: List[Document], max_tokens: int) -> List[Document]:
-    docs = []
-    for doc in documents:
-        token_length = len(tiktoken.get_encoding("cl100k_base").encode(doc.text))
-        if token_length <= max_tokens:
-            docs.append(doc)
-        else:
-            header, body = separate_header_and_body(doc.text)
-            if len(tiktoken.get_encoding("cl100k_base").encode(header)) > max_tokens:
-                body = doc.text
-                header = ""
-            num_body_parts = ceil(token_length / max_tokens)
-            part_length = ceil(len(body) / num_body_parts)
-            body_parts = [body[i:i + part_length] for i in range(0, len(body), part_length)]
-            for i, body_part in enumerate(body_parts):
-                new_doc = Document(text=header + body_part.strip(),
-                                   doc_id=f"{doc.doc_id}-{i}",
-                                   embedding=doc.embedding,
-                                   extra_info=doc.extra_info)
-                docs.append(new_doc)
-    return docs
-
-
-def group_split(documents: List[Document], max_tokens: int = 2000, min_tokens: int = 150, token_check: bool = True):
-    if not token_check:
-        return documents
-    print("Grouping small documents")
-    try:
-        documents = group_documents(documents=documents, min_tokens=min_tokens, max_tokens=max_tokens)
-    except Exception:
-        print("Grouping failed, try running without token_check")
-    print("Separating large documents")
-    try:
-        documents = split_documents(documents=documents, max_tokens=max_tokens)
-    except Exception:
-        print("Grouping failed, try running without token_check")
-    return documents

application/requirements.txt

@@ -1,25 +1,27 @@
-anthropic==0.34.2
-boto3==1.34.153
+anthropic==0.40.0
+boto3==1.35.97
 beautifulsoup4==4.12.3
-celery==5.3.6
+celery==5.4.0
 dataclasses-json==0.6.7
 docx2txt==0.8
 duckduckgo-search==6.3.0
 ebooklib==0.18
-elastic-transport==8.15.0
-elasticsearch==8.15.1
+elastic-transport==8.17.0
+elasticsearch==8.17.0
 escodegen==1.0.11
 esprima==4.0.1
 esutils==1.0.1
-Flask==3.0.3
-faiss-cpu==1.8.0.post1
+Flask==3.1.0
+faiss-cpu==1.9.0.post1
 flask-restx==1.3.0
-gTTS==2.3.2
+google-genai==0.5.0
+google-generativeai==0.8.3
+gTTS==2.5.4
 gunicorn==23.0.0
 html2text==2024.2.26
 javalang==0.13.0
-jinja2==3.1.4
-jiter==0.5.0
+jinja2==3.1.5
+jiter==0.8.2
 jmespath==1.0.1
 joblib==1.4.2
 jsonpatch==1.33
@@ -28,62 +30,66 @@ jsonschema==4.23.0
 jsonschema-spec==0.2.4
 jsonschema-specifications==2023.7.1
 kombu==5.4.2
-langchain==0.3.0
-langchain-community==0.3.0
-langchain-core==0.3.2
-langchain-openai==0.2.0
-langchain-text-splitters==0.3.0
-langsmith==0.1.125
+langchain==0.3.14
+langchain-community==0.3.14
+langchain-core==0.3.29
+langchain-openai==0.3.0
+langchain-text-splitters==0.3.5
+langsmith==0.2.10
 lazy-object-proxy==1.10.0
 lxml==5.3.0
-markupsafe==2.1.5
-marshmallow==3.22.0
+markupsafe==3.0.2
+marshmallow==3.24.1
 mpmath==1.3.0
 multidict==6.1.0
 mypy-extensions==1.0.0
-networkx==3.3
-numpy==1.26.4
-openai==1.46.1
+networkx==3.4.2
+numpy==2.2.1
+openai==1.59.5
 openapi-schema-validator==0.6.2
 openapi-spec-validator==0.6.0
-openapi3-parser==1.1.18
-orjson==3.10.7
+openapi3-parser==1.1.19
+orjson==3.10.14
 packaging==24.1
 pandas==2.2.3
 openpyxl==3.1.5
-pathable==0.4.3
-pillow==10.4.0
+pathable==0.4.4
+pillow==11.1.0
 portalocker==2.10.1
 prance==23.6.21.0
-primp==0.6.3
-prompt-toolkit==3.0.47
-protobuf==5.28.2
+primp==0.10.0
+prompt-toolkit==3.0.48
+protobuf==5.29.3
+psycopg2-binary==2.9.10
 py==1.11.0
-pydantic==2.9.2
-pydantic-core==2.23.4
-pydantic-settings==2.4.0
-pymongo==4.8.0
+pydantic==2.10.4
+pydantic-core==2.27.2
+pydantic-settings==2.7.1
+pymongo==4.10.1
 pypdf2==3.0.1
 python-dateutil==2.9.0.post0
 python-dotenv==1.0.1
 python-pptx==1.0.2
-qdrant-client==1.11.0
-redis==5.0.1
+qdrant-client==1.12.2
+redis==5.2.1
 referencing==0.30.2
-regex==2024.9.11
+regex==2024.11.6
 requests==2.32.3
 retry==0.9.2
-sentence-transformers==3.0.1
-tiktoken==0.7.0
-tokenizers==0.19.1
-torch==2.4.1
-tqdm==4.66.5
-transformers==4.44.2
+sentence-transformers==3.3.1
+tiktoken==0.8.0
+tokenizers==0.21.0
+torch==2.5.1
+tqdm==4.67.1
+transformers==4.48.0
 typing-extensions==4.12.2
 typing-inspect==0.9.0
 tzdata==2024.2
-urllib3==2.2.3
+urllib3==2.3.0
 vine==5.1.0
 wcwidth==0.2.13
-werkzeug==3.0.4
-yarl==1.11.1
+werkzeug==3.1.3
+yarl==1.18.3
+markdownify==0.14.1
+tldextract==5.1.3
+websockets==14.1

application/retriever/brave_search.py

@@ -2,7 +2,6 @@ import json
 from application.retriever.base import BaseRetriever
 from application.core.settings import settings
 from application.llm.llm_creator import LLMCreator
-from application.utils import num_tokens_from_string
 from langchain_community.tools import BraveSearch
 
 
@@ -72,22 +71,13 @@ class BraveRetSearch(BaseRetriever):
         for doc in docs:
             yield {"source": doc}
 
-        if len(self.chat_history) > 1:
-            tokens_current_history = 0
-            # count tokens in history
+        if len(self.chat_history) > 0:
             for i in self.chat_history:
                 if "prompt" in i and "response" in i:
-                    tokens_batch = num_tokens_from_string(i["prompt"]) + num_tokens_from_string(
-                        i["response"]
+                    messages_combine.append({"role": "user", "content": i["prompt"]})
+                    messages_combine.append(
+                        {"role": "assistant", "content": i["response"]}
                     )
-                    if tokens_current_history + tokens_batch < self.token_limit:
-                        tokens_current_history += tokens_batch
-                        messages_combine.append(
-                            {"role": "user", "content": i["prompt"]}
-                        )
-                        messages_combine.append(
-                            {"role": "system", "content": i["response"]}
-                        )
         messages_combine.append({"role": "user", "content": self.question})
 
         llm = LLMCreator.create_llm(

application/retriever/classic_rag.py

@@ -1,9 +1,8 @@
-from application.retriever.base import BaseRetriever
 from application.core.settings import settings
-from application.vectorstore.vector_creator import VectorCreator
-from application.llm.llm_creator import LLMCreator
+from application.retriever.base import BaseRetriever
+from application.tools.agent import Agent
 
-from application.utils import num_tokens_from_string
+from application.vectorstore.vector_creator import VectorCreator
 
 
 class ClassicRAG(BaseRetriever):
@@ -20,7 +19,7 @@ class ClassicRAG(BaseRetriever):
         user_api_key=None,
     ):
         self.question = question
-        self.vectorstore = source['active_docs'] if 'active_docs' in source else None
+        self.vectorstore = source["active_docs"] if "active_docs" in source else None
         self.chat_history = chat_history
         self.prompt = prompt
         self.chunks = chunks
@@ -72,34 +71,31 @@ class ClassicRAG(BaseRetriever):
         for doc in docs:
             yield {"source": doc}
 
-        if len(self.chat_history) > 1:
-            tokens_current_history = 0
-            # count tokens in history
+        if len(self.chat_history) > 0:
             for i in self.chat_history:
                 if "prompt" in i and "response" in i:
-                    tokens_batch = num_tokens_from_string(i["prompt"]) + num_tokens_from_string(
-                        i["response"]
+                    messages_combine.append({"role": "user", "content": i["prompt"]})
+                    messages_combine.append(
+                        {"role": "assistant", "content": i["response"]}
                     )
-                    if tokens_current_history + tokens_batch < self.token_limit:
-                        tokens_current_history += tokens_batch
-                        messages_combine.append(
-                            {"role": "user", "content": i["prompt"]}
-                        )
-                        messages_combine.append(
-                            {"role": "system", "content": i["response"]}
-                        )
         messages_combine.append({"role": "user", "content": self.question})
-
-        llm = LLMCreator.create_llm(
-            settings.LLM_NAME, api_key=settings.API_KEY, user_api_key=self.user_api_key
+        # llm = LLMCreator.create_llm(
+        #     settings.LLM_NAME, api_key=settings.API_KEY, user_api_key=self.user_api_key
+        # )
+        # completion = llm.gen_stream(model=self.gpt_model, messages=messages_combine)
+        agent = Agent(
+            llm_name=settings.LLM_NAME,
+            gpt_model=self.gpt_model,
+            api_key=settings.API_KEY,
+            user_api_key=self.user_api_key,
         )
-        completion = llm.gen_stream(model=self.gpt_model, messages=messages_combine)
+        completion = agent.gen(messages_combine)
         for line in completion:
             yield {"answer": str(line)}
 
     def search(self):
         return self._get_data()
-    
+
     def get_params(self):
         return {
             "question": self.question,
@@ -109,5 +105,5 @@ class ClassicRAG(BaseRetriever):
             "chunks": self.chunks,
             "token_limit": self.token_limit,
             "gpt_model": self.gpt_model,
-            "user_api_key": self.user_api_key
+            "user_api_key": self.user_api_key,
         }

application/retriever/duckduck_search.py

@@ -1,7 +1,6 @@
 from application.retriever.base import BaseRetriever
 from application.core.settings import settings
 from application.llm.llm_creator import LLMCreator
-from application.utils import num_tokens_from_string
 from langchain_community.tools import DuckDuckGoSearchResults
 from langchain_community.utilities import DuckDuckGoSearchAPIWrapper
 
@@ -89,21 +88,12 @@ class DuckDuckSearch(BaseRetriever):
         for doc in docs:
             yield {"source": doc}
 
-        if len(self.chat_history) > 1:
-            tokens_current_history = 0
-            # count tokens in history
+        if len(self.chat_history) > 0:      
             for i in self.chat_history:
-                if "prompt" in i and "response" in i:
-                    tokens_batch = num_tokens_from_string(i["prompt"]) + num_tokens_from_string(
-                        i["response"]
-                    )
-                    if tokens_current_history + tokens_batch < self.token_limit:
-                        tokens_current_history += tokens_batch
+                    if "prompt" in i and "response" in i:
+                        messages_combine.append({"role": "user", "content": i["prompt"]})
                         messages_combine.append(
-                            {"role": "user", "content": i["prompt"]}
-                        )
-                        messages_combine.append(
-                            {"role": "system", "content": i["response"]}
+                            {"role": "assistant", "content": i["response"]}
                         )
         messages_combine.append({"role": "user", "content": self.question})
 

application/tools/agent.py

@@ -0,0 +1,160 @@
+from application.core.mongo_db import MongoDB
+from application.llm.llm_creator import LLMCreator
+from application.tools.llm_handler import get_llm_handler
+from application.tools.tool_action_parser import ToolActionParser
+from application.tools.tool_manager import ToolManager
+
+
+class Agent:
+    def __init__(self, llm_name, gpt_model, api_key, user_api_key=None):
+        # Initialize the LLM with the provided parameters
+        self.llm = LLMCreator.create_llm(
+            llm_name, api_key=api_key, user_api_key=user_api_key
+        )
+        self.llm_handler = get_llm_handler(llm_name)
+        self.gpt_model = gpt_model
+        # Static tool configuration (to be replaced later)
+        self.tools = []
+        self.tool_config = {}
+
+    def _get_user_tools(self, user="local"):
+        mongo = MongoDB.get_client()
+        db = mongo["docsgpt"]
+        user_tools_collection = db["user_tools"]
+        user_tools = user_tools_collection.find({"user": user, "status": True})
+        user_tools = list(user_tools)
+        tools_by_id = {str(tool["_id"]): tool for tool in user_tools}
+        return tools_by_id
+
+    def _build_tool_parameters(self, action):
+        params = {"type": "object", "properties": {}, "required": []}
+        for param_type in ["query_params", "headers", "body", "parameters"]:
+            if param_type in action and action[param_type].get("properties"):
+                for k, v in action[param_type]["properties"].items():
+                    if v.get("filled_by_llm", True):
+                        params["properties"][k] = {
+                            key: value
+                            for key, value in v.items()
+                            if key != "filled_by_llm" and key != "value"
+                        }
+
+                        params["required"].append(k)
+        return params
+
+    def _prepare_tools(self, tools_dict):
+        self.tools = [
+            {
+                "type": "function",
+                "function": {
+                    "name": f"{action['name']}_{tool_id}",
+                    "description": action["description"],
+                    "parameters": self._build_tool_parameters(action),
+                },
+            }
+            for tool_id, tool in tools_dict.items()
+            for action in (
+                tool["config"]["actions"].values()
+                if tool["name"] == "api_tool"
+                else tool["actions"]
+            )
+            if action.get("active", True)
+        ]
+
+    def _execute_tool_action(self, tools_dict, call):
+        parser = ToolActionParser(self.llm.__class__.__name__)
+        tool_id, action_name, call_args = parser.parse_args(call)
+
+        tool_data = tools_dict[tool_id]
+        action_data = (
+            tool_data["config"]["actions"][action_name]
+            if tool_data["name"] == "api_tool"
+            else next(
+                action
+                for action in tool_data["actions"]
+                if action["name"] == action_name
+            )
+        )
+
+        query_params, headers, body, parameters = {}, {}, {}, {}
+        param_types = {
+            "query_params": query_params,
+            "headers": headers,
+            "body": body,
+            "parameters": parameters,
+        }
+
+        for param_type, target_dict in param_types.items():
+            if param_type in action_data and action_data[param_type].get("properties"):
+                for param, details in action_data[param_type]["properties"].items():
+                    if param not in call_args and "value" in details:
+                        target_dict[param] = details["value"]
+
+        for param, value in call_args.items():
+            for param_type, target_dict in param_types.items():
+                if param_type in action_data and param in action_data[param_type].get(
+                    "properties", {}
+                ):
+                    target_dict[param] = value
+
+        tm = ToolManager(config={})
+        tool = tm.load_tool(
+            tool_data["name"],
+            tool_config=(
+                {
+                    "url": tool_data["config"]["actions"][action_name]["url"],
+                    "method": tool_data["config"]["actions"][action_name]["method"],
+                    "headers": headers,
+                    "query_params": query_params,
+                }
+                if tool_data["name"] == "api_tool"
+                else tool_data["config"]
+            ),
+        )
+        if tool_data["name"] == "api_tool":
+            print(
+                f"Executing api: {action_name} with query_params: {query_params}, headers: {headers}, body: {body}"
+            )
+            result = tool.execute_action(action_name, **body)
+        else:
+            print(f"Executing tool: {action_name} with args: {call_args}")
+            result = tool.execute_action(action_name, **parameters)
+        call_id = getattr(call, "id", None)
+        return result, call_id
+
+    def _simple_tool_agent(self, messages):
+        tools_dict = self._get_user_tools()
+        self._prepare_tools(tools_dict)
+
+        resp = self.llm.gen(model=self.gpt_model, messages=messages, tools=self.tools)
+
+        if isinstance(resp, str):
+            yield resp
+            return
+        if hasattr(resp, "message") and hasattr(resp.message, "content"):
+            yield resp.message.content
+            return
+
+        resp = self.llm_handler.handle_response(self, resp, tools_dict, messages)
+
+        if isinstance(resp, str):
+            yield resp
+        elif hasattr(resp, "message") and hasattr(resp.message, "content"):
+            yield resp.message.content
+        else:
+            completion = self.llm.gen_stream(
+                model=self.gpt_model, messages=messages, tools=self.tools
+            )
+            for line in completion:
+                yield line
+
+        return
+
+    def gen(self, messages):
+        if self.llm.supports_tools():
+            resp = self._simple_tool_agent(messages)
+            for line in resp:
+                yield line
+        else:
+            resp = self.llm.gen_stream(model=self.gpt_model, messages=messages)
+            for line in resp:
+                yield line

application/tools/base.py

@@ -0,0 +1,21 @@
+from abc import ABC, abstractmethod
+
+
+class Tool(ABC):
+    @abstractmethod
+    def execute_action(self, action_name: str, **kwargs):
+        pass
+
+    @abstractmethod
+    def get_actions_metadata(self):
+        """
+        Returns a list of JSON objects describing the actions supported by the tool.
+        """
+        pass
+
+    @abstractmethod
+    def get_config_requirements(self):
+        """
+        Returns a dictionary describing the configuration requirements for the tool.
+        """
+        pass

application/tools/implementations/api_tool.py

@@ -0,0 +1,54 @@
+import json
+
+import requests
+from application.tools.base import Tool
+
+
+class APITool(Tool):
+    """
+    API Tool
+    A flexible tool for performing various API actions (e.g., sending messages, retrieving data) via custom user-specified APIs
+    """
+
+    def __init__(self, config):
+        self.config = config
+        self.url = config.get("url", "")
+        self.method = config.get("method", "GET")
+        self.headers = config.get("headers", {"Content-Type": "application/json"})
+        self.query_params = config.get("query_params", {})
+
+    def execute_action(self, action_name, **kwargs):
+        return self._make_api_call(
+            self.url, self.method, self.headers, self.query_params, kwargs
+        )
+
+    def _make_api_call(self, url, method, headers, query_params, body):
+        if query_params:
+            url = f"{url}?{requests.compat.urlencode(query_params)}"
+        if isinstance(body, dict):
+            body = json.dumps(body)
+        try:
+            print(f"Making API call: {method} {url} with body: {body}")
+            response = requests.request(method, url, headers=headers, data=body)
+            response.raise_for_status()
+            try:
+                data = response.json()
+            except ValueError:
+                data = None
+
+            return {
+                "status_code": response.status_code,
+                "data": data,
+                "message": "API call successful.",
+            }
+        except requests.exceptions.RequestException as e:
+            return {
+                "status_code": response.status_code if response else None,
+                "message": f"API call failed: {str(e)}",
+            }
+
+    def get_actions_metadata(self):
+        return []
+
+    def get_config_requirements(self):
+        return {}

application/tools/implementations/cryptoprice.py

@@ -0,0 +1,77 @@
+import requests
+from application.tools.base import Tool
+
+
+class CryptoPriceTool(Tool):
+    """
+    CryptoPrice
+    A tool for retrieving cryptocurrency prices using the CryptoCompare public API
+    """
+
+    def __init__(self, config):
+        self.config = config
+
+    def execute_action(self, action_name, **kwargs):
+        actions = {"cryptoprice_get": self._get_price}
+
+        if action_name in actions:
+            return actions[action_name](**kwargs)
+        else:
+            raise ValueError(f"Unknown action: {action_name}")
+
+    def _get_price(self, symbol, currency):
+        """
+        Fetches the current price of a given cryptocurrency symbol in the specified currency.
+        Example:
+            symbol = "BTC"
+            currency = "USD"
+            returns price in USD.
+        """
+        url = f"https://min-api.cryptocompare.com/data/price?fsym={symbol.upper()}&tsyms={currency.upper()}"
+        response = requests.get(url)
+        if response.status_code == 200:
+            data = response.json()
+            # data will be like {"USD": <price>} if the call is successful
+            if currency.upper() in data:
+                return {
+                    "status_code": response.status_code,
+                    "price": data[currency.upper()],
+                    "message": f"Price of {symbol.upper()} in {currency.upper()} retrieved successfully.",
+                }
+            else:
+                return {
+                    "status_code": response.status_code,
+                    "message": f"Couldn't find price for {symbol.upper()} in {currency.upper()}.",
+                }
+        else:
+            return {
+                "status_code": response.status_code,
+                "message": "Failed to retrieve price.",
+            }
+
+    def get_actions_metadata(self):
+        return [
+            {
+                "name": "cryptoprice_get",
+                "description": "Retrieve the price of a specified cryptocurrency in a given currency",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "symbol": {
+                            "type": "string",
+                            "description": "The cryptocurrency symbol (e.g. BTC)",
+                        },
+                        "currency": {
+                            "type": "string",
+                            "description": "The currency in which you want the price (e.g. USD)",
+                        },
+                    },
+                    "required": ["symbol", "currency"],
+                    "additionalProperties": False,
+                },
+            }
+        ]
+
+    def get_config_requirements(self):
+        # No specific configuration needed for this tool as it just queries a public endpoint
+        return {}

application/tools/implementations/postgres.py

@@ -0,0 +1,163 @@
+import psycopg2
+from application.tools.base import Tool
+
+class PostgresTool(Tool):
+    """
+    PostgreSQL Database Tool
+    A tool for connecting to a PostgreSQL database using a connection string,
+    executing SQL queries, and retrieving schema information.
+    """
+
+    def __init__(self, config):
+        self.config = config
+        self.connection_string = config.get("token", "")
+
+    def execute_action(self, action_name, **kwargs):
+        actions = {
+            "postgres_execute_sql": self._execute_sql,
+            "postgres_get_schema": self._get_schema,
+        }
+
+        if action_name in actions:
+            return actions[action_name](**kwargs)
+        else:
+            raise ValueError(f"Unknown action: {action_name}")
+
+    def _execute_sql(self, sql_query):
+        """
+        Executes an SQL query against the PostgreSQL database using a connection string.
+        """
+        conn = None  # Initialize conn to None for error handling
+        try:
+            conn = psycopg2.connect(self.connection_string)
+            cur = conn.cursor()
+            cur.execute(sql_query)
+            conn.commit()
+
+            if sql_query.strip().lower().startswith("select"):
+                column_names = [desc[0] for desc in cur.description] if cur.description else []
+                results = []
+                rows = cur.fetchall()
+                for row in rows:
+                    results.append(dict(zip(column_names, row)))
+                response_data = {"data": results, "column_names": column_names}
+            else:
+                row_count = cur.rowcount
+                response_data = {"message": f"Query executed successfully, {row_count} rows affected."}
+
+            cur.close()
+            return {
+                "status_code": 200,
+                "message": "SQL query executed successfully.",
+                "response_data": response_data,
+            }
+
+        except psycopg2.Error as e:
+            error_message = f"Database error: {e}"
+            print(f"Database error: {e}")
+            return {
+                "status_code": 500,
+                "message": "Failed to execute SQL query.",
+                "error": error_message,
+            }
+        finally:
+            if conn:  # Ensure connection is closed even if errors occur
+                conn.close()
+
+    def _get_schema(self, db_name):
+        """
+        Retrieves the schema of the PostgreSQL database using a connection string.
+        """
+        conn = None # Initialize conn to None for error handling
+        try:
+            conn = psycopg2.connect(self.connection_string)
+            cur = conn.cursor()
+
+            cur.execute("""
+                SELECT
+                    table_name,
+                    column_name,
+                    data_type,
+                    column_default,
+                    is_nullable
+                FROM
+                    information_schema.columns
+                WHERE
+                    table_schema = 'public'
+                ORDER BY
+                    table_name,
+                    ordinal_position;
+            """)
+
+            schema_data = {}
+            for row in cur.fetchall():
+                table_name, column_name, data_type, column_default, is_nullable = row
+                if table_name not in schema_data:
+                    schema_data[table_name] = []
+                schema_data[table_name].append({
+                    "column_name": column_name,
+                    "data_type": data_type,
+                    "column_default": column_default,
+                    "is_nullable": is_nullable
+                })
+
+            cur.close()
+            return {
+                "status_code": 200,
+                "message": "Database schema retrieved successfully.",
+                "schema": schema_data,
+            }
+
+        except psycopg2.Error as e:
+            error_message = f"Database error: {e}"
+            print(f"Database error: {e}")
+            return {
+                "status_code": 500,
+                "message": "Failed to retrieve database schema.",
+                "error": error_message,
+            }
+        finally:
+            if conn: # Ensure connection is closed even if errors occur
+                conn.close()
+
+    def get_actions_metadata(self):
+        return [
+            {
+                "name": "postgres_execute_sql",
+                "description": "Execute an SQL query against the PostgreSQL database and return the results. Use this tool to interact with the database, e.g., retrieve specific data or perform updates. Only SELECT queries will return data, other queries will return execution status.",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "sql_query": {
+                            "type": "string",
+                            "description": "The SQL query to execute.",
+                        },
+                    },
+                    "required": ["sql_query"],
+                    "additionalProperties": False,
+                },
+            },
+            {
+                "name": "postgres_get_schema",
+                "description": "Retrieve the schema of the PostgreSQL database, including tables and their columns. Use this to understand the database structure before executing queries. db_name is 'default' if not provided.",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "db_name": {
+                            "type": "string",
+                            "description": "The name of the database to retrieve the schema for.",
+                        },
+                    },
+                    "required": ["db_name"],
+                    "additionalProperties": False,
+                },
+            },
+        ]
+
+    def get_config_requirements(self):
+        return {
+            "token": {
+                "type": "string",
+                "description": "PostgreSQL database connection string (e.g., 'postgresql://user:password@host:port/dbname')",
+            },
+        }

application/tools/implementations/telegram.py

@@ -0,0 +1,86 @@
+import requests
+from application.tools.base import Tool
+
+
+class TelegramTool(Tool):
+    """
+    Telegram Bot
+    A flexible Telegram tool for performing various actions (e.g., sending messages, images).
+    Requires a bot token and chat ID for configuration
+    """
+
+    def __init__(self, config):
+        self.config = config
+        self.token = config.get("token", "")
+
+    def execute_action(self, action_name, **kwargs):
+        actions = {
+            "telegram_send_message": self._send_message,
+            "telegram_send_image": self._send_image,
+        }
+
+        if action_name in actions:
+            return actions[action_name](**kwargs)
+        else:
+            raise ValueError(f"Unknown action: {action_name}")
+
+    def _send_message(self, text, chat_id):
+        print(f"Sending message: {text}")
+        url = f"https://api.telegram.org/bot{self.token}/sendMessage"
+        payload = {"chat_id": chat_id, "text": text}
+        response = requests.post(url, data=payload)
+        return {"status_code": response.status_code, "message": "Message sent"}
+
+    def _send_image(self, image_url, chat_id):
+        print(f"Sending image: {image_url}")
+        url = f"https://api.telegram.org/bot{self.token}/sendPhoto"
+        payload = {"chat_id": chat_id, "photo": image_url}
+        response = requests.post(url, data=payload)
+        return {"status_code": response.status_code, "message": "Image sent"}
+
+    def get_actions_metadata(self):
+        return [
+            {
+                "name": "telegram_send_message",
+                "description": "Send a notification to Telegram chat",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "text": {
+                            "type": "string",
+                            "description": "Text to send in the notification",
+                        },
+                        "chat_id": {
+                            "type": "string",
+                            "description": "Chat ID to send the notification to",
+                        },
+                    },
+                    "required": ["text"],
+                    "additionalProperties": False,
+                },
+            },
+            {
+                "name": "telegram_send_image",
+                "description": "Send an image to the Telegram chat",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "image_url": {
+                            "type": "string",
+                            "description": "URL of the image to send",
+                        },
+                        "chat_id": {
+                            "type": "string",
+                            "description": "Chat ID to send the image to",
+                        },
+                    },
+                    "required": ["image_url"],
+                    "additionalProperties": False,
+                },
+            },
+        ]
+
+    def get_config_requirements(self):
+        return {
+            "token": {"type": "string", "description": "Bot token for authentication"},
+        }

application/tools/llm_handler.py

@@ -0,0 +1,97 @@
+import json
+from abc import ABC, abstractmethod
+
+
+class LLMHandler(ABC):
+    @abstractmethod
+    def handle_response(self, agent, resp, tools_dict, messages, **kwargs):
+        pass
+
+
+class OpenAILLMHandler(LLMHandler):
+    def handle_response(self, agent, resp, tools_dict, messages):
+        while resp.finish_reason == "tool_calls":
+            message = json.loads(resp.model_dump_json())["message"]
+            keys_to_remove = {"audio", "function_call", "refusal"}
+            filtered_data = {
+                k: v for k, v in message.items() if k not in keys_to_remove
+            }
+            messages.append(filtered_data)
+
+            tool_calls = resp.message.tool_calls
+            for call in tool_calls:
+                try:
+                    tool_response, call_id = agent._execute_tool_action(
+                        tools_dict, call
+                    )
+                    messages.append(
+                        {
+                            "role": "tool",
+                            "content": str(tool_response),
+                            "tool_call_id": call_id,
+                        }
+                    )
+                except Exception as e:
+                    messages.append(
+                        {
+                            "role": "tool",
+                            "content": f"Error executing tool: {str(e)}",
+                            "tool_call_id": call_id,
+                        }
+                    )
+            resp = agent.llm.gen(
+                model=agent.gpt_model, messages=messages, tools=agent.tools
+            )
+        return resp
+
+
+class GoogleLLMHandler(LLMHandler):
+    def handle_response(self, agent, resp, tools_dict, messages):
+        from google.genai import types
+
+        while True:
+            response = agent.llm.gen(
+                model=agent.gpt_model, messages=messages, tools=agent.tools
+            )
+            if response.candidates and response.candidates[0].content.parts:
+                tool_call_found = False
+                for part in response.candidates[0].content.parts:
+                    if part.function_call:
+                        tool_call_found = True
+                        tool_response, call_id = agent._execute_tool_action(
+                            tools_dict, part.function_call
+                        )
+                        function_response_part = types.Part.from_function_response(
+                            name=part.function_call.name,
+                            response={"result": tool_response},
+                        )
+
+                        messages.append(
+                            {"role": "model", "content": [part.to_json_dict()]}
+                        )
+                        messages.append(
+                            {
+                                "role": "tool",
+                                "content": [function_response_part.to_json_dict()],
+                            }
+                        )
+
+                if (
+                    not tool_call_found
+                    and response.candidates[0].content.parts
+                    and response.candidates[0].content.parts[0].text
+                ):
+                    return response.candidates[0].content.parts[0].text
+                elif not tool_call_found:
+                    return response.candidates[0].content.parts
+
+            else:
+                return response
+
+
+def get_llm_handler(llm_type):
+    handlers = {
+        "openai": OpenAILLMHandler(),
+        "google": GoogleLLMHandler(),
+    }
+    return handlers.get(llm_type, OpenAILLMHandler())

application/tools/tool_action_parser.py

@@ -0,0 +1,26 @@
+import json
+
+
+class ToolActionParser:
+    def __init__(self, llm_type):
+        self.llm_type = llm_type
+        self.parsers = {
+            "OpenAILLM": self._parse_openai_llm,
+            "GoogleLLM": self._parse_google_llm,
+        }
+
+    def parse_args(self, call):
+        parser = self.parsers.get(self.llm_type, self._parse_openai_llm)
+        return parser(call)
+
+    def _parse_openai_llm(self, call):
+        call_args = json.loads(call.function.arguments)
+        tool_id = call.function.name.split("_")[-1]
+        action_name = call.function.name.rsplit("_", 1)[0]
+        return tool_id, action_name, call_args
+
+    def _parse_google_llm(self, call):
+        call_args = call.args
+        tool_id = call.name.split("_")[-1]
+        action_name = call.name.rsplit("_", 1)[0]
+        return tool_id, action_name, call_args

application/tools/tool_manager.py

@@ -0,0 +1,46 @@
+import importlib
+import inspect
+import os
+import pkgutil
+
+from application.tools.base import Tool
+
+
+class ToolManager:
+    def __init__(self, config):
+        self.config = config
+        self.tools = {}
+        self.load_tools()
+
+    def load_tools(self):
+        tools_dir = os.path.join(os.path.dirname(__file__), "implementations")
+        for finder, name, ispkg in pkgutil.iter_modules([tools_dir]):
+            if name == "base" or name.startswith("__"):
+                continue
+            module = importlib.import_module(
+                f"application.tools.implementations.{name}"
+            )
+            for member_name, obj in inspect.getmembers(module, inspect.isclass):
+                if issubclass(obj, Tool) and obj is not Tool:
+                    tool_config = self.config.get(name, {})
+                    self.tools[name] = obj(tool_config)
+
+    def load_tool(self, tool_name, tool_config):
+        self.config[tool_name] = tool_config
+        module = importlib.import_module(
+            f"application.tools.implementations.{tool_name}"
+        )
+        for member_name, obj in inspect.getmembers(module, inspect.isclass):
+            if issubclass(obj, Tool) and obj is not Tool:
+                return obj(tool_config)
+
+    def execute_action(self, tool_name, action_name, **kwargs):
+        if tool_name not in self.tools:
+            raise ValueError(f"Tool '{tool_name}' not loaded")
+        return self.tools[tool_name].execute_action(action_name, **kwargs)
+
+    def get_all_actions_metadata(self):
+        metadata = []
+        for tool in self.tools.values():
+            metadata.extend(tool.get_actions_metadata())
+        return metadata

application/tts/elevenlabs.py

@@ -1,29 +1,84 @@
-from io import BytesIO
+import asyncio
+import websockets
+import json
 import base64
+from io import BytesIO
 from application.tts.base import BaseTTS
 
 
 class ElevenlabsTTS(BaseTTS):
-    def __init__(self):
-        from elevenlabs.client import ElevenLabs
-
-        self.client = ElevenLabs(
-            api_key="ELEVENLABS_API_KEY",
-            )
-    
+    def __init__(self):        
+        self.api_key = 'ELEVENLABS_API_KEY'# here you should put your api key
+        self.model = "eleven_flash_v2_5"
+        self.voice = "VOICE_ID" # this is the hash code for the voice not the name!
+        self.write_audio = 1
 
     def text_to_speech(self, text):
-        lang = "en"
-        audio = self.client.generate(
-            text=text,
-            model="eleven_multilingual_v2",
-            voice="Brian",
-        )
-        audio_data = BytesIO()
-        for chunk in audio:
-            audio_data.write(chunk)
-        audio_bytes = audio_data.getvalue()
+        asyncio.run(self._text_to_speech_websocket(text))
 
-        # Encode to base64
-        audio_base64 = base64.b64encode(audio_bytes).decode("utf-8")
-        return audio_base64, lang
+    async def _text_to_speech_websocket(self, text):
+        uri = f"wss://api.elevenlabs.io/v1/text-to-speech/{self.voice}/stream-input?model_id={self.model}"
+        websocket = await websockets.connect(uri)
+        payload = {
+            "text": " ",
+            "voice_settings": {
+                "stability": 0.5,
+                "similarity_boost": 0.8,
+            },
+            "xi_api_key": self.api_key,
+        }
+
+        await websocket.send(json.dumps(payload))
+        
+        async def listen():
+            while 1:
+                try:
+                    msg = await websocket.recv()
+                    data = json.loads(msg)
+
+                    if data.get("audio"):
+                        print("audio received")
+                        yield base64.b64decode(data["audio"])
+                    elif data.get("isFinal"):
+                        break
+                except websockets.exceptions.ConnectionClosed:
+                    print("websocket closed")
+                    break
+        listen_task =  asyncio.create_task(self.stream(listen()))
+        
+        await websocket.send(json.dumps({"text": text}))
+        # this is to signal the end of the text, either use this or flush
+        await websocket.send(json.dumps({"text": ""})) 
+
+        await listen_task
+    
+    async def stream(self, audio_stream):
+        if self.write_audio:
+            audio_bytes = BytesIO()
+            async for chunk in audio_stream:
+                if chunk:
+                    audio_bytes.write(chunk)
+            with open("output_audio.mp3", "wb") as f:
+                f.write(audio_bytes.getvalue())
+        
+        else:
+            async for chunk in audio_stream:
+                pass # depends on the streamer!
+
+
+def test_elevenlabs_websocket():
+    """
+    Tests the ElevenlabsTTS text_to_speech method with a sample prompt.
+    Prints out the base64-encoded result and writes it to 'output_audio.mp3'.
+    """
+    # Instantiate your TTS class
+    tts = ElevenlabsTTS()
+
+    # Call the method with some sample text
+    tts.text_to_speech("Hello from ElevenLabs WebSocket!")
+
+    print("Saved audio to output_audio.mp3.")
+
+
+if __name__ == "__main__":
+    test_elevenlabs_websocket()

application/usage.py

@@ -1,7 +1,7 @@
 import sys
 from datetime import datetime
 from application.core.mongo_db import MongoDB
-from application.utils import num_tokens_from_string
+from application.utils import num_tokens_from_string, num_tokens_from_object_or_list
 
 mongo = MongoDB.get_client()
 db = mongo["docsgpt"]
@@ -21,11 +21,16 @@ def update_token_usage(user_api_key, token_usage):
 
 
 def gen_token_usage(func):
-    def wrapper(self, model, messages, stream, **kwargs):
+    def wrapper(self, model, messages, stream, tools, **kwargs):
         for message in messages:
-            self.token_usage["prompt_tokens"] += num_tokens_from_string(message["content"])
-        result = func(self, model, messages, stream, **kwargs)
-        self.token_usage["generated_tokens"] += num_tokens_from_string(result)
+            if message["content"]:
+                self.token_usage["prompt_tokens"] += num_tokens_from_string(message["content"])
+        result = func(self, model, messages, stream, tools, **kwargs)
+        # check if result is a string
+        if isinstance(result, str):
+            self.token_usage["generated_tokens"] += num_tokens_from_string(result)
+        else:
+            self.token_usage["generated_tokens"] += num_tokens_from_object_or_list(result)
         update_token_usage(self.user_api_key, self.token_usage)
         return result
 
@@ -33,11 +38,11 @@ def gen_token_usage(func):
 
 
 def stream_token_usage(func):
-    def wrapper(self, model, messages, stream, **kwargs):
+    def wrapper(self, model, messages, stream, tools, **kwargs):
         for message in messages:
             self.token_usage["prompt_tokens"] += num_tokens_from_string(message["content"])
         batch = []
-        result = func(self, model, messages, stream, **kwargs)
+        result = func(self, model, messages, stream, tools, **kwargs)
         for r in result:
             batch.append(r)
             yield r

application/utils.py

@@ -15,9 +15,21 @@ def get_encoding():
 
 def num_tokens_from_string(string: str) -> int:
     encoding = get_encoding()
-    num_tokens = len(encoding.encode(string))
-    return num_tokens
+    if isinstance(string, str):
+        num_tokens = len(encoding.encode(string))
+        return num_tokens
+    else:
+        return 0
 
+def num_tokens_from_object_or_list(thing):
+    if isinstance(thing, list):
+        return sum([num_tokens_from_object_or_list(x) for x in thing])
+    elif isinstance(thing, dict):
+        return sum([num_tokens_from_object_or_list(x) for x in thing.values()])
+    elif isinstance(thing, str):
+        return num_tokens_from_string(thing)
+    else:
+        return 0
 
 def count_tokens_docs(docs):
     docs_content = ""
@@ -46,3 +58,40 @@ def check_required_fields(data, required_fields):
 def get_hash(data):
     return hashlib.md5(data.encode()).hexdigest()
 
+def limit_chat_history(history, max_token_limit=None, gpt_model="docsgpt"):
+    """
+    Limits chat history based on token count.
+    Returns a list of messages that fit within the token limit.
+    """
+    from application.core.settings import settings
+
+    max_token_limit = (
+            max_token_limit
+            if max_token_limit and 
+            max_token_limit < settings.MODEL_TOKEN_LIMITS.get(
+                gpt_model, settings.DEFAULT_MAX_HISTORY
+            )
+            else settings.MODEL_TOKEN_LIMITS.get(
+                gpt_model, settings.DEFAULT_MAX_HISTORY
+            )
+        )
+    
+
+    if not history:
+        return []
+        
+    tokens_current_history = 0
+    trimmed_history = []
+    
+    for message in reversed(history):
+        if "prompt" in message and "response" in message:
+            tokens_batch = num_tokens_from_string(message["prompt"]) + num_tokens_from_string(
+                message["response"]
+            )
+            if tokens_current_history + tokens_batch < max_token_limit:
+                tokens_current_history += tokens_batch
+                trimmed_history.insert(0, message)
+            else:
+                break
+
+    return trimmed_history

application/worker.py

@@ -12,10 +12,10 @@ from bson.objectid import ObjectId
 from application.core.mongo_db import MongoDB
 from application.core.settings import settings
 from application.parser.file.bulk import SimpleDirectoryReader
-from application.parser.open_ai_func import call_openai_api
+from application.parser.embedding_pipeline import embed_and_store_documents
 from application.parser.remote.remote_creator import RemoteCreator
 from application.parser.schema.base import Document
-from application.parser.token_func import group_split
+from application.parser.chunking import Chunker
 from application.utils import count_tokens_docs
 
 mongo = MongoDB.get_client()
@@ -126,7 +126,6 @@ def ingest_worker(
     limit = None
     exclude = True
     sample = False
-    token_check = True
     full_path = os.path.join(directory, user, name_job)
 
     logging.info(f"Ingest file: {full_path}", extra={"user": user, "job": name_job})
@@ -153,17 +152,19 @@ def ingest_worker(
         exclude_hidden=exclude,
         file_metadata=metadata_from_filename,
     ).load_data()
-    raw_docs = group_split(
-        documents=raw_docs,
-        min_tokens=MIN_TOKENS,
+
+    chunker = Chunker(
+        chunking_strategy="classic_chunk",
         max_tokens=MAX_TOKENS,
-        token_check=token_check,
+        min_tokens=MIN_TOKENS,
+        duplicate_headers=False
     )
+    raw_docs = chunker.chunk(documents=raw_docs)
 
     docs = [Document.to_langchain_format(raw_doc) for raw_doc in raw_docs]
     id = ObjectId()
 
-    call_openai_api(docs, full_path, id, self)
+    embed_and_store_documents(docs, full_path, id, self)
     tokens = count_tokens_docs(docs)
     self.update_state(state="PROGRESS", meta={"current": 100})
 
@@ -202,52 +203,61 @@ def remote_worker(
     sync_frequency="never",
     operation_mode="upload",
     doc_id=None,
-):
-    token_check = True
+):  
     full_path = os.path.join(directory, user, name_job)
-
     if not os.path.exists(full_path):
         os.makedirs(full_path)
+
     self.update_state(state="PROGRESS", meta={"current": 1})
-    logging.info(
-        f"Remote job: {full_path}",
-        extra={"user": user, "job": name_job, "source_data": source_data},
-    )
+    try:
+        logging.info("Initializing remote loader with type: %s", loader)
+        remote_loader = RemoteCreator.create_loader(loader)
+        raw_docs = remote_loader.load_data(source_data)
 
-    remote_loader = RemoteCreator.create_loader(loader)
-    raw_docs = remote_loader.load_data(source_data)
+        chunker = Chunker(
+            chunking_strategy="classic_chunk",
+            max_tokens=MAX_TOKENS,
+            min_tokens=MIN_TOKENS,
+            duplicate_headers=False
+        )
+        docs = chunker.chunk(documents=raw_docs)
+        docs = [Document.to_langchain_format(raw_doc) for raw_doc in raw_docs]
+        tokens = count_tokens_docs(docs)
+        logging.info("Total tokens calculated: %d", tokens)
 
-    docs = group_split(
-        documents=raw_docs,
-        min_tokens=MIN_TOKENS,
-        max_tokens=MAX_TOKENS,
-        token_check=token_check,
-    )
-    tokens = count_tokens_docs(docs)
-    if operation_mode == "upload":
-        id = ObjectId()
-        call_openai_api(docs, full_path, id, self)
-    elif operation_mode == "sync":
-        if not doc_id or not ObjectId.is_valid(doc_id):
-            raise ValueError("doc_id must be provided for sync operation.")
-        id = ObjectId(doc_id)
-        call_openai_api(docs, full_path, id, self)
-    self.update_state(state="PROGRESS", meta={"current": 100})
+        if operation_mode == "upload":
+            id = ObjectId()
+            embed_and_store_documents(docs, full_path, id, self)
+        elif operation_mode == "sync":
+            if not doc_id or not ObjectId.is_valid(doc_id):
+                logging.error("Invalid doc_id provided for sync operation: %s", doc_id)
+                raise ValueError("doc_id must be provided for sync operation.")
+            id = ObjectId(doc_id)
+            embed_and_store_documents(docs, full_path, id, self)
 
-    file_data = {
-        "name": name_job,
-        "user": user,
-        "tokens": tokens,
-        "retriever": retriever,
-        "id": str(id),
-        "type": loader,
-        "remote_data": source_data,
-        "sync_frequency": sync_frequency,
-    }
-    upload_index(full_path, file_data)
+        self.update_state(state="PROGRESS", meta={"current": 100})
 
-    shutil.rmtree(full_path)
+        file_data = {
+            "name": name_job,
+            "user": user,
+            "tokens": tokens,
+            "retriever": retriever,
+            "id": str(id),
+            "type": loader,
+            "remote_data": source_data,
+            "sync_frequency": sync_frequency,
+        }
+        upload_index(full_path, file_data)
 
+    except Exception as e:
+        logging.error("Error in remote_worker task: %s", str(e), exc_info=True)
+        raise
+
+    finally:
+        if os.path.exists(full_path):
+            shutil.rmtree(full_path)
+
+    logging.info("remote_worker task completed successfully")
     return {"urls": source_data, "name_job": name_job, "user": user, "limited": False}
 
 def sync(

docs/package-lock.json

@@ -7,8 +7,8 @@
       "license": "MIT",
       "dependencies": {
         "@vercel/analytics": "^1.1.1",
-        "docsgpt": "^0.4.7",
-        "next": "^14.2.12",
+        "docsgpt-react": "^0.4.9",
+        "next": "^14.2.22",
         "nextra": "^2.13.2",
         "nextra-theme-docs": "^2.13.2",
         "react": "^18.2.0",
@@ -931,14 +931,14 @@
       }
     },
     "node_modules/@next/env": {
-      "version": "14.2.12",
-      "resolved": "https://registry.npmjs.org/@next/env/-/env-14.2.12.tgz",
-      "integrity": "sha512-3fP29GIetdwVIfIRyLKM7KrvJaqepv+6pVodEbx0P5CaMLYBtx+7eEg8JYO5L9sveJO87z9eCReceZLi0hxO1Q=="
+      "version": "14.2.22",
+      "resolved": "https://registry.npmjs.org/@next/env/-/env-14.2.22.tgz",
+      "integrity": "sha512-EQ6y1QeNQglNmNIXvwP/Bb+lf7n9WtgcWvtoFsHquVLCJUuxRs+6SfZ5EK0/EqkkLex4RrDySvKgKNN7PXip7Q=="
     },
     "node_modules/@next/swc-darwin-arm64": {
-      "version": "14.2.12",
-      "resolved": "https://registry.npmjs.org/@next/swc-darwin-arm64/-/swc-darwin-arm64-14.2.12.tgz",
-      "integrity": "sha512-crHJ9UoinXeFbHYNok6VZqjKnd8rTd7K3Z2zpyzF1ch7vVNKmhjv/V7EHxep3ILoN8JB9AdRn/EtVVyG9AkCXw==",
+      "version": "14.2.22",
+      "resolved": "https://registry.npmjs.org/@next/swc-darwin-arm64/-/swc-darwin-arm64-14.2.22.tgz",
+      "integrity": "sha512-HUaLiehovgnqY4TMBZJ3pDaOsTE1spIXeR10pWgdQVPYqDGQmHJBj3h3V6yC0uuo/RoY2GC0YBFRkOX3dI9WVQ==",
       "cpu": [
         "arm64"
       ],
@@ -951,9 +951,9 @@
       }
     },
     "node_modules/@next/swc-darwin-x64": {
-      "version": "14.2.12",
-      "resolved": "https://registry.npmjs.org/@next/swc-darwin-x64/-/swc-darwin-x64-14.2.12.tgz",
-      "integrity": "sha512-JbEaGbWq18BuNBO+lCtKfxl563Uw9oy2TodnN2ioX00u7V1uzrsSUcg3Ep9ce+P0Z9es+JmsvL2/rLphz+Frcw==",
+      "version": "14.2.22",
+      "resolved": "https://registry.npmjs.org/@next/swc-darwin-x64/-/swc-darwin-x64-14.2.22.tgz",
+      "integrity": "sha512-ApVDANousaAGrosWvxoGdLT0uvLBUC+srqOcpXuyfglA40cP2LBFaGmBjhgpxYk5z4xmunzqQvcIgXawTzo2uQ==",
       "cpu": [
         "x64"
       ],
@@ -966,9 +966,9 @@
       }
     },
     "node_modules/@next/swc-linux-arm64-gnu": {
-      "version": "14.2.12",
-      "resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-gnu/-/swc-linux-arm64-gnu-14.2.12.tgz",
-      "integrity": "sha512-qBy7OiXOqZrdp88QEl2H4fWalMGnSCrr1agT/AVDndlyw2YJQA89f3ttR/AkEIP9EkBXXeGl6cC72/EZT5r6rw==",
+      "version": "14.2.22",
+      "resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-gnu/-/swc-linux-arm64-gnu-14.2.22.tgz",
+      "integrity": "sha512-3O2J99Bk9aM+d4CGn9eEayJXHuH9QLx0BctvWyuUGtJ3/mH6lkfAPRI4FidmHMBQBB4UcvLMfNf8vF0NZT7iKw==",
       "cpu": [
         "arm64"
       ],
@@ -981,9 +981,9 @@
       }
     },
     "node_modules/@next/swc-linux-arm64-musl": {
-      "version": "14.2.12",
-      "resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-musl/-/swc-linux-arm64-musl-14.2.12.tgz",
-      "integrity": "sha512-EfD9L7o9biaQxjwP1uWXnk3vYZi64NVcKUN83hpVkKocB7ogJfyH2r7o1pPnMtir6gHZiGCeHKagJ0yrNSLNHw==",
+      "version": "14.2.22",
+      "resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-musl/-/swc-linux-arm64-musl-14.2.22.tgz",
+      "integrity": "sha512-H/hqfRz75yy60y5Eg7DxYfbmHMjv60Dsa6IWHzpJSz4MRkZNy5eDnEW9wyts9bkxwbOVZNPHeb3NkqanP+nGPg==",
       "cpu": [
         "arm64"
       ],
@@ -996,9 +996,9 @@
       }
     },
     "node_modules/@next/swc-linux-x64-gnu": {
-      "version": "14.2.12",
-      "resolved": "https://registry.npmjs.org/@next/swc-linux-x64-gnu/-/swc-linux-x64-gnu-14.2.12.tgz",
-      "integrity": "sha512-iQ+n2pxklJew9IpE47hE/VgjmljlHqtcD5UhZVeHICTPbLyrgPehaKf2wLRNjYH75udroBNCgrSSVSVpAbNoYw==",
+      "version": "14.2.22",
+      "resolved": "https://registry.npmjs.org/@next/swc-linux-x64-gnu/-/swc-linux-x64-gnu-14.2.22.tgz",
+      "integrity": "sha512-LckLwlCLcGR1hlI5eiJymR8zSHPsuruuwaZ3H2uudr25+Dpzo6cRFjp/3OR5UYJt8LSwlXv9mmY4oI2QynwpqQ==",
       "cpu": [
         "x64"
       ],
@@ -1011,9 +1011,9 @@
       }
     },
     "node_modules/@next/swc-linux-x64-musl": {
-      "version": "14.2.12",
-      "resolved": "https://registry.npmjs.org/@next/swc-linux-x64-musl/-/swc-linux-x64-musl-14.2.12.tgz",
-      "integrity": "sha512-rFkUkNwcQ0ODn7cxvcVdpHlcOpYxMeyMfkJuzaT74xjAa5v4fxP4xDk5OoYmPi8QNLDs3UgZPMSBmpBuv9zKWA==",
+      "version": "14.2.22",
+      "resolved": "https://registry.npmjs.org/@next/swc-linux-x64-musl/-/swc-linux-x64-musl-14.2.22.tgz",
+      "integrity": "sha512-qGUutzmh0PoFU0fCSu0XYpOfT7ydBZgDfcETIeft46abPqP+dmePhwRGLhFKwZWxNWQCPprH26TjaTxM0Nv8mw==",
       "cpu": [
         "x64"
       ],
@@ -1026,9 +1026,9 @@
       }
     },
     "node_modules/@next/swc-win32-arm64-msvc": {
-      "version": "14.2.12",
-      "resolved": "https://registry.npmjs.org/@next/swc-win32-arm64-msvc/-/swc-win32-arm64-msvc-14.2.12.tgz",
-      "integrity": "sha512-PQFYUvwtHs/u0K85SG4sAdDXYIPXpETf9mcEjWc0R4JmjgMKSDwIU/qfZdavtP6MPNiMjuKGXHCtyhR/M5zo8g==",
+      "version": "14.2.22",
+      "resolved": "https://registry.npmjs.org/@next/swc-win32-arm64-msvc/-/swc-win32-arm64-msvc-14.2.22.tgz",
+      "integrity": "sha512-K6MwucMWmIvMb9GlvT0haYsfIPxfQD8yXqxwFy4uLFMeXIb2TcVYQimxkaFZv86I7sn1NOZnpOaVk5eaxThGIw==",
       "cpu": [
         "arm64"
       ],
@@ -1041,9 +1041,9 @@
       }
     },
     "node_modules/@next/swc-win32-ia32-msvc": {
-      "version": "14.2.12",
-      "resolved": "https://registry.npmjs.org/@next/swc-win32-ia32-msvc/-/swc-win32-ia32-msvc-14.2.12.tgz",
-      "integrity": "sha512-FAj2hMlcbeCV546eU2tEv41dcJb4NeqFlSXU/xL/0ehXywHnNpaYajOUvn3P8wru5WyQe6cTZ8fvckj/2XN4Vw==",
+      "version": "14.2.22",
+      "resolved": "https://registry.npmjs.org/@next/swc-win32-ia32-msvc/-/swc-win32-ia32-msvc-14.2.22.tgz",
+      "integrity": "sha512-5IhDDTPEbzPR31ZzqHe90LnNe7BlJUZvC4sA1thPJV6oN5WmtWjZ0bOYfNsyZx00FJt7gggNs6SrsX0UEIcIpA==",
       "cpu": [
         "ia32"
       ],
@@ -1056,9 +1056,9 @@
       }
     },
     "node_modules/@next/swc-win32-x64-msvc": {
-      "version": "14.2.12",
-      "resolved": "https://registry.npmjs.org/@next/swc-win32-x64-msvc/-/swc-win32-x64-msvc-14.2.12.tgz",
-      "integrity": "sha512-yu8QvV53sBzoIVRHsxCHqeuS8jYq6Lrmdh0briivuh+Brsp6xjg80MAozUsBTAV9KNmY08KlX0KYTWz1lbPzEg==",
+      "version": "14.2.22",
+      "resolved": "https://registry.npmjs.org/@next/swc-win32-x64-msvc/-/swc-win32-x64-msvc-14.2.22.tgz",
+      "integrity": "sha512-nvRaB1PyG4scn9/qNzlkwEwLzuoPH3Gjp7Q/pLuwUgOTt1oPMlnCI3A3rgkt+eZnU71emOiEv/mR201HoURPGg==",
       "cpu": [
         "x64"
       ],
@@ -1170,6 +1170,407 @@
         "node": ">=8"
       }
     },
+    "node_modules/@parcel/core": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/core/-/core-2.13.2.tgz",
+      "integrity": "sha512-1zC5Au4z9or5XyP6ipfvJqHktuB0jD7WuxMcV1CWAZGARHKylLe+0ccl+Wx7HN5O+xAvfCDtTlKrATY8qyrIyw==",
+      "peer": true,
+      "dependencies": {
+        "@mischnic/json-sourcemap": "^0.1.0",
+        "@parcel/cache": "2.13.2",
+        "@parcel/diagnostic": "2.13.2",
+        "@parcel/events": "2.13.2",
+        "@parcel/feature-flags": "2.13.2",
+        "@parcel/fs": "2.13.2",
+        "@parcel/graph": "3.3.2",
+        "@parcel/logger": "2.13.2",
+        "@parcel/package-manager": "2.13.2",
+        "@parcel/plugin": "2.13.2",
+        "@parcel/profiler": "2.13.2",
+        "@parcel/rust": "2.13.2",
+        "@parcel/source-map": "^2.1.1",
+        "@parcel/types": "2.13.2",
+        "@parcel/utils": "2.13.2",
+        "@parcel/workers": "2.13.2",
+        "base-x": "^3.0.8",
+        "browserslist": "^4.6.6",
+        "clone": "^2.1.1",
+        "dotenv": "^16.4.5",
+        "dotenv-expand": "^11.0.6",
+        "json5": "^2.2.0",
+        "msgpackr": "^1.9.9",
+        "nullthrows": "^1.1.1",
+        "semver": "^7.5.2"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/cache": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/cache/-/cache-2.13.2.tgz",
+      "integrity": "sha512-Y0nWlCMWDSp1lxiPI5zCWTGD0InnVZ+IfqeyLWmROAqValYyd0QZCvnSljKJ144jWTr0jXxDveir+DVF8sAYaA==",
+      "peer": true,
+      "dependencies": {
+        "@parcel/fs": "2.13.2",
+        "@parcel/logger": "2.13.2",
+        "@parcel/utils": "2.13.2",
+        "lmdb": "2.8.5"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      },
+      "peerDependencies": {
+        "@parcel/core": "^2.13.2"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/codeframe": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/codeframe/-/codeframe-2.13.2.tgz",
+      "integrity": "sha512-qFMiS14orb6QSQj5/J/QN+gJElUfedVAKBTNkp9QB4i8ObdLHDqHRUzFb55ZQJI3G4vsxOOWAOUXGirtLwrxGQ==",
+      "peer": true,
+      "dependencies": {
+        "chalk": "^4.1.2"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/diagnostic": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/diagnostic/-/diagnostic-2.13.2.tgz",
+      "integrity": "sha512-6Au0JEJ5SY2gYrY0/m0i0sTuqTvK0k2E9azhBJR+zzCREbUxLiDdLZ+vXAfLW7t/kPAcWtdNU0Bj7pnZcMiMXg==",
+      "peer": true,
+      "dependencies": {
+        "@mischnic/json-sourcemap": "^0.1.0",
+        "nullthrows": "^1.1.1"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/events": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/events/-/events-2.13.2.tgz",
+      "integrity": "sha512-BVB9hW1RGh/tMaDHfpa+uIgz5PMULorCnjmWr/KvrlhdUSUQoaPYfRcTDYrKhoKuNIKsWSnTGvXrxE53L5qo0w==",
+      "peer": true,
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/fs": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/fs/-/fs-2.13.2.tgz",
+      "integrity": "sha512-bdeIMuAXhMnROvqV55JWRUmjD438/T7h3r3NsFnkq+Mp4z2nuAn0STxbqDNxIgTMJHNunSDzncqRNMT7xJCe8A==",
+      "peer": true,
+      "dependencies": {
+        "@parcel/feature-flags": "2.13.2",
+        "@parcel/rust": "2.13.2",
+        "@parcel/types-internal": "2.13.2",
+        "@parcel/utils": "2.13.2",
+        "@parcel/watcher": "^2.0.7",
+        "@parcel/workers": "2.13.2"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      },
+      "peerDependencies": {
+        "@parcel/core": "^2.13.2"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/logger": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/logger/-/logger-2.13.2.tgz",
+      "integrity": "sha512-SFVABAMqaT9jIDn4maPgaQQauPDz8fpoKUGEuLF44Q0aQFbBUy7vX7KYs/EvYSWZo4VyJcUDHvIInBlepA0/ZQ==",
+      "peer": true,
+      "dependencies": {
+        "@parcel/diagnostic": "2.13.2",
+        "@parcel/events": "2.13.2"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/markdown-ansi": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/markdown-ansi/-/markdown-ansi-2.13.2.tgz",
+      "integrity": "sha512-MIEoetfT/snk1GqWzBI3AhifV257i2xke9dvyQl14PPiMl+TlVhwnbQyA09WJBvDor+MuxZypHL7xoFdW8ff3A==",
+      "peer": true,
+      "dependencies": {
+        "chalk": "^4.1.2"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/node-resolver-core": {
+      "version": "3.4.2",
+      "resolved": "https://registry.npmjs.org/@parcel/node-resolver-core/-/node-resolver-core-3.4.2.tgz",
+      "integrity": "sha512-SwnKLcZRG1VdB5JeM/Ax5VMWWh2QfXufmMQCKKx0/Kk41nUpie+aIZKj3LH6Z/fJsnKig/vXpeWoxGhmG523qg==",
+      "peer": true,
+      "dependencies": {
+        "@mischnic/json-sourcemap": "^0.1.0",
+        "@parcel/diagnostic": "2.13.2",
+        "@parcel/fs": "2.13.2",
+        "@parcel/rust": "2.13.2",
+        "@parcel/utils": "2.13.2",
+        "nullthrows": "^1.1.1",
+        "semver": "^7.5.2"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/package-manager": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/package-manager/-/package-manager-2.13.2.tgz",
+      "integrity": "sha512-6HjfbdJUjHyNKzYB7GSYnOCtLwqCGW7yT95GlnnTKyFffvXYsqvBSyepMuPRlbX0mFUm4S9l2DH3OVZrk108AA==",
+      "peer": true,
+      "dependencies": {
+        "@parcel/diagnostic": "2.13.2",
+        "@parcel/fs": "2.13.2",
+        "@parcel/logger": "2.13.2",
+        "@parcel/node-resolver-core": "3.4.2",
+        "@parcel/types": "2.13.2",
+        "@parcel/utils": "2.13.2",
+        "@parcel/workers": "2.13.2",
+        "@swc/core": "^1.7.26",
+        "semver": "^7.5.2"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      },
+      "peerDependencies": {
+        "@parcel/core": "^2.13.2"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/plugin": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/plugin/-/plugin-2.13.2.tgz",
+      "integrity": "sha512-Q+RIENS1B185yLPhrGdzBK1oJrZmh/RXrYMnzJs78Tog8SpihjeNBNR6z4PT85o2F+Gy2y1S9A26fpiGq161qQ==",
+      "peer": true,
+      "dependencies": {
+        "@parcel/types": "2.13.2"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/profiler": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/profiler/-/profiler-2.13.2.tgz",
+      "integrity": "sha512-fur6Oq2HkX6AiM8rtqmDvldH5JWz0sqXA1ylz8cE3XOiDZIuvCulZmQ+hH+4odaNH6QocI1MwfV+GDh3HlQoCA==",
+      "peer": true,
+      "dependencies": {
+        "@parcel/diagnostic": "2.13.2",
+        "@parcel/events": "2.13.2",
+        "@parcel/types-internal": "2.13.2",
+        "chrome-trace-event": "^1.0.2"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/rust": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/rust/-/rust-2.13.2.tgz",
+      "integrity": "sha512-XFIewSwxkrDYOnnSP/XZ1LDLdXTs7L9CjQUWtl46Vir5Pq/rinemwLJeKGIwKLHy7fhUZQjYxquH6fBL+AY8DA==",
+      "peer": true,
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/types": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/types/-/types-2.13.2.tgz",
+      "integrity": "sha512-6ixqjk2pjKELn4sQ/jdvpbCVTeH6xXQTdotkN8Wzk68F2K2MtSPIRAEocumlexScfffbRQplr2MdIf1JJWLogA==",
+      "peer": true,
+      "dependencies": {
+        "@parcel/types-internal": "2.13.2",
+        "@parcel/workers": "2.13.2"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/utils": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/utils/-/utils-2.13.2.tgz",
+      "integrity": "sha512-BkFtRo5xenmonwnBy+X4sVbHIRrx+ZHMPpS/6hFqyTvoUUFq2yTFQnfRGVVOOvscVUxpGom+kewnrTG3HHbZoA==",
+      "peer": true,
+      "dependencies": {
+        "@parcel/codeframe": "2.13.2",
+        "@parcel/diagnostic": "2.13.2",
+        "@parcel/logger": "2.13.2",
+        "@parcel/markdown-ansi": "2.13.2",
+        "@parcel/rust": "2.13.2",
+        "@parcel/source-map": "^2.1.1",
+        "chalk": "^4.1.2",
+        "nullthrows": "^1.1.1"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/@parcel/workers": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/workers/-/workers-2.13.2.tgz",
+      "integrity": "sha512-P78BpH0yTT9KK09wgK4eabtlb5OlcWAmZebOToN5UYuwWEylKt0gWZx1+d+LPQupvK84/iZ+AutDScsATjgUMw==",
+      "peer": true,
+      "dependencies": {
+        "@parcel/diagnostic": "2.13.2",
+        "@parcel/logger": "2.13.2",
+        "@parcel/profiler": "2.13.2",
+        "@parcel/types-internal": "2.13.2",
+        "@parcel/utils": "2.13.2",
+        "nullthrows": "^1.1.1"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      },
+      "peerDependencies": {
+        "@parcel/core": "^2.13.2"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/ansi-styles": {
+      "version": "4.3.0",
+      "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz",
+      "integrity": "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==",
+      "peer": true,
+      "dependencies": {
+        "color-convert": "^2.0.1"
+      },
+      "engines": {
+        "node": ">=8"
+      },
+      "funding": {
+        "url": "https://github.com/chalk/ansi-styles?sponsor=1"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/chalk": {
+      "version": "4.1.2",
+      "resolved": "https://registry.npmjs.org/chalk/-/chalk-4.1.2.tgz",
+      "integrity": "sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA==",
+      "peer": true,
+      "dependencies": {
+        "ansi-styles": "^4.1.0",
+        "supports-color": "^7.1.0"
+      },
+      "engines": {
+        "node": ">=10"
+      },
+      "funding": {
+        "url": "https://github.com/chalk/chalk?sponsor=1"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/color-convert": {
+      "version": "2.0.1",
+      "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz",
+      "integrity": "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==",
+      "peer": true,
+      "dependencies": {
+        "color-name": "~1.1.4"
+      },
+      "engines": {
+        "node": ">=7.0.0"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/color-name": {
+      "version": "1.1.4",
+      "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz",
+      "integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==",
+      "peer": true
+    },
+    "node_modules/@parcel/core/node_modules/has-flag": {
+      "version": "4.0.0",
+      "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz",
+      "integrity": "sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ==",
+      "peer": true,
+      "engines": {
+        "node": ">=8"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/semver": {
+      "version": "7.6.3",
+      "resolved": "https://registry.npmjs.org/semver/-/semver-7.6.3.tgz",
+      "integrity": "sha512-oVekP1cKtI+CTDvHWYFUcMtsK/00wmAEfyqKfNdARm8u1wNVhSgaX7A8d4UuIlUI5e84iEwOhs7ZPYRmzU9U6A==",
+      "peer": true,
+      "bin": {
+        "semver": "bin/semver.js"
+      },
+      "engines": {
+        "node": ">=10"
+      }
+    },
+    "node_modules/@parcel/core/node_modules/supports-color": {
+      "version": "7.2.0",
+      "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-7.2.0.tgz",
+      "integrity": "sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw==",
+      "peer": true,
+      "dependencies": {
+        "has-flag": "^4.0.0"
+      },
+      "engines": {
+        "node": ">=8"
+      }
+    },
     "node_modules/@parcel/diagnostic": {
       "version": "2.12.0",
       "resolved": "https://registry.npmjs.org/@parcel/diagnostic/-/diagnostic-2.12.0.tgz",
@@ -1198,6 +1599,19 @@
         "url": "https://opencollective.com/parcel"
       }
     },
+    "node_modules/@parcel/feature-flags": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/feature-flags/-/feature-flags-2.13.2.tgz",
+      "integrity": "sha512-cCwDAKD4Er24EkuQ+loVZXSURpM0gAGRsLJVoBtFiCSbB3nmIJJ6FLRwSBI/5OsOUExiUXDvSpfUCA5ldGTzbw==",
+      "peer": true,
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      }
+    },
     "node_modules/@parcel/fs": {
       "version": "2.12.0",
       "resolved": "https://registry.npmjs.org/@parcel/fs/-/fs-2.12.0.tgz",
@@ -1220,6 +1634,23 @@
         "@parcel/core": "^2.12.0"
       }
     },
+    "node_modules/@parcel/graph": {
+      "version": "3.3.2",
+      "resolved": "https://registry.npmjs.org/@parcel/graph/-/graph-3.3.2.tgz",
+      "integrity": "sha512-aAysQLRr8SOonSHWqdKHMJzfcrDFXKK8IYZEurlOzosiSgZXrAK7q8b8JcaJ4r84/jlvQYNYneNZeFQxKjHXkA==",
+      "peer": true,
+      "dependencies": {
+        "@parcel/feature-flags": "2.13.2",
+        "nullthrows": "^1.1.1"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      }
+    },
     "node_modules/@parcel/logger": {
       "version": "2.12.0",
       "resolved": "https://registry.npmjs.org/@parcel/logger/-/logger-2.12.0.tgz",
@@ -1569,6 +2000,35 @@
         "utility-types": "^3.10.0"
       }
     },
+    "node_modules/@parcel/types-internal": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/types-internal/-/types-internal-2.13.2.tgz",
+      "integrity": "sha512-j0zb3WNM8O/+d8CArll7/4w4AyBED3Jbo32/unz89EPVN0VklmgBrRCAI5QXDKuJAGdAZSL5/a8bNYbwl7/Wxw==",
+      "peer": true,
+      "dependencies": {
+        "@parcel/diagnostic": "2.13.2",
+        "@parcel/feature-flags": "2.13.2",
+        "@parcel/source-map": "^2.1.1",
+        "utility-types": "^3.10.0"
+      }
+    },
+    "node_modules/@parcel/types-internal/node_modules/@parcel/diagnostic": {
+      "version": "2.13.2",
+      "resolved": "https://registry.npmjs.org/@parcel/diagnostic/-/diagnostic-2.13.2.tgz",
+      "integrity": "sha512-6Au0JEJ5SY2gYrY0/m0i0sTuqTvK0k2E9azhBJR+zzCREbUxLiDdLZ+vXAfLW7t/kPAcWtdNU0Bj7pnZcMiMXg==",
+      "peer": true,
+      "dependencies": {
+        "@mischnic/json-sourcemap": "^0.1.0",
+        "nullthrows": "^1.1.1"
+      },
+      "engines": {
+        "node": ">= 16.0.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/parcel"
+      }
+    },
     "node_modules/@parcel/utils": {
       "version": "2.12.0",
       "resolved": "https://registry.npmjs.org/@parcel/utils/-/utils-2.12.0.tgz",
@@ -2279,13 +2739,13 @@
       }
     },
     "node_modules/@swc/core": {
-      "version": "1.4.5",
-      "resolved": "https://registry.npmjs.org/@swc/core/-/core-1.4.5.tgz",
-      "integrity": "sha512-4/JGkG4b1Z/QwCGgx+Ub46MlzrsZvBk5JSkxm9PcZ4bSX81c+4Y94Xm3iLp5Ka8NxzS5rD4mJSpcYuN3Tw0ceg==",
+      "version": "1.10.1",
+      "resolved": "https://registry.npmjs.org/@swc/core/-/core-1.10.1.tgz",
+      "integrity": "sha512-rQ4dS6GAdmtzKiCRt3LFVxl37FaY1cgL9kSUTnhQ2xc3fmHOd7jdJK/V4pSZMG1ruGTd0bsi34O2R0Olg9Zo/w==",
       "hasInstallScript": true,
       "dependencies": {
-        "@swc/counter": "^0.1.2",
-        "@swc/types": "^0.1.5"
+        "@swc/counter": "^0.1.3",
+        "@swc/types": "^0.1.17"
       },
       "engines": {
         "node": ">=10"
@@ -2295,19 +2755,19 @@
         "url": "https://opencollective.com/swc"
       },
       "optionalDependencies": {
-        "@swc/core-darwin-arm64": "1.4.5",
-        "@swc/core-darwin-x64": "1.4.5",
-        "@swc/core-linux-arm-gnueabihf": "1.4.5",
-        "@swc/core-linux-arm64-gnu": "1.4.5",
-        "@swc/core-linux-arm64-musl": "1.4.5",
-        "@swc/core-linux-x64-gnu": "1.4.5",
-        "@swc/core-linux-x64-musl": "1.4.5",
-        "@swc/core-win32-arm64-msvc": "1.4.5",
-        "@swc/core-win32-ia32-msvc": "1.4.5",
-        "@swc/core-win32-x64-msvc": "1.4.5"
+        "@swc/core-darwin-arm64": "1.10.1",
+        "@swc/core-darwin-x64": "1.10.1",
+        "@swc/core-linux-arm-gnueabihf": "1.10.1",
+        "@swc/core-linux-arm64-gnu": "1.10.1",
+        "@swc/core-linux-arm64-musl": "1.10.1",
+        "@swc/core-linux-x64-gnu": "1.10.1",
+        "@swc/core-linux-x64-musl": "1.10.1",
+        "@swc/core-win32-arm64-msvc": "1.10.1",
+        "@swc/core-win32-ia32-msvc": "1.10.1",
+        "@swc/core-win32-x64-msvc": "1.10.1"
       },
       "peerDependencies": {
-        "@swc/helpers": "^0.5.0"
+        "@swc/helpers": "*"
       },
       "peerDependenciesMeta": {
         "@swc/helpers": {
@@ -2316,9 +2776,9 @@
       }
     },
     "node_modules/@swc/core-darwin-arm64": {
-      "version": "1.4.5",
-      "resolved": "https://registry.npmjs.org/@swc/core-darwin-arm64/-/core-darwin-arm64-1.4.5.tgz",
-      "integrity": "sha512-toMSkbByHNfGXESyY1aiq5L3KutgijrNWB/THgdHIA1aIbwtrgMdFQfxpSE+INuuvWYi/Fxarv86EnU7ewbI0Q==",
+      "version": "1.10.1",
+      "resolved": "https://registry.npmjs.org/@swc/core-darwin-arm64/-/core-darwin-arm64-1.10.1.tgz",
+      "integrity": "sha512-NyELPp8EsVZtxH/mEqvzSyWpfPJ1lugpTQcSlMduZLj1EASLO4sC8wt8hmL1aizRlsbjCX+r0PyL+l0xQ64/6Q==",
       "cpu": [
         "arm64"
       ],
@@ -2331,9 +2791,9 @@
       }
     },
     "node_modules/@swc/core-darwin-x64": {
-      "version": "1.4.5",
-      "resolved": "https://registry.npmjs.org/@swc/core-darwin-x64/-/core-darwin-x64-1.4.5.tgz",
-      "integrity": "sha512-LN8cbnmb4Gav8UcbBc+L/DEthmzCWZz22rQr6fIEHMN+f0d71fuKnV0ca0hoKbpZn33dlzUmXQE53HRjlRUQbw==",
+      "version": "1.10.1",
+      "resolved": "https://registry.npmjs.org/@swc/core-darwin-x64/-/core-darwin-x64-1.10.1.tgz",
+      "integrity": "sha512-L4BNt1fdQ5ZZhAk5qoDfUnXRabDOXKnXBxMDJ+PWLSxOGBbWE6aJTnu4zbGjJvtot0KM46m2LPAPY8ttknqaZA==",
       "cpu": [
         "x64"
       ],
@@ -2346,9 +2806,9 @@
       }
     },
     "node_modules/@swc/core-linux-arm-gnueabihf": {
-      "version": "1.4.5",
-      "resolved": "https://registry.npmjs.org/@swc/core-linux-arm-gnueabihf/-/core-linux-arm-gnueabihf-1.4.5.tgz",
-      "integrity": "sha512-suRFkhBWmOQxlM4frpos1uqjmHfaEI8FuJ0LL5+yRE7IunNDeQJBKujGZt6taeuxo1KqC0N0Ajr8IluN2wrKpA==",
+      "version": "1.10.1",
+      "resolved": "https://registry.npmjs.org/@swc/core-linux-arm-gnueabihf/-/core-linux-arm-gnueabihf-1.10.1.tgz",
+      "integrity": "sha512-Y1u9OqCHgvVp2tYQAJ7hcU9qO5brDMIrA5R31rwWQIAKDkJKtv3IlTHF0hrbWk1wPR0ZdngkQSJZple7G+Grvw==",
       "cpu": [
         "arm"
       ],
@@ -2361,9 +2821,9 @@
       }
     },
     "node_modules/@swc/core-linux-arm64-gnu": {
-      "version": "1.4.5",
-      "resolved": "https://registry.npmjs.org/@swc/core-linux-arm64-gnu/-/core-linux-arm64-gnu-1.4.5.tgz",
-      "integrity": "sha512-mLKxasQArDGmR6k9c0tkPVUdoo8VfUecocMG1Mx9NYvpidJNaZ3xq9nYM77v7uq1fQqrs/59DM1fJTNRWvv/UQ==",
+      "version": "1.10.1",
+      "resolved": "https://registry.npmjs.org/@swc/core-linux-arm64-gnu/-/core-linux-arm64-gnu-1.10.1.tgz",
+      "integrity": "sha512-tNQHO/UKdtnqjc7o04iRXng1wTUXPgVd8Y6LI4qIbHVoVPwksZydISjMcilKNLKIwOoUQAkxyJ16SlOAeADzhQ==",
       "cpu": [
         "arm64"
       ],
@@ -2376,9 +2836,9 @@
       }
     },
     "node_modules/@swc/core-linux-arm64-musl": {
-      "version": "1.4.5",
-      "resolved": "https://registry.npmjs.org/@swc/core-linux-arm64-musl/-/core-linux-arm64-musl-1.4.5.tgz",
-      "integrity": "sha512-pgKuyRP7S29U/HMDTx+x8dFcklWxwB9cHFNCNWSE6bS4vHR93jc4quwPX9OEQX5CVHxm+c8+xof043I4OGkAXw==",
+      "version": "1.10.1",
+      "resolved": "https://registry.npmjs.org/@swc/core-linux-arm64-musl/-/core-linux-arm64-musl-1.10.1.tgz",
+      "integrity": "sha512-x0L2Pd9weQ6n8dI1z1Isq00VHFvpBClwQJvrt3NHzmR+1wCT/gcYl1tp9P5xHh3ldM8Cn4UjWCw+7PaUgg8FcQ==",
       "cpu": [
         "arm64"
       ],
@@ -2391,9 +2851,9 @@
       }
     },
     "node_modules/@swc/core-linux-x64-gnu": {
-      "version": "1.4.5",
-      "resolved": "https://registry.npmjs.org/@swc/core-linux-x64-gnu/-/core-linux-x64-gnu-1.4.5.tgz",
-      "integrity": "sha512-srR+YN86Oerzoghd0DPCzTbTp08feeJPSr9kkNdmtQWENOa4l/9cJV3+XY6vviw0sEjezPmYnc3SwRxJRaxvEw==",
+      "version": "1.10.1",
+      "resolved": "https://registry.npmjs.org/@swc/core-linux-x64-gnu/-/core-linux-x64-gnu-1.10.1.tgz",
+      "integrity": "sha512-yyYEwQcObV3AUsC79rSzN9z6kiWxKAVJ6Ntwq2N9YoZqSPYph+4/Am5fM1xEQYf/kb99csj0FgOelomJSobxQA==",
       "cpu": [
         "x64"
       ],
@@ -2406,9 +2866,9 @@
       }
     },
     "node_modules/@swc/core-linux-x64-musl": {
-      "version": "1.4.5",
-      "resolved": "https://registry.npmjs.org/@swc/core-linux-x64-musl/-/core-linux-x64-musl-1.4.5.tgz",
-      "integrity": "sha512-aSf41LZtDeG5VXI4RCnzcu0UInPyNm3ip8Kw+sCK+sSqW9o7DgBkyqqbip3RZq84fNUHBQQQQdKXetltsyRRqw==",
+      "version": "1.10.1",
+      "resolved": "https://registry.npmjs.org/@swc/core-linux-x64-musl/-/core-linux-x64-musl-1.10.1.tgz",
+      "integrity": "sha512-tcaS43Ydd7Fk7sW5ROpaf2Kq1zR+sI5K0RM+0qYLYYurvsJruj3GhBCaiN3gkzd8m/8wkqNqtVklWaQYSDsyqA==",
       "cpu": [
         "x64"
       ],
@@ -2421,9 +2881,9 @@
       }
     },
     "node_modules/@swc/core-win32-arm64-msvc": {
-      "version": "1.4.5",
-      "resolved": "https://registry.npmjs.org/@swc/core-win32-arm64-msvc/-/core-win32-arm64-msvc-1.4.5.tgz",
-      "integrity": "sha512-vU3k8JwRUlTkJMfJQY9E4VvLrsIFOpfhnvbuXB84Amo1cJsz+bYQcC6RSvY7qpaDzDKFdUGbJco4uZTRoRf7Mg==",
+      "version": "1.10.1",
+      "resolved": "https://registry.npmjs.org/@swc/core-win32-arm64-msvc/-/core-win32-arm64-msvc-1.10.1.tgz",
+      "integrity": "sha512-D3Qo1voA7AkbOzQ2UGuKNHfYGKL6eejN8VWOoQYtGHHQi1p5KK/Q7V1ku55oxXBsj79Ny5FRMqiRJpVGad7bjQ==",
       "cpu": [
         "arm64"
       ],
@@ -2436,9 +2896,9 @@
       }
     },
     "node_modules/@swc/core-win32-ia32-msvc": {
-      "version": "1.4.5",
-      "resolved": "https://registry.npmjs.org/@swc/core-win32-ia32-msvc/-/core-win32-ia32-msvc-1.4.5.tgz",
-      "integrity": "sha512-856YRh3frRK2XbrSjDOFBgoAqWJLNRkaEtfGzXfeEoyJlOz0BFsSJHxKlHAFkxRfHe2li9DJRUQFTEhXn4OUWw==",
+      "version": "1.10.1",
+      "resolved": "https://registry.npmjs.org/@swc/core-win32-ia32-msvc/-/core-win32-ia32-msvc-1.10.1.tgz",
+      "integrity": "sha512-WalYdFoU3454Og+sDKHM1MrjvxUGwA2oralknXkXL8S0I/8RkWZOB++p3pLaGbTvOO++T+6znFbQdR8KRaa7DA==",
       "cpu": [
         "ia32"
       ],
@@ -2451,9 +2911,9 @@
       }
     },
     "node_modules/@swc/core-win32-x64-msvc": {
-      "version": "1.4.5",
-      "resolved": "https://registry.npmjs.org/@swc/core-win32-x64-msvc/-/core-win32-x64-msvc-1.4.5.tgz",
-      "integrity": "sha512-j1+kV7jmWY1+NbXAvxAEW165781yLXVZKLcoXIZKmw18EatqMF6w8acg1gDG8C+Iw5aWLkRZVS4pijSh7+DtCQ==",
+      "version": "1.10.1",
+      "resolved": "https://registry.npmjs.org/@swc/core-win32-x64-msvc/-/core-win32-x64-msvc-1.10.1.tgz",
+      "integrity": "sha512-JWobfQDbTnoqaIwPKQ3DVSywihVXlQMbDuwik/dDWlj33A8oEHcjPOGs4OqcA3RHv24i+lfCQpM3Mn4FAMfacA==",
       "cpu": [
         "x64"
       ],
@@ -2480,9 +2940,12 @@
       }
     },
     "node_modules/@swc/types": {
-      "version": "0.1.5",
-      "resolved": "https://registry.npmjs.org/@swc/types/-/types-0.1.5.tgz",
-      "integrity": "sha512-myfUej5naTBWnqOCc/MdVOLVjXUXtIA+NpDrDBKJtLLg2shUjBu3cZmB/85RyitKc55+lUUyl7oRfLOvkr2hsw=="
+      "version": "0.1.17",
+      "resolved": "https://registry.npmjs.org/@swc/types/-/types-0.1.17.tgz",
+      "integrity": "sha512-V5gRru+aD8YVyCOMAjMpWR1Ui577DD5KSJsHP8RAxopAH22jFz6GZd/qxqjO6MJHQhcsjvjOFXyDhyLQUnMveQ==",
+      "dependencies": {
+        "@swc/counter": "^0.1.3"
+      }
     },
     "node_modules/@theguild/remark-mermaid": {
       "version": "0.0.5",
@@ -2723,6 +3186,15 @@
         "url": "https://github.com/sponsors/wooorm"
       }
     },
+    "node_modules/base-x": {
+      "version": "3.0.10",
+      "resolved": "https://registry.npmjs.org/base-x/-/base-x-3.0.10.tgz",
+      "integrity": "sha512-7d0s06rR9rYaIWHkpfLIFICM/tkSVdoPC9qYAQRpxn9DdKNWNsKC0uk++akckyLq16Tx2WIinnZ6WRriAt6njQ==",
+      "peer": true,
+      "dependencies": {
+        "safe-buffer": "^5.0.1"
+      }
+    },
     "node_modules/boolbase": {
       "version": "1.0.0",
       "resolved": "https://registry.npmjs.org/boolbase/-/boolbase-1.0.0.tgz",
@@ -2937,6 +3409,15 @@
         "node": ">=4"
       }
     },
+    "node_modules/clone": {
+      "version": "2.1.2",
+      "resolved": "https://registry.npmjs.org/clone/-/clone-2.1.2.tgz",
+      "integrity": "sha512-3Pe/CF1Nn94hyhIYpjtiLhdCoEoz0DqQ+988E9gmeEdQZlojxnOb74wctFyuwWQHzqyf9X7C7MG8juUpqBJT8w==",
+      "peer": true,
+      "engines": {
+        "node": ">=0.8"
+      }
+    },
     "node_modules/clsx": {
       "version": "2.1.0",
       "resolved": "https://registry.npmjs.org/clsx/-/clsx-2.1.0.tgz",
@@ -3575,11 +4056,10 @@
         "node": ">=0.3.1"
       }
     },
-    "node_modules/docsgpt": {
-      "version": "0.4.7",
-      "resolved": "https://registry.npmjs.org/docsgpt/-/docsgpt-0.4.7.tgz",
-      "integrity": "sha512-4YZzLZo6ybudFrJVUQflDFeWzFiTATRWB9myrGSpLigyuMMzax1ZAY2xFallZLuEG9VVm0mOgkx3ssWHLrXWkQ==",
-      "license": "Apache-2.0",
+    "node_modules/docsgpt-react": {
+      "version": "0.4.9",
+      "resolved": "https://registry.npmjs.org/docsgpt-react/-/docsgpt-react-0.4.9.tgz",
+      "integrity": "sha512-mGGbd4IGVHrQVVdgoej991Vpl/hYkTuKz5Ax95hvqSbWDZELZnEx2/AZajAII5AayUZKWYaEFRluewUiGJVSbA==",
       "dependencies": {
         "@babel/plugin-transform-flow-strip-types": "^7.23.3",
         "@parcel/resolver-glob": "^2.12.0",
@@ -3665,6 +4145,33 @@
         "url": "https://github.com/fb55/domutils?sponsor=1"
       }
     },
+    "node_modules/dotenv": {
+      "version": "16.4.7",
+      "resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.4.7.tgz",
+      "integrity": "sha512-47qPchRCykZC03FhkYAhrvwU4xDBFIj1QPqaarj6mdM/hgUzfPHcpkHJOn3mJAufFeeAxAzeGsr5X0M4k6fLZQ==",
+      "peer": true,
+      "engines": {
+        "node": ">=12"
+      },
+      "funding": {
+        "url": "https://dotenvx.com"
+      }
+    },
+    "node_modules/dotenv-expand": {
+      "version": "11.0.7",
+      "resolved": "https://registry.npmjs.org/dotenv-expand/-/dotenv-expand-11.0.7.tgz",
+      "integrity": "sha512-zIHwmZPRshsCdpMDyVsqGmgyP0yT8GAgXUnkdAoJisxvf33k7yO6OuoKmcTGuXPWSsm8Oh88nZicRLA9Y0rUeA==",
+      "peer": true,
+      "dependencies": {
+        "dotenv": "^16.4.5"
+      },
+      "engines": {
+        "node": ">=12"
+      },
+      "funding": {
+        "url": "https://dotenvx.com"
+      }
+    },
     "node_modules/electron-to-chromium": {
       "version": "1.4.693",
       "resolved": "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.4.693.tgz",
@@ -4678,9 +5185,9 @@
       "integrity": "sha512-gfFQZrcTc8CnKXp6Y4/CBT3fTc0OVuDofpre4aEeEpSBPV5X5v4+Vmx+8snU7RLPrNHPKSgLxGo9YuQzz20o+w=="
     },
     "node_modules/katex": {
-      "version": "0.16.10",
-      "resolved": "https://registry.npmjs.org/katex/-/katex-0.16.10.tgz",
-      "integrity": "sha512-ZiqaC04tp2O5utMsl2TEZTXxa6WSC4yo0fv5ML++D3QZv/vx2Mct0mTlRx3O+uUkjfuAgOkzsCmq5MiUEsDDdA==",
+      "version": "0.16.21",
+      "resolved": "https://registry.npmjs.org/katex/-/katex-0.16.21.tgz",
+      "integrity": "sha512-XvqR7FgOHtWupfMiigNzmh+MgUVmDGU2kXZm899ZkPfcuoPuFxyHmXsgATDpFZDAXCI8tvinaVcDo8PIIJSo4A==",
       "funding": [
         "https://opencollective.com/katex",
         "https://github.com/sponsors/katex"
@@ -6234,9 +6741,9 @@
       }
     },
     "node_modules/nanoid": {
-      "version": "3.3.7",
-      "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.7.tgz",
-      "integrity": "sha512-eSRppjcPIatRIMC1U6UngP8XFcz8MQWGQdt1MTBQ7NaAmvXDfvNxbvWV3x2y6CdEUciCSsDHDQZbhYaB8QEo2g==",
+      "version": "3.3.8",
+      "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.8.tgz",
+      "integrity": "sha512-WNLf5Sd8oZxOm+TzppcYk8gVOgP+l58xNy58D0nbUnOxOWRWvlcCV4kUF7ltmI6PsrLl/BgKEyS4mqsGChFN0w==",
       "funding": [
         {
           "type": "github",
@@ -6251,11 +6758,11 @@
       }
     },
     "node_modules/next": {
-      "version": "14.2.12",
-      "resolved": "https://registry.npmjs.org/next/-/next-14.2.12.tgz",
-      "integrity": "sha512-cDOtUSIeoOvt1skKNihdExWMTybx3exnvbFbb9ecZDIxlvIbREQzt9A5Km3Zn3PfU+IFjyYGsHS+lN9VInAGKA==",
+      "version": "14.2.22",
+      "resolved": "https://registry.npmjs.org/next/-/next-14.2.22.tgz",
+      "integrity": "sha512-Ps2caobQ9hlEhscLPiPm3J3SYhfwfpMqzsoCMZGWxt9jBRK9hoBZj2A37i8joKhsyth2EuVKDVJCTF5/H4iEDw==",
       "dependencies": {
-        "@next/env": "14.2.12",
+        "@next/env": "14.2.22",
         "@swc/helpers": "0.5.5",
         "busboy": "1.6.0",
         "caniuse-lite": "^1.0.30001579",
@@ -6270,15 +6777,15 @@
         "node": ">=18.17.0"
       },
       "optionalDependencies": {
-        "@next/swc-darwin-arm64": "14.2.12",
-        "@next/swc-darwin-x64": "14.2.12",
-        "@next/swc-linux-arm64-gnu": "14.2.12",
-        "@next/swc-linux-arm64-musl": "14.2.12",
-        "@next/swc-linux-x64-gnu": "14.2.12",
-        "@next/swc-linux-x64-musl": "14.2.12",
-        "@next/swc-win32-arm64-msvc": "14.2.12",
-        "@next/swc-win32-ia32-msvc": "14.2.12",
-        "@next/swc-win32-x64-msvc": "14.2.12"
+        "@next/swc-darwin-arm64": "14.2.22",
+        "@next/swc-darwin-x64": "14.2.22",
+        "@next/swc-linux-arm64-gnu": "14.2.22",
+        "@next/swc-linux-arm64-musl": "14.2.22",
+        "@next/swc-linux-x64-gnu": "14.2.22",
+        "@next/swc-linux-x64-musl": "14.2.22",
+        "@next/swc-win32-arm64-msvc": "14.2.22",
+        "@next/swc-win32-ia32-msvc": "14.2.22",
+        "@next/swc-win32-x64-msvc": "14.2.22"
       },
       "peerDependencies": {
         "@opentelemetry/api": "^1.1.0",
@@ -9598,6 +10105,26 @@
         "node": ">=6"
       }
     },
+    "node_modules/safe-buffer": {
+      "version": "5.2.1",
+      "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
+      "integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==",
+      "funding": [
+        {
+          "type": "github",
+          "url": "https://github.com/sponsors/feross"
+        },
+        {
+          "type": "patreon",
+          "url": "https://www.patreon.com/feross"
+        },
+        {
+          "type": "consulting",
+          "url": "https://feross.org/support"
+        }
+      ],
+      "peer": true
+    },
     "node_modules/safer-buffer": {
       "version": "2.1.2",
       "resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz",
@@ -9942,6 +10469,19 @@
         "url": "https://github.com/sponsors/sindresorhus"
       }
     },
+    "node_modules/typescript": {
+      "version": "5.7.2",
+      "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.7.2.tgz",
+      "integrity": "sha512-i5t66RHxDvVN40HfDd1PsEThGNnlMCMT3jMUuoh9/0TaqWevNontacunWyN02LA9/fIbEWlcHZcgTKb9QoaLfg==",
+      "peer": true,
+      "bin": {
+        "tsc": "bin/tsc",
+        "tsserver": "bin/tsserver"
+      },
+      "engines": {
+        "node": ">=14.17"
+      }
+    },
     "node_modules/uc.micro": {
       "version": "2.1.0",
       "resolved": "https://registry.npmjs.org/uc.micro/-/uc.micro-2.1.0.tgz",

docs/package.json

@@ -7,8 +7,8 @@
   "license": "MIT",
   "dependencies": {
     "@vercel/analytics": "^1.1.1",
-    "docsgpt": "^0.4.7",
-    "next": "^14.2.12",
+    "docsgpt-react": "^0.4.9",
+    "next": "^14.2.22",
     "nextra": "^2.13.2",
     "nextra-theme-docs": "^2.13.2",
     "react": "^18.2.0",

docs/pages/Deploying/Development-Environment.md

@@ -0,0 +1,78 @@
+## Development Environments
+
+### Spin up Mongo and Redis
+
+For development, only two containers are used from [docker-compose.yaml](https://github.com/arc53/DocsGPT/blob/main/docker-compose.yaml) (by deleting all services except for Redis and Mongo).
+See file [docker-compose-dev.yaml](https://github.com/arc53/DocsGPT/blob/main/docker-compose-dev.yaml).
+
+Run
+
+```
+docker compose -f docker-compose-dev.yaml build
+docker compose -f docker-compose-dev.yaml up -d
+```
+
+### Run the Backend
+
+> [!Note]
+> Make sure you have Python 3.12 installed.
+
+1. Export required environment variables or prepare a `.env` file in the project folder:
+   - Copy [.env-template](https://github.com/arc53/DocsGPT/blob/main/application/.env-template) and create `.env`.
+
+(check out [`application/core/settings.py`](application/core/settings.py) if you want to see more config options.)
+
+2. (optional) Create a Python virtual environment:
+   You can follow the [Python official documentation](https://docs.python.org/3/tutorial/venv.html) for virtual environments.
+
+a) On Mac OS and Linux
+
+```commandline
+python -m venv venv
+. venv/bin/activate
+```
+
+b) On Windows
+
+```commandline
+python -m venv venv
+ venv/Scripts/activate
+```
+
+3. Download embedding model and save it in the `model/` folder:
+You can use the script below, or download it manually from [here](https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip), unzip it and save it in the `model/` folder.
+
+```commandline
+wget https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip
+unzip mpnet-base-v2.zip -d model
+rm mpnet-base-v2.zip
+```
+
+4. Install dependencies for the backend:
+
+```commandline
+pip install -r application/requirements.txt
+```
+
+5. Run the app using `flask --app application/app.py run --host=0.0.0.0 --port=7091`.
+6. Start worker with `celery -A application.app.celery worker -l INFO`.
+
+> [!Note]
+> You can also launch the in a debugger mode in vscode by accessing SHIFT + CMD + D or SHIFT + Windows + D on windows and selecting Flask or Celery.
+
+
+### Start Frontend
+
+> [!Note]
+> Make sure you have Node version 16 or higher.
+
+1. Navigate to the [/frontend](https://github.com/arc53/DocsGPT/tree/main/frontend) folder.
+2. Install the required packages `husky` and `vite` (ignore if already installed).
+
+```commandline
+npm install husky -g
+npm install vite -g
+```
+
+3. Install dependencies by running `npm install --include=dev`.
+4. Run the app using `npm run dev`.

docs/pages/Deploying/Quickstart.md

@@ -15,11 +15,21 @@ If you prefer to follow manual steps, refer to this guide:
 1. Open and download this repository with 
    ```bash
    git clone https://github.com/arc53/DocsGPT.git
+   cd DocsGPT
    ```
-2. Create a `.env` file in your root directory and set your `API_KEY` with your [OpenAI API key](https://platform.openai.com/account/api-keys). (optional in case you want to use OpenAI)
+2. Create a `.env` file in your root directory and set the env variables.
+   It should look like this inside:
+
+   ```
+   LLM_NAME=[docsgpt or openai or others] 
+   API_KEY=[if LLM_NAME is openai]
+   ```
+
+   See optional environment variables in the [/application/.env_sample](https://github.com/arc53/DocsGPT/blob/main/application/.env_sample) file.
+   
 3. Run the following commands:
    ```bash
-   docker-compose build && docker-compose up
+   docker compose up
    ```
 4. Navigate to http://localhost:5173/.
 
@@ -27,43 +37,28 @@ To stop, simply press **Ctrl + C**.
 
 **For WINDOWS:**
 
-To run the setup on Windows, you have two options: using the Windows Subsystem for Linux (WSL) or using Git Bash or Command Prompt.
-
-**Option 1: Using Windows Subsystem for Linux (WSL):**
-
-1. Install WSL if you haven't already. You can follow the official Microsoft documentation for installation: (https://learn.microsoft.com/en-us/windows/wsl/install).
-2. After setting up WSL, open the WSL terminal.
-3. Clone the repository and create the `.env` file:
+1. Open and download this repository with 
    ```bash
    git clone https://github.com/arc53/DocsGPT.git
    cd DocsGPT
-   echo "API_KEY=Yourkey" > .env
-   echo "VITE_API_STREAMING=true" >> .env
    ```
-4. Run the following command to start the setup with Docker Compose:
-   ```bash
-   ./run-with-docker-compose.sh
-   ```
-6. Open your web browser and navigate to http://localhost:5173/.
-7. To stop the setup, just press **Ctrl + C** in the WSL terminal
 
-**Option 2: Using Git Bash or Command Prompt (CMD):**
+2. Create a `.env` file in your root directory and set the env variables.
+   It should look like this inside:
 
-1. Install Git for Windows if you haven't already. Download it from the official website: (https://gitforwindows.org/).
-2. Open Git Bash or Command Prompt.
-3. Clone the repository and create the `.env` file:
-   ```bash
-   git clone https://github.com/arc53/DocsGPT.git
-   cd DocsGPT
-   echo "API_KEY=Yourkey" > .env
-   echo "VITE_API_STREAMING=true" >> .env
    ```
-4. Run the following command to start the setup with Docker Compose:
-   ```bash
-   ./run-with-docker-compose.sh
+   LLM_NAME=[docsgpt or openai or others] 
+   API_KEY=[if LLM_NAME is openai]
    ```
-5. Open your web browser and navigate to http://localhost:5173/.
-6. To stop the setup, just press **Ctrl + C** in the Git Bash or Command Prompt terminal.
 
-These steps should help you set up and run the project on Windows using either WSL or Git Bash/Command Prompt. 
+   See optional environment variables in the [/application/.env_sample](https://github.com/arc53/DocsGPT/blob/main/application/.env_sample) file.
+
+3. Run the following command:
+
+   ```bash
+   docker-compose up
+   ```
+4. Navigate to http://localhost:5173/.
+5. To stop the setup, just press **Ctrl + C** in the WSL terminal
+
 **Important:** Ensure that Docker is installed and properly configured on your Windows system for these steps to work.

docs/pages/Deploying/_meta.json

@@ -7,6 +7,10 @@
     "title": "⚡️Quickstart",
     "href": "/Deploying/Quickstart"
   },
+  "Development-Environment": {
+    "title": "🛠️Development Environment",
+    "href": "/Deploying/Development-Environment"
+  },
   "Railway-Deploying": {
     "title": "🚂Deploying on Railway",
     "href": "/Deploying/Railway-Deploying"

docs/pages/Extensions/react-widget.md

@@ -29,18 +29,30 @@ Now, you can use the widget in your component like this :
   buttonBg = "#222327"
 />
 ```
-To tailor the widget to your needs, you can configure the following props in your component:
-1. `apiHost` — The URL of your DocsGPT API.
-2. `theme` — Allows to select your specific theme (dark or light).
-3. `apiKey` — Usually, it's empty.
-4. `avatar`: Specifies the URL of the avatar or image representing the chatbot.
-5. `title`: Sets the title text displayed in the chatbot interface.
-6. `description`: Provides a brief description of the chatbot's purpose or functionality.
-7. `heroTitle`: Displays a welcome title when users interact with the chatbot.
-8. `heroDescription`: Provide additional introductory text or information about the chatbot's capabilities.
-9. `buttonIcon`: Specifies the url of the icon image for the widget.
-10. `buttonBg`: Allows to specify the Background color of the widget.
-11. `size`: Sets the size of the widget ( small, medium).
+### Props Table for DocsGPT Widget
+
+| **Prop**          | **Type**         | **Default Value**                                           | **Description**                                                                                     |
+|--------------------|------------------|-------------------------------------------------------------|-----------------------------------------------------------------------------------------------------|
+| **`apiHost`**      | `string`         | `"https://gptcloud.arc53.com"`                                     | The URL of your DocsGPT API for vector search and chatbot queries.                                 |
+| **`apiKey`**       | `string`         | `""`                                                        | Your API key for authentication. Can be left empty if authentication is not required.              |
+| **`avatar`**       | `string`         | `"https://d3dg1063dc54p9.cloudfront.net/cute-docsgpt.png"`   | Specifies the URL of the avatar or image representing the chatbot.                                 |
+| **`title`**        | `string`         | `"Get AI assistance"`                                       | Sets the title text displayed in the chatbot interface.                                            |
+| **`description`**  | `string`         | `"DocsGPT's AI Chatbot is here to help"`                     | Provides a brief description of the chatbot's purpose or functionality.                            |
+| **`heroTitle`**    | `string`         | `"Welcome to DocsGPT !"`                                    | Displays a welcome title when users interact with the chatbot.                                     |
+| **`heroDescription`** | `string`     | `"This chatbot is built with DocsGPT and utilises GenAI, please review important information using sources."` | Provides additional introductory text or information about the chatbot's capabilities.             |
+| **`theme`**        | `"dark" \| "light"` | `"dark"`                                                  | Allows you to select the theme for the chatbot interface. Accepts `"dark"` or `"light"`.           |
+| **`buttonIcon`**   | `string`         | `"https://your-icon"`                                        | Specifies the URL of the icon image for the widget's launch button.                                |
+| **`buttonBg`**     | `string`         | `"#222327"`                                                 | Sets the background color of the widget's launch button.                                           |
+| **`size`**         | `"small" \| "medium"` | `"medium"`                                              | Sets the size of the widget. Options are `"small"` or `"medium"`.                                  |
+
+---
+
+### Notes
+- **Customizing Props:** All properties can be overridden when embedding the widget. For example, you can provide a unique avatar, title, or color scheme to better align with your brand.
+- **Default Theme:** The widget defaults to the dark theme unless explicitly set to `"light"`. 
+- **API Key:** If the `apiKey` is not required for your application, leave it empty.
+
+This table provides a clear overview of the customization options available for tailoring the DocsGPT widget to fit your application.
 
 
 ### How to use DocsGPTWidget with [Nextra](https://nextra.site/) (Next.js + MDX)
@@ -121,5 +133,80 @@ To link the widget to your api and your documents you can pass parameters to the
 </html>
 ```
 
+# SearchBar
+
+The `SearchBar` component is an interactive search bar designed to provide search results based on **vector similarity search**. It also includes the capability to open the AI Chatbot, enabling users to query.
+
+---
+
+### Importing the Component
+```tsx
+import { SearchBar } from "docsgpt-react";
+```
+
+---
+
+### Usage Example
+```tsx
+<SearchBar 
+    apiKey="your-api-key"
+    apiHost="https://gptcloud.arc53.com"
+    theme="light"
+    placeholder="Search or Ask AI..."
+    width="300px"
+/>
+```
+
+---
+
+## HTML embedding for Search bar
+
+```html
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1.0">
+  <title>SearchBar Embedding</title>
+  <script src="https://unpkg.com/docsgpt/dist/modern/main.js"></script> <!-- The bundled JavaScript file -->
+</head>
+<body>
+  <!-- Element where the SearchBar will render -->
+  <div id="search-bar-container"></div>
+
+  <script>
+    // Render the SearchBar into the specified element
+    renderSearchBar('search-bar-container', {
+      apiKey: 'your-api-key-here',
+      apiHost: 'https://your-api-host.com',
+      theme: 'light',
+      placeholder: 'Search here...',
+      width: '300px'
+    });
+  </script>
+</body>
+</html>
+
+```
+
+### Props
+
+| **Prop**       | **Type**  | **Default Value**                   | **Description**                                                                                  |
+|-----------------|-----------|-------------------------------------|--------------------------------------------------------------------------------------------------|
+| **`apiKey`**    | `string`  | `"74039c6d-bff7-44ce-ae55-2973cbf13837"` | Your API key generated from the app. Used for authenticating requests.                         |
+| **`apiHost`**   | `string`  | `"https://gptcloud.arc53.com"`       | The base URL of the server hosting the vector similarity search and chatbot services.           |
+| **`theme`**     | `"dark" \| "light"` | `"dark"`                            | The theme of the search bar. Accepts `"dark"` or `"light"`.                                     |
+| **`placeholder`** | `string` | `"Search or Ask AI..."`             | Placeholder text displayed in the search input field.                                           |
+| **`width`**     | `string`  | `"256px"`                          | Width of the search bar. Accepts any valid CSS width value (e.g., `"300px"`, `"100%"`, `"20rem"`). |
+
+
+Feel free to reach out if you need help customizing or extending the `SearchBar`!
+
+## Our github
+
+[DocsGPT](https://github.com/arc53/DocsGPT)
+
+You can find the source code in the extensions/react-widget folder.
+
 For more information about React, refer to this [link here](https://react.dev/learn)
 

docs/pages/_app.mdx

@@ -1,4 +1,4 @@
-import { DocsGPTWidget } from "docsgpt";
+import { DocsGPTWidget } from "docsgpt-react";
 
 export default function MyApp({ Component, pageProps }) {
   return (

docs/pages/index.mdx

@@ -25,7 +25,10 @@ DocsGPT 🦖 is an innovative open-source tool designed to simplify the retrieva
 
 
 
-<Image src="/homevideo.gif" alt="homedemo" width={800} height={500}/>
+<video controls width={1920} height={1080} muted autoPlay loop playsInline>
+  <source src="https://d3dg1063dc54p9.cloudfront.net/videos/demov4.mp4" type="video/mp4" />
+  Your browser does not support the video tag.
+</video>
 
 
 Try it yourself: [https://www.docsgpt.cloud/](https://www.docsgpt.cloud/)

extensions/react-widget/README.md

@@ -13,7 +13,7 @@ npm install  docsgpt
 ### React
 
 ```javascript
-    import { DocsGPTWidget } from "docsgpt";
+    import { DocsGPTWidget } from "docsgpt-react";
 
     const App = () => {
       return <DocsGPTWidget />;
@@ -23,11 +23,11 @@ npm install  docsgpt
 To link the widget to your api and your documents you can pass parameters to the <DocsGPTWidget /> component.
 
 ```javascript
-    import { DocsGPTWidget } from "docsgpt";
+    import { DocsGPTWidget } from "docsgpt-react";
 
     const App = () => {
       return <DocsGPTWidget
-               apiHost="https://your-docsgpt-api.com"
+               apiHost="https://gptcloud.arc53.com"
                apiKey=""
                avatar = "https://d3dg1063dc54p9.cloudfront.net/cute-docsgpt.png"
                title = "Get AI assistance"
@@ -101,6 +101,75 @@ To link the widget to your api and your documents you can pass parameters to the
     </html>
 ```
 
+# SearchBar
+
+The `SearchBar` component is an interactive search bar designed to provide search results based on **vector similarity search**. It also includes the capability to open the AI Chatbot, enabling users to query.
+
+---
+
+### Importing the Component
+```tsx
+import { SearchBar } from "docsgpt-react";
+```
+
+---
+
+### Usage Example
+```tsx
+<SearchBar 
+    apiKey="your-api-key"
+    apiHost="https://gptcloud.arc53.com"
+    theme="light"
+    placeholder="Search or Ask AI..."
+    width="300px"
+/>
+```
+
+---
+
+## HTML embedding for Search bar
+
+```html
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1.0">
+  <title>SearchBar Embedding</title>
+  <script src="https://unpkg.com/docsgpt/dist/modern/main.js"></script> <!-- The bundled JavaScript file -->
+</head>
+<body>
+  <!-- Element where the SearchBar will render -->
+  <div id="search-bar-container"></div>
+
+  <script>
+    // Render the SearchBar into the specified element
+    renderSearchBar('search-bar-container', {
+      apiKey: 'your-api-key-here',
+      apiHost: 'https://your-api-host.com',
+      theme: 'light',
+      placeholder: 'Search here...',
+      width: '300px'
+    });
+  </script>
+</body>
+</html>
+
+```
+
+### Props
+
+| **Prop**       | **Type**  | **Default Value**                   | **Description**                                                                                  |
+|-----------------|-----------|-------------------------------------|--------------------------------------------------------------------------------------------------|
+| **`apiKey`**    | `string`  | `"74039c6d-bff7-44ce-ae55-2973cbf13837"` | Your API key generated from the app. Used for authenticating requests.                         |
+| **`apiHost`**   | `string`  | `"https://gptcloud.arc53.com"`       | The base URL of the server hosting the vector similarity search and chatbot services.           |
+| **`theme`**     | `"dark" \| "light"` | `"dark"`                            | The theme of the search bar. Accepts `"dark"` or `"light"`.                                     |
+| **`placeholder`** | `string` | `"Search or Ask AI..."`             | Placeholder text displayed in the search input field.                                           |
+| **`width`**     | `string`  | `"256px"`                          | Width of the search bar. Accepts any valid CSS width value (e.g., `"300px"`, `"100%"`, `"20rem"`). |
+
+
+Feel free to reach out if you need help customizing or extending the `SearchBar`!
+
 ## Our github
 
 [DocsGPT](https://github.com/arc53/DocsGPT)

extensions/react-widget/package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "docsgpt",
-  "version": "0.4.7",
+  "version": "0.4.9",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "docsgpt",
-      "version": "0.4.7",
+      "version": "0.4.9",
       "license": "Apache-2.0",
       "dependencies": {
         "@babel/plugin-transform-flow-strip-types": "^7.23.3",
@@ -1885,6 +1885,17 @@
         "node": ">=6.0.0"
       }
     },
+    "node_modules/@jridgewell/source-map": {
+      "version": "0.3.6",
+      "resolved": "https://registry.npmjs.org/@jridgewell/source-map/-/source-map-0.3.6.tgz",
+      "integrity": "sha512-1ZJTZebgqllO79ue2bm3rIGud/bOe0pP5BjSRCRxxYkEZS8STV7zN84UBbiYu7jy+eCKSnVIUgoWWE/tt+shMQ==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@jridgewell/gen-mapping": "^0.3.5",
+        "@jridgewell/trace-mapping": "^0.3.25"
+      }
+    },
     "node_modules/@jridgewell/sourcemap-codec": {
       "version": "1.4.15",
       "resolved": "https://registry.npmjs.org/@jridgewell/sourcemap-codec/-/sourcemap-codec-1.4.15.tgz",
@@ -2258,7 +2269,6 @@
       "version": "2.12.0",
       "resolved": "https://registry.npmjs.org/@parcel/core/-/core-2.12.0.tgz",
       "integrity": "sha512-s+6pwEj+GfKf7vqGUzN9iSEPueUssCCQrCBUlcAfKrJe0a22hTUCjewpB0I7lNrCIULt8dkndD+sMdOrXsRl6Q==",
-      "dev": true,
       "dependencies": {
         "@mischnic/json-sourcemap": "^0.1.0",
         "@parcel/cache": "2.12.0",
@@ -2298,7 +2308,6 @@
       "version": "7.6.2",
       "resolved": "https://registry.npmjs.org/semver/-/semver-7.6.2.tgz",
       "integrity": "sha512-FNAIBWCx9qcRhoHcgcJ0gvU7SN1lYU2ZXuSfl04bSC5OpvDHFyJCjdNHomPXxjQlCBU67YW64PzY7/VIEH7F2w==",
-      "dev": true,
       "bin": {
         "semver": "bin/semver.js"
       },
@@ -2360,7 +2369,6 @@
       "version": "3.2.0",
       "resolved": "https://registry.npmjs.org/@parcel/graph/-/graph-3.2.0.tgz",
       "integrity": "sha512-xlrmCPqy58D4Fg5umV7bpwDx5Vyt7MlnQPxW68vae5+BA4GSWetfZt+Cs5dtotMG2oCHzZxhIPt7YZ7NRyQzLA==",
-      "dev": true,
       "dependencies": {
         "nullthrows": "^1.1.1"
       },
@@ -4560,7 +4568,7 @@
       "version": "0.5.11",
       "resolved": "https://registry.npmjs.org/@swc/helpers/-/helpers-0.5.11.tgz",
       "integrity": "sha512-YNlnKRWF2sVojTpIyzwou9XoTNbzbzONwRhOoniEioF1AtaitTvVZblaQRrAzChWQ1bLYyYSWzM18y4WwgzJ+A==",
-      "dev": true,
+      "devOptional": true,
       "dependencies": {
         "tslib": "^2.4.0"
       }
@@ -4590,6 +4598,35 @@
         "@types/trusted-types": "*"
       }
     },
+    "node_modules/@types/eslint": {
+      "version": "9.6.1",
+      "resolved": "https://registry.npmjs.org/@types/eslint/-/eslint-9.6.1.tgz",
+      "integrity": "sha512-FXx2pKgId/WyYo2jXw63kk7/+TY7u7AziEJxJAnSFzHlqTAS3Ync6SvgYAN/k4/PQpnnVuzoMuVnByKK2qp0ag==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@types/estree": "*",
+        "@types/json-schema": "*"
+      }
+    },
+    "node_modules/@types/eslint-scope": {
+      "version": "3.7.7",
+      "resolved": "https://registry.npmjs.org/@types/eslint-scope/-/eslint-scope-3.7.7.tgz",
+      "integrity": "sha512-MzMFlSLBqNF2gcHWO0G1vP/YQyfvrxZ0bF+u7mzUdZ1/xK4A4sru+nraZz5i3iEIk1l1uyicaDVTB4QbbEkAYg==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@types/eslint": "*",
+        "@types/estree": "*"
+      }
+    },
+    "node_modules/@types/estree": {
+      "version": "1.0.6",
+      "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.6.tgz",
+      "integrity": "sha512-AYnb1nQyY49te+VRAVgmzfcgjYS91mY5P0TKUDCLEM+gNnA+3T6rWITXRLYCpahpqSQbN5cE+gHpnPyXjHWxcw==",
+      "dev": true,
+      "peer": true
+    },
     "node_modules/@types/json-schema": {
       "version": "7.0.15",
       "resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.15.tgz",
@@ -4621,6 +4658,16 @@
       "dev": true,
       "license": "MIT"
     },
+    "node_modules/@types/node": {
+      "version": "22.10.1",
+      "resolved": "https://registry.npmjs.org/@types/node/-/node-22.10.1.tgz",
+      "integrity": "sha512-qKgsUwfHZV2WCWLAnVP1JqnpE6Im6h3Y0+fYgMTasNQ7V++CBX5OT1as0g0f+OyubbFqhf6XVNIsmN4IIhEgGQ==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "undici-types": "~6.20.0"
+      }
+    },
     "node_modules/@types/parse-json": {
       "version": "4.0.2",
       "resolved": "https://registry.npmjs.org/@types/parse-json/-/parse-json-4.0.2.tgz",
@@ -4662,11 +4709,198 @@
       "integrity": "sha512-ScaPdn1dQczgbl0QFTeTOmVHFULt394XJgOQNoyVhZ6r2vLnMLJfBPd53SB52T/3G36VI1/g2MZaX0cwDuXsfw==",
       "dev": true
     },
+    "node_modules/@webassemblyjs/ast": {
+      "version": "1.14.1",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/ast/-/ast-1.14.1.tgz",
+      "integrity": "sha512-nuBEDgQfm1ccRp/8bCQrx1frohyufl4JlbMMZ4P1wpeOfDhF6FQkxZJ1b/e+PLwr6X1Nhw6OLme5usuBWYBvuQ==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@webassemblyjs/helper-numbers": "1.13.2",
+        "@webassemblyjs/helper-wasm-bytecode": "1.13.2"
+      }
+    },
+    "node_modules/@webassemblyjs/floating-point-hex-parser": {
+      "version": "1.13.2",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/floating-point-hex-parser/-/floating-point-hex-parser-1.13.2.tgz",
+      "integrity": "sha512-6oXyTOzbKxGH4steLbLNOu71Oj+C8Lg34n6CqRvqfS2O71BxY6ByfMDRhBytzknj9yGUPVJ1qIKhRlAwO1AovA==",
+      "dev": true,
+      "peer": true
+    },
+    "node_modules/@webassemblyjs/helper-api-error": {
+      "version": "1.13.2",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/helper-api-error/-/helper-api-error-1.13.2.tgz",
+      "integrity": "sha512-U56GMYxy4ZQCbDZd6JuvvNV/WFildOjsaWD3Tzzvmw/mas3cXzRJPMjP83JqEsgSbyrmaGjBfDtV7KDXV9UzFQ==",
+      "dev": true,
+      "peer": true
+    },
+    "node_modules/@webassemblyjs/helper-buffer": {
+      "version": "1.14.1",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/helper-buffer/-/helper-buffer-1.14.1.tgz",
+      "integrity": "sha512-jyH7wtcHiKssDtFPRB+iQdxlDf96m0E39yb0k5uJVhFGleZFoNw1c4aeIcVUPPbXUVJ94wwnMOAqUHyzoEPVMA==",
+      "dev": true,
+      "peer": true
+    },
+    "node_modules/@webassemblyjs/helper-numbers": {
+      "version": "1.13.2",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/helper-numbers/-/helper-numbers-1.13.2.tgz",
+      "integrity": "sha512-FE8aCmS5Q6eQYcV3gI35O4J789wlQA+7JrqTTpJqn5emA4U2hvwJmvFRC0HODS+3Ye6WioDklgd6scJ3+PLnEA==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@webassemblyjs/floating-point-hex-parser": "1.13.2",
+        "@webassemblyjs/helper-api-error": "1.13.2",
+        "@xtuc/long": "4.2.2"
+      }
+    },
+    "node_modules/@webassemblyjs/helper-wasm-bytecode": {
+      "version": "1.13.2",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/helper-wasm-bytecode/-/helper-wasm-bytecode-1.13.2.tgz",
+      "integrity": "sha512-3QbLKy93F0EAIXLh0ogEVR6rOubA9AoZ+WRYhNbFyuB70j3dRdwH9g+qXhLAO0kiYGlg3TxDV+I4rQTr/YNXkA==",
+      "dev": true,
+      "peer": true
+    },
+    "node_modules/@webassemblyjs/helper-wasm-section": {
+      "version": "1.14.1",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/helper-wasm-section/-/helper-wasm-section-1.14.1.tgz",
+      "integrity": "sha512-ds5mXEqTJ6oxRoqjhWDU83OgzAYjwsCV8Lo/N+oRsNDmx/ZDpqalmrtgOMkHwxsG0iI//3BwWAErYRHtgn0dZw==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@webassemblyjs/ast": "1.14.1",
+        "@webassemblyjs/helper-buffer": "1.14.1",
+        "@webassemblyjs/helper-wasm-bytecode": "1.13.2",
+        "@webassemblyjs/wasm-gen": "1.14.1"
+      }
+    },
+    "node_modules/@webassemblyjs/ieee754": {
+      "version": "1.13.2",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/ieee754/-/ieee754-1.13.2.tgz",
+      "integrity": "sha512-4LtOzh58S/5lX4ITKxnAK2USuNEvpdVV9AlgGQb8rJDHaLeHciwG4zlGr0j/SNWlr7x3vO1lDEsuePvtcDNCkw==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@xtuc/ieee754": "^1.2.0"
+      }
+    },
+    "node_modules/@webassemblyjs/leb128": {
+      "version": "1.13.2",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/leb128/-/leb128-1.13.2.tgz",
+      "integrity": "sha512-Lde1oNoIdzVzdkNEAWZ1dZ5orIbff80YPdHx20mrHwHrVNNTjNr8E3xz9BdpcGqRQbAEa+fkrCb+fRFTl/6sQw==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@xtuc/long": "4.2.2"
+      }
+    },
+    "node_modules/@webassemblyjs/utf8": {
+      "version": "1.13.2",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/utf8/-/utf8-1.13.2.tgz",
+      "integrity": "sha512-3NQWGjKTASY1xV5m7Hr0iPeXD9+RDobLll3T9d2AO+g3my8xy5peVyjSag4I50mR1bBSN/Ct12lo+R9tJk0NZQ==",
+      "dev": true,
+      "peer": true
+    },
+    "node_modules/@webassemblyjs/wasm-edit": {
+      "version": "1.14.1",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/wasm-edit/-/wasm-edit-1.14.1.tgz",
+      "integrity": "sha512-RNJUIQH/J8iA/1NzlE4N7KtyZNHi3w7at7hDjvRNm5rcUXa00z1vRz3glZoULfJ5mpvYhLybmVcwcjGrC1pRrQ==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@webassemblyjs/ast": "1.14.1",
+        "@webassemblyjs/helper-buffer": "1.14.1",
+        "@webassemblyjs/helper-wasm-bytecode": "1.13.2",
+        "@webassemblyjs/helper-wasm-section": "1.14.1",
+        "@webassemblyjs/wasm-gen": "1.14.1",
+        "@webassemblyjs/wasm-opt": "1.14.1",
+        "@webassemblyjs/wasm-parser": "1.14.1",
+        "@webassemblyjs/wast-printer": "1.14.1"
+      }
+    },
+    "node_modules/@webassemblyjs/wasm-gen": {
+      "version": "1.14.1",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/wasm-gen/-/wasm-gen-1.14.1.tgz",
+      "integrity": "sha512-AmomSIjP8ZbfGQhumkNvgC33AY7qtMCXnN6bL2u2Js4gVCg8fp735aEiMSBbDR7UQIj90n4wKAFUSEd0QN2Ukg==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@webassemblyjs/ast": "1.14.1",
+        "@webassemblyjs/helper-wasm-bytecode": "1.13.2",
+        "@webassemblyjs/ieee754": "1.13.2",
+        "@webassemblyjs/leb128": "1.13.2",
+        "@webassemblyjs/utf8": "1.13.2"
+      }
+    },
+    "node_modules/@webassemblyjs/wasm-opt": {
+      "version": "1.14.1",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/wasm-opt/-/wasm-opt-1.14.1.tgz",
+      "integrity": "sha512-PTcKLUNvBqnY2U6E5bdOQcSM+oVP/PmrDY9NzowJjislEjwP/C4an2303MCVS2Mg9d3AJpIGdUFIQQWbPds0Sw==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@webassemblyjs/ast": "1.14.1",
+        "@webassemblyjs/helper-buffer": "1.14.1",
+        "@webassemblyjs/wasm-gen": "1.14.1",
+        "@webassemblyjs/wasm-parser": "1.14.1"
+      }
+    },
+    "node_modules/@webassemblyjs/wasm-parser": {
+      "version": "1.14.1",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/wasm-parser/-/wasm-parser-1.14.1.tgz",
+      "integrity": "sha512-JLBl+KZ0R5qB7mCnud/yyX08jWFw5MsoalJ1pQ4EdFlgj9VdXKGuENGsiCIjegI1W7p91rUlcB/LB5yRJKNTcQ==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@webassemblyjs/ast": "1.14.1",
+        "@webassemblyjs/helper-api-error": "1.13.2",
+        "@webassemblyjs/helper-wasm-bytecode": "1.13.2",
+        "@webassemblyjs/ieee754": "1.13.2",
+        "@webassemblyjs/leb128": "1.13.2",
+        "@webassemblyjs/utf8": "1.13.2"
+      }
+    },
+    "node_modules/@webassemblyjs/wast-printer": {
+      "version": "1.14.1",
+      "resolved": "https://registry.npmjs.org/@webassemblyjs/wast-printer/-/wast-printer-1.14.1.tgz",
+      "integrity": "sha512-kPSSXE6De1XOR820C90RIo2ogvZG+c3KiHzqUoO/F34Y2shGzesfqv7o57xrxovZJH/MetF5UjroJ/R/3isoiw==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@webassemblyjs/ast": "1.14.1",
+        "@xtuc/long": "4.2.2"
+      }
+    },
+    "node_modules/@xtuc/ieee754": {
+      "version": "1.2.0",
+      "resolved": "https://registry.npmjs.org/@xtuc/ieee754/-/ieee754-1.2.0.tgz",
+      "integrity": "sha512-DX8nKgqcGwsc0eJSqYt5lwP4DH5FlHnmuWWBRy7X0NcaGR0ZtuyeESgMwTYVEtxmsNGY+qit4QYT/MIYTOTPeA==",
+      "dev": true,
+      "peer": true
+    },
+    "node_modules/@xtuc/long": {
+      "version": "4.2.2",
+      "resolved": "https://registry.npmjs.org/@xtuc/long/-/long-4.2.2.tgz",
+      "integrity": "sha512-NuHqBY1PB/D8xU6s/thBgOAiAP7HOYDQ32+BFZILJ8ivkUkAHQnWfn6WhL79Owj1qmUnoN/YPhktdIoucipkAQ==",
+      "dev": true,
+      "peer": true
+    },
     "node_modules/abortcontroller-polyfill": {
       "version": "1.7.5",
       "resolved": "https://registry.npmjs.org/abortcontroller-polyfill/-/abortcontroller-polyfill-1.7.5.tgz",
-      "integrity": "sha512-JMJ5soJWP18htbbxJjG7bG6yuI6pRhgJ0scHHTfkUjf6wjP912xZWvM+A4sJK3gqd9E8fcPbDnOefbA9Th/FIQ==",
-      "dev": true
+      "integrity": "sha512-JMJ5soJWP18htbbxJjG7bG6yuI6pRhgJ0scHHTfkUjf6wjP912xZWvM+A4sJK3gqd9E8fcPbDnOefbA9Th/FIQ=="
+    },
+    "node_modules/acorn": {
+      "version": "8.14.0",
+      "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.14.0.tgz",
+      "integrity": "sha512-cl669nCJTZBsL97OF4kUQm5g5hC2uihk0NxY3WENAC0TYdILVkAyHymAntgxGkl7K+t0cXIrH5siy5S4XkFycA==",
+      "dev": true,
+      "peer": true,
+      "bin": {
+        "acorn": "bin/acorn"
+      },
+      "engines": {
+        "node": ">=0.4.0"
+      }
     },
     "node_modules/ajv": {
       "version": "6.12.6",
@@ -4771,7 +5005,6 @@
       "version": "3.0.9",
       "resolved": "https://registry.npmjs.org/base-x/-/base-x-3.0.9.tgz",
       "integrity": "sha512-H7JU6iBHTal1gp56aKoaa//YUxEaAOUiydvrV/pILqIHXTtqxSkATOnDA2u+jZ/61sD+L/412+7kzXRtWukhpQ==",
-      "dev": true,
       "dependencies": {
         "safe-buffer": "^5.0.1"
       }
@@ -4802,9 +5035,9 @@
       }
     },
     "node_modules/browserslist": {
-      "version": "4.23.0",
-      "resolved": "https://registry.npmjs.org/browserslist/-/browserslist-4.23.0.tgz",
-      "integrity": "sha512-QW8HiM1shhT2GuzkvklfjcKDiWFXHOeFCIA/huJPwHsslwcydgk7X+z2zXpEijP98UCY7HbubZt5J2Zgvf0CaQ==",
+      "version": "4.24.2",
+      "resolved": "https://registry.npmjs.org/browserslist/-/browserslist-4.24.2.tgz",
+      "integrity": "sha512-ZIc+Q62revdMcqC6aChtW4jz3My3klmCO1fEmINZY/8J3EpBg5/A/D0AKmBveUh6pgoeycoMkVMko84tuYS+Gg==",
       "funding": [
         {
           "type": "opencollective",
@@ -4820,10 +5053,10 @@
         }
       ],
       "dependencies": {
-        "caniuse-lite": "^1.0.30001587",
-        "electron-to-chromium": "^1.4.668",
-        "node-releases": "^2.0.14",
-        "update-browserslist-db": "^1.0.13"
+        "caniuse-lite": "^1.0.30001669",
+        "electron-to-chromium": "^1.5.41",
+        "node-releases": "^2.0.18",
+        "update-browserslist-db": "^1.1.1"
       },
       "bin": {
         "browserslist": "cli.js"
@@ -4832,6 +5065,13 @@
         "node": "^6 || ^7 || ^8 || ^9 || ^10 || ^11 || ^12 || >=13.7"
       }
     },
+    "node_modules/buffer-from": {
+      "version": "1.1.2",
+      "resolved": "https://registry.npmjs.org/buffer-from/-/buffer-from-1.1.2.tgz",
+      "integrity": "sha512-E+XQCRwSbaaiChtv6k6Dwgc+bx+Bs6vuKJHHl5kox/BaKbhiXzqQOwK4cO22yElGp2OCmjwVhT3HmxgyPGnJfQ==",
+      "dev": true,
+      "peer": true
+    },
     "node_modules/callsites": {
       "version": "3.1.0",
       "resolved": "https://registry.npmjs.org/callsites/-/callsites-3.1.0.tgz",
@@ -4860,9 +5100,9 @@
       }
     },
     "node_modules/caniuse-lite": {
-      "version": "1.0.30001625",
-      "resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001625.tgz",
-      "integrity": "sha512-4KE9N2gcRH+HQhpeiRZXd+1niLB/XNLAhSy4z7fI8EzcbcPoAqjNInxVHTiTwWfTIV4w096XG8OtCOCQQKPv3w==",
+      "version": "1.0.30001680",
+      "resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001680.tgz",
+      "integrity": "sha512-rPQy70G6AGUMnbwS1z6Xg+RkHYPAi18ihs47GH0jcxIG7wArmPgY3XbS2sRdBbxJljp3thdT8BIqv9ccCypiPA==",
       "funding": [
         {
           "type": "opencollective",
@@ -4876,7 +5116,8 @@
           "type": "github",
           "url": "https://github.com/sponsors/ai"
         }
-      ]
+      ],
+      "license": "CC-BY-4.0"
     },
     "node_modules/chalk": {
       "version": "2.4.2",
@@ -4922,7 +5163,6 @@
       "version": "2.1.2",
       "resolved": "https://registry.npmjs.org/clone/-/clone-2.1.2.tgz",
       "integrity": "sha512-3Pe/CF1Nn94hyhIYpjtiLhdCoEoz0DqQ+988E9gmeEdQZlojxnOb74wctFyuwWQHzqyf9X7C7MG8juUpqBJT8w==",
-      "dev": true,
       "engines": {
         "node": ">=0.8"
       }
@@ -5132,7 +5372,6 @@
       "version": "7.0.0",
       "resolved": "https://registry.npmjs.org/dotenv/-/dotenv-7.0.0.tgz",
       "integrity": "sha512-M3NhsLbV1i6HuGzBUH8vXrtxOk+tWmzWKDMbAVSUp3Zsjm7ywFeuwrUXhmhQyRK1q5B5GGy7hcXPbj3bnfZg2g==",
-      "dev": true,
       "engines": {
         "node": ">=6"
       }
@@ -5140,13 +5379,12 @@
     "node_modules/dotenv-expand": {
       "version": "5.1.0",
       "resolved": "https://registry.npmjs.org/dotenv-expand/-/dotenv-expand-5.1.0.tgz",
-      "integrity": "sha512-YXQl1DSa4/PQyRfgrv6aoNjhasp/p4qs9FjJ4q4cQk+8m4r6k4ZSiEyytKG8f8W9gi8WsQtIObNmKd+tMzNTmA==",
-      "dev": true
+      "integrity": "sha512-YXQl1DSa4/PQyRfgrv6aoNjhasp/p4qs9FjJ4q4cQk+8m4r6k4ZSiEyytKG8f8W9gi8WsQtIObNmKd+tMzNTmA=="
     },
     "node_modules/electron-to-chromium": {
-      "version": "1.4.788",
-      "resolved": "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.4.788.tgz",
-      "integrity": "sha512-ubp5+Ev/VV8KuRoWnfP2QF2Bg+O2ZFdb49DiiNbz2VmgkIqrnyYaqIOqj8A6K/3p1xV0QcU5hBQ1+BmB6ot1OA=="
+      "version": "1.5.72",
+      "resolved": "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.5.72.tgz",
+      "integrity": "sha512-ZpSAUOZ2Izby7qnZluSrAlGgGQzucmFbN0n64dYzocYxnxV5ufurpj3VgEe4cUp7ir9LmeLxNYo8bVnlM8bQHw=="
     },
     "node_modules/emojis-list": {
       "version": "3.0.0",
@@ -5157,6 +5395,20 @@
         "node": ">= 4"
       }
     },
+    "node_modules/enhanced-resolve": {
+      "version": "5.17.1",
+      "resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-5.17.1.tgz",
+      "integrity": "sha512-LMHl3dXhTcfv8gM4kEzIUeTQ+7fpdA0l2tUf34BddXPkz2A5xJ5L/Pchd5BL6rdccM9QGvu0sWZzK1Z1t4wwyg==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "graceful-fs": "^4.2.4",
+        "tapable": "^2.2.0"
+      },
+      "engines": {
+        "node": ">=10.13.0"
+      }
+    },
     "node_modules/entities": {
       "version": "4.5.0",
       "resolved": "https://registry.npmjs.org/entities/-/entities-4.5.0.tgz",
@@ -5185,10 +5437,17 @@
         "is-arrayish": "^0.2.1"
       }
     },
+    "node_modules/es-module-lexer": {
+      "version": "1.5.4",
+      "resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-1.5.4.tgz",
+      "integrity": "sha512-MVNK56NiMrOwitFB7cqDwq0CQutbw+0BvLshJSse0MUNU+y1FC3bUS/AQg7oUng+/wKrrki7JfmwtVHkVfPLlw==",
+      "dev": true,
+      "peer": true
+    },
     "node_modules/escalade": {
-      "version": "3.1.2",
-      "resolved": "https://registry.npmjs.org/escalade/-/escalade-3.1.2.tgz",
-      "integrity": "sha512-ErCHMCae19vR8vQGe50xIsVomy19rg6gFu3+r3jkEO46suLMWBksvVyoGgQV+jOfl84ZSOSlmv6Gxa89PmTGmA==",
+      "version": "3.2.0",
+      "resolved": "https://registry.npmjs.org/escalade/-/escalade-3.2.0.tgz",
+      "integrity": "sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA==",
       "engines": {
         "node": ">=6"
       }
@@ -5201,6 +5460,53 @@
         "node": ">=0.8.0"
       }
     },
+    "node_modules/eslint-scope": {
+      "version": "5.1.1",
+      "resolved": "https://registry.npmjs.org/eslint-scope/-/eslint-scope-5.1.1.tgz",
+      "integrity": "sha512-2NxwbF/hZ0KpepYN0cNbo+FN6XoK7GaHlQhgx/hIZl6Va0bF45RQOOwhLIy8lQDbuCiadSLCBnH2CFYquit5bw==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "esrecurse": "^4.3.0",
+        "estraverse": "^4.1.1"
+      },
+      "engines": {
+        "node": ">=8.0.0"
+      }
+    },
+    "node_modules/esrecurse": {
+      "version": "4.3.0",
+      "resolved": "https://registry.npmjs.org/esrecurse/-/esrecurse-4.3.0.tgz",
+      "integrity": "sha512-KmfKL3b6G+RXvP8N1vr3Tq1kL/oCFgn2NYXEtqP8/L3pKapUA4G8cFVaoF3SU323CD4XypR/ffioHmkti6/Tag==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "estraverse": "^5.2.0"
+      },
+      "engines": {
+        "node": ">=4.0"
+      }
+    },
+    "node_modules/esrecurse/node_modules/estraverse": {
+      "version": "5.3.0",
+      "resolved": "https://registry.npmjs.org/estraverse/-/estraverse-5.3.0.tgz",
+      "integrity": "sha512-MMdARuVEQziNTeJD8DgMqmhwR11BRQ/cBP+pLtYdSTnf3MIO8fFeiINEbX36ZdNlfU/7A9f3gUw49B3oQsvwBA==",
+      "dev": true,
+      "peer": true,
+      "engines": {
+        "node": ">=4.0"
+      }
+    },
+    "node_modules/estraverse": {
+      "version": "4.3.0",
+      "resolved": "https://registry.npmjs.org/estraverse/-/estraverse-4.3.0.tgz",
+      "integrity": "sha512-39nnKffWz8xN1BU/2c79n9nB9HDzo0niYUqx6xyqUnyoAnQyyWpOTdZEeiCch8BBu515t4wp9ZmgVfVhn9EBpw==",
+      "dev": true,
+      "peer": true,
+      "engines": {
+        "node": ">=4.0"
+      }
+    },
     "node_modules/esutils": {
       "version": "2.0.3",
       "resolved": "https://registry.npmjs.org/esutils/-/esutils-2.0.3.tgz",
@@ -5210,6 +5516,16 @@
         "node": ">=0.10.0"
       }
     },
+    "node_modules/events": {
+      "version": "3.3.0",
+      "resolved": "https://registry.npmjs.org/events/-/events-3.3.0.tgz",
+      "integrity": "sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q==",
+      "dev": true,
+      "peer": true,
+      "engines": {
+        "node": ">=0.8.x"
+      }
+    },
     "node_modules/fast-deep-equal": {
       "version": "3.1.3",
       "resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-3.1.3.tgz",
@@ -5300,6 +5616,13 @@
         "node": ">=6"
       }
     },
+    "node_modules/glob-to-regexp": {
+      "version": "0.4.1",
+      "resolved": "https://registry.npmjs.org/glob-to-regexp/-/glob-to-regexp-0.4.1.tgz",
+      "integrity": "sha512-lkX1HJXwyMcprw/5YUZc2s7DrpAiHB21/V+E1rHUrVNokkvB6bqMzT0VfV6/86ZNabt1k14YOIaT7nDvOX3Iiw==",
+      "dev": true,
+      "peer": true
+    },
     "node_modules/globals": {
       "version": "11.12.0",
       "resolved": "https://registry.npmjs.org/globals/-/globals-11.12.0.tgz",
@@ -5308,6 +5631,13 @@
         "node": ">=4"
       }
     },
+    "node_modules/graceful-fs": {
+      "version": "4.2.11",
+      "resolved": "https://registry.npmjs.org/graceful-fs/-/graceful-fs-4.2.11.tgz",
+      "integrity": "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==",
+      "dev": true,
+      "peer": true
+    },
     "node_modules/has-flag": {
       "version": "3.0.0",
       "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-3.0.0.tgz",
@@ -5513,6 +5843,47 @@
         "node": ">=0.12.0"
       }
     },
+    "node_modules/jest-worker": {
+      "version": "27.5.1",
+      "resolved": "https://registry.npmjs.org/jest-worker/-/jest-worker-27.5.1.tgz",
+      "integrity": "sha512-7vuh85V5cdDofPyxn58nrPjBktZo0u9x1g8WtjQol+jZDaE+fhN+cIvTj11GndBnMnyfrUOG1sZQxCdjKh+DKg==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@types/node": "*",
+        "merge-stream": "^2.0.0",
+        "supports-color": "^8.0.0"
+      },
+      "engines": {
+        "node": ">= 10.13.0"
+      }
+    },
+    "node_modules/jest-worker/node_modules/has-flag": {
+      "version": "4.0.0",
+      "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz",
+      "integrity": "sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ==",
+      "dev": true,
+      "peer": true,
+      "engines": {
+        "node": ">=8"
+      }
+    },
+    "node_modules/jest-worker/node_modules/supports-color": {
+      "version": "8.1.1",
+      "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-8.1.1.tgz",
+      "integrity": "sha512-MpUEN2OodtUzxvKQl72cUF7RQ5EiHsGvSsVG0ia9c5RbWGL2CI4C7EpPS8UTBIplnlzZiNuV56w+FuNxy3ty2Q==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "has-flag": "^4.0.0"
+      },
+      "engines": {
+        "node": ">=10"
+      },
+      "funding": {
+        "url": "https://github.com/chalk/supports-color?sponsor=1"
+      }
+    },
     "node_modules/js-tokens": {
       "version": "4.0.0",
       "resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-4.0.0.tgz",
@@ -5813,6 +6184,16 @@
       "resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-6.1.0.tgz",
       "integrity": "sha512-+eawOlIgy680F0kBzPUNFhMZGtJ1YmqM6l4+Crf4IkImjYrO/mqPwRMh352g23uIaQKFItcQ64I7KMaJxHgAVA=="
     },
+    "node_modules/loader-runner": {
+      "version": "4.3.0",
+      "resolved": "https://registry.npmjs.org/loader-runner/-/loader-runner-4.3.0.tgz",
+      "integrity": "sha512-3R/1M+yS3j5ou80Me59j7F9IMs4PXs3VqRrm0TU3AbKPxlmpoY1TNscJV/oGJXo8qCatFGTfDbY6W6ipGOYXfg==",
+      "dev": true,
+      "peer": true,
+      "engines": {
+        "node": ">=6.11.5"
+      }
+    },
     "node_modules/loader-utils": {
       "version": "2.0.4",
       "resolved": "https://registry.npmjs.org/loader-utils/-/loader-utils-2.0.4.tgz",
@@ -5902,6 +6283,13 @@
       "integrity": "sha512-Lf+9+2r+Tdp5wXDXC4PcIBjTDtq4UKjCPMQhKIuzpJNW0b96kVqSwW0bT7FhRSfmAiFYgP+SCRvdrDozfh0U5w==",
       "license": "MIT"
     },
+    "node_modules/merge-stream": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/merge-stream/-/merge-stream-2.0.0.tgz",
+      "integrity": "sha512-abv/qOcuPfk3URPfDzmZU1LKmuw8kT+0nIHvKrKgFrwifol/doWcdA4ZqsWQ8ENrFKkd67Mfpo/LovbIUsbt3w==",
+      "dev": true,
+      "peer": true
+    },
     "node_modules/micromatch": {
       "version": "4.0.8",
       "resolved": "https://registry.npmjs.org/micromatch/-/micromatch-4.0.8.tgz",
@@ -5914,6 +6302,29 @@
         "node": ">=8.6"
       }
     },
+    "node_modules/mime-db": {
+      "version": "1.52.0",
+      "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
+      "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
+      "dev": true,
+      "peer": true,
+      "engines": {
+        "node": ">= 0.6"
+      }
+    },
+    "node_modules/mime-types": {
+      "version": "2.1.35",
+      "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
+      "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "mime-db": "1.52.0"
+      },
+      "engines": {
+        "node": ">= 0.6"
+      }
+    },
     "node_modules/ms": {
       "version": "2.1.2",
       "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz",
@@ -5960,9 +6371,9 @@
       }
     },
     "node_modules/nanoid": {
-      "version": "3.3.7",
-      "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.7.tgz",
-      "integrity": "sha512-eSRppjcPIatRIMC1U6UngP8XFcz8MQWGQdt1MTBQ7NaAmvXDfvNxbvWV3x2y6CdEUciCSsDHDQZbhYaB8QEo2g==",
+      "version": "3.3.8",
+      "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.8.tgz",
+      "integrity": "sha512-WNLf5Sd8oZxOm+TzppcYk8gVOgP+l58xNy58D0nbUnOxOWRWvlcCV4kUF7ltmI6PsrLl/BgKEyS4mqsGChFN0w==",
       "funding": [
         {
           "type": "github",
@@ -5976,6 +6387,13 @@
         "node": "^10 || ^12 || ^13.7 || ^14 || >=15.0.1"
       }
     },
+    "node_modules/neo-async": {
+      "version": "2.6.2",
+      "resolved": "https://registry.npmjs.org/neo-async/-/neo-async-2.6.2.tgz",
+      "integrity": "sha512-Yd3UES5mWCSqR+qNT93S3UoYUkqAZ9lLg8a7g9rimsWmYGK8cVToA4/sF3RrshdyV3sAGMXVUmpMYOw+dLpOuw==",
+      "dev": true,
+      "peer": true
+    },
     "node_modules/node-addon-api": {
       "version": "7.1.0",
       "resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-7.1.0.tgz",
@@ -6006,9 +6424,9 @@
       }
     },
     "node_modules/node-releases": {
-      "version": "2.0.14",
-      "resolved": "https://registry.npmjs.org/node-releases/-/node-releases-2.0.14.tgz",
-      "integrity": "sha512-y10wOWt8yZpqXmOgRo77WaHEmhYQYGNA6y421PKsKYWEK8aW+cqAphborZDhqfyKrbZEN92CN1X2KbafY2s7Yw=="
+      "version": "2.0.19",
+      "resolved": "https://registry.npmjs.org/node-releases/-/node-releases-2.0.19.tgz",
+      "integrity": "sha512-xxOWJsBKtzAq7DY0J+DTzuz58K8e7sJbdgwkbMWQe8UYB6ekmsQ45q0M/tJDsGaZmbC+l7n57UV8Hl5tHxO9uw=="
     },
     "node_modules/npm": {
       "version": "10.8.1",
@@ -8615,9 +9033,9 @@
       }
     },
     "node_modules/picocolors": {
-      "version": "1.0.1",
-      "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.0.1.tgz",
-      "integrity": "sha512-anP1Z8qwhkbmu7MFP5iTt+wQKXgwzf7zTyGlcdzabySa9vd0Xt392U0rVmz9poOaBj0uHJKyyo9/upk0HrEQew=="
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz",
+      "integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA=="
     },
     "node_modules/picomatch": {
       "version": "2.3.1",
@@ -8750,6 +9168,16 @@
         "node": ">=6"
       }
     },
+    "node_modules/randombytes": {
+      "version": "2.1.0",
+      "resolved": "https://registry.npmjs.org/randombytes/-/randombytes-2.1.0.tgz",
+      "integrity": "sha512-vYl3iOX+4CKUWuxGi9Ukhie6fsqXqS9FE2Zaic4tNFD2N2QQaXOMFbuKK4QmDHC0JO6B1Zp41J0LpT0oR68amQ==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "safe-buffer": "^5.1.0"
+      }
+    },
     "node_modules/react": {
       "version": "18.3.1",
       "resolved": "https://registry.npmjs.org/react/-/react-18.3.1.tgz",
@@ -8888,7 +9316,6 @@
       "version": "5.2.1",
       "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
       "integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==",
-      "dev": true,
       "funding": [
         {
           "type": "github",
@@ -8938,6 +9365,16 @@
         "semver": "bin/semver.js"
       }
     },
+    "node_modules/serialize-javascript": {
+      "version": "6.0.2",
+      "resolved": "https://registry.npmjs.org/serialize-javascript/-/serialize-javascript-6.0.2.tgz",
+      "integrity": "sha512-Saa1xPByTTq2gdeFZYLLo+RFE35NHZkAbqZeWNd3BpzppeVisAqpDjcp8dyf6uIvEqJRd46jemmyA4iFIeVk8g==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "randombytes": "^2.1.0"
+      }
+    },
     "node_modules/shallowequal": {
       "version": "1.1.0",
       "resolved": "https://registry.npmjs.org/shallowequal/-/shallowequal-1.1.0.tgz",
@@ -8959,6 +9396,17 @@
         "node": ">=0.10.0"
       }
     },
+    "node_modules/source-map-support": {
+      "version": "0.5.21",
+      "resolved": "https://registry.npmjs.org/source-map-support/-/source-map-support-0.5.21.tgz",
+      "integrity": "sha512-uBHU3L3czsIyYXKX88fdrGovxdSCoTGDRZ6SYXtSRxLZUzHg5P/66Ht6uoUlHu9EZod+inXhKo3qQgwXUT/y1w==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "buffer-from": "^1.0.0",
+        "source-map": "^0.6.0"
+      }
+    },
     "node_modules/stable": {
       "version": "0.1.8",
       "resolved": "https://registry.npmjs.org/stable/-/stable-0.1.8.tgz",
@@ -9025,6 +9473,16 @@
       "resolved": "https://registry.npmjs.org/svg-parser/-/svg-parser-2.0.4.tgz",
       "integrity": "sha512-e4hG1hRwoOdRb37cIMSgzNsxyzKfayW6VOflrwvR+/bzrkyxY/31WkbgnQpgtrNp1SdpJvpUAGTa/ZoiPNDuRQ=="
     },
+    "node_modules/tapable": {
+      "version": "2.2.1",
+      "resolved": "https://registry.npmjs.org/tapable/-/tapable-2.2.1.tgz",
+      "integrity": "sha512-GNzQvQTOIP6RyTfE2Qxb8ZVlNmw0n88vp1szwWRimP02mnTsx3Wtn5qRdqY9w2XduFNUgvOwhNnQsjwCp+kqaQ==",
+      "dev": true,
+      "peer": true,
+      "engines": {
+        "node": ">=6"
+      }
+    },
     "node_modules/term-size": {
       "version": "2.2.1",
       "resolved": "https://registry.npmjs.org/term-size/-/term-size-2.2.1.tgz",
@@ -9037,6 +9495,86 @@
         "url": "https://github.com/sponsors/sindresorhus"
       }
     },
+    "node_modules/terser": {
+      "version": "5.37.0",
+      "resolved": "https://registry.npmjs.org/terser/-/terser-5.37.0.tgz",
+      "integrity": "sha512-B8wRRkmre4ERucLM/uXx4MOV5cbnOlVAqUst+1+iLKPI0dOgFO28f84ptoQt9HEI537PMzfYa/d+GEPKTRXmYA==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@jridgewell/source-map": "^0.3.3",
+        "acorn": "^8.8.2",
+        "commander": "^2.20.0",
+        "source-map-support": "~0.5.20"
+      },
+      "bin": {
+        "terser": "bin/terser"
+      },
+      "engines": {
+        "node": ">=10"
+      }
+    },
+    "node_modules/terser-webpack-plugin": {
+      "version": "5.3.10",
+      "resolved": "https://registry.npmjs.org/terser-webpack-plugin/-/terser-webpack-plugin-5.3.10.tgz",
+      "integrity": "sha512-BKFPWlPDndPs+NGGCr1U59t0XScL5317Y0UReNrHaw9/FwhPENlq6bfgs+4yPfyP51vqC1bQ4rp1EfXW5ZSH9w==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@jridgewell/trace-mapping": "^0.3.20",
+        "jest-worker": "^27.4.5",
+        "schema-utils": "^3.1.1",
+        "serialize-javascript": "^6.0.1",
+        "terser": "^5.26.0"
+      },
+      "engines": {
+        "node": ">= 10.13.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/webpack"
+      },
+      "peerDependencies": {
+        "webpack": "^5.1.0"
+      },
+      "peerDependenciesMeta": {
+        "@swc/core": {
+          "optional": true
+        },
+        "esbuild": {
+          "optional": true
+        },
+        "uglify-js": {
+          "optional": true
+        }
+      }
+    },
+    "node_modules/terser-webpack-plugin/node_modules/schema-utils": {
+      "version": "3.3.0",
+      "resolved": "https://registry.npmjs.org/schema-utils/-/schema-utils-3.3.0.tgz",
+      "integrity": "sha512-pN/yOAvcC+5rQ5nERGuwrjLlYvLTbCibnZ1I7B1LaiAz9BRBlE9GMgE/eqV30P7aJQUf7Ddimy/RsbYO/GrVGg==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@types/json-schema": "^7.0.8",
+        "ajv": "^6.12.5",
+        "ajv-keywords": "^3.5.2"
+      },
+      "engines": {
+        "node": ">= 10.13.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/webpack"
+      }
+    },
+    "node_modules/terser/node_modules/commander": {
+      "version": "2.20.3",
+      "resolved": "https://registry.npmjs.org/commander/-/commander-2.20.3.tgz",
+      "integrity": "sha512-GpVkmM8vF2vQUkj2LvZmD35JxeJOLCwJ9cUkugyk2nuhbv3+mJvpLYYt+0+USMxE+oj+ey/lJEnhZw75x/OMcQ==",
+      "dev": true,
+      "peer": true
+    },
     "node_modules/timsort": {
       "version": "0.3.0",
       "resolved": "https://registry.npmjs.org/timsort/-/timsort-0.3.0.tgz",
@@ -9083,7 +9621,6 @@
       "version": "5.4.5",
       "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.4.5.tgz",
       "integrity": "sha512-vcI4UpRgg81oIRUFwR0WSIHKt11nJ7SAVlYNIu+QpqeyXP+gpQJy/Z4+F0aGxSE4MqwjyXvW/TzgkLAx2AGHwQ==",
-      "dev": true,
       "bin": {
         "tsc": "bin/tsc",
         "tsserver": "bin/tsserver"
@@ -9098,6 +9635,13 @@
       "integrity": "sha512-ARDJmphmdvUk6Glw7y9DQ2bFkKBHwQHLi2lsaH6PPmz/Ka9sFOBsBluozhDltWmnv9u/cF6Rt87znRTPV+yp/A==",
       "license": "MIT"
     },
+    "node_modules/undici-types": {
+      "version": "6.20.0",
+      "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.20.0.tgz",
+      "integrity": "sha512-Ny6QZ2Nju20vw1SRHe3d9jVu6gJ+4e3+MMpqu7pqE5HT6WsTSlce++GQmK5UXS8mzV8DSYHrQH+Xrf2jVcuKNg==",
+      "dev": true,
+      "peer": true
+    },
     "node_modules/unicode-canonical-property-names-ecmascript": {
       "version": "2.0.0",
       "resolved": "https://registry.npmjs.org/unicode-canonical-property-names-ecmascript/-/unicode-canonical-property-names-ecmascript-2.0.0.tgz",
@@ -9139,9 +9683,9 @@
       }
     },
     "node_modules/update-browserslist-db": {
-      "version": "1.0.16",
-      "resolved": "https://registry.npmjs.org/update-browserslist-db/-/update-browserslist-db-1.0.16.tgz",
-      "integrity": "sha512-KVbTxlBYlckhF5wgfyZXTWnMn7MMZjMu9XG8bPlliUOP9ThaF4QnhP8qrjrH7DRzHfSk0oQv1wToW+iA5GajEQ==",
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/update-browserslist-db/-/update-browserslist-db-1.1.1.tgz",
+      "integrity": "sha512-R8UzCaa9Az+38REPiJ1tXlImTJXlVfgHZsglwBD/k6nj76ctsH1E3q4doGrukiLQd3sGQYu56r5+lo5r94l29A==",
       "funding": [
         {
           "type": "opencollective",
@@ -9157,8 +9701,8 @@
         }
       ],
       "dependencies": {
-        "escalade": "^3.1.2",
-        "picocolors": "^1.0.1"
+        "escalade": "^3.2.0",
+        "picocolors": "^1.1.0"
       },
       "bin": {
         "update-browserslist-db": "cli.js"
@@ -9184,11 +9728,101 @@
         "node": ">= 4"
       }
     },
+    "node_modules/watchpack": {
+      "version": "2.4.2",
+      "resolved": "https://registry.npmjs.org/watchpack/-/watchpack-2.4.2.tgz",
+      "integrity": "sha512-TnbFSbcOCcDgjZ4piURLCbJ3nJhznVh9kw6F6iokjiFPl8ONxe9A6nMDVXDiNbrSfLILs6vB07F7wLBrwPYzJw==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "glob-to-regexp": "^0.4.1",
+        "graceful-fs": "^4.1.2"
+      },
+      "engines": {
+        "node": ">=10.13.0"
+      }
+    },
     "node_modules/weak-lru-cache": {
       "version": "1.2.2",
       "resolved": "https://registry.npmjs.org/weak-lru-cache/-/weak-lru-cache-1.2.2.tgz",
       "integrity": "sha512-DEAoo25RfSYMuTGc9vPJzZcZullwIqRDSI9LOy+fkCJPi6hykCnfKaXTuPBDuXAUcqHXyOgFtHNp/kB2FjYHbw=="
     },
+    "node_modules/webpack": {
+      "version": "5.97.1",
+      "resolved": "https://registry.npmjs.org/webpack/-/webpack-5.97.1.tgz",
+      "integrity": "sha512-EksG6gFY3L1eFMROS/7Wzgrii5mBAFe4rIr3r2BTfo7bcc+DWwFZ4OJ/miOuHJO/A85HwyI4eQ0F6IKXesO7Fg==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@types/eslint-scope": "^3.7.7",
+        "@types/estree": "^1.0.6",
+        "@webassemblyjs/ast": "^1.14.1",
+        "@webassemblyjs/wasm-edit": "^1.14.1",
+        "@webassemblyjs/wasm-parser": "^1.14.1",
+        "acorn": "^8.14.0",
+        "browserslist": "^4.24.0",
+        "chrome-trace-event": "^1.0.2",
+        "enhanced-resolve": "^5.17.1",
+        "es-module-lexer": "^1.2.1",
+        "eslint-scope": "5.1.1",
+        "events": "^3.2.0",
+        "glob-to-regexp": "^0.4.1",
+        "graceful-fs": "^4.2.11",
+        "json-parse-even-better-errors": "^2.3.1",
+        "loader-runner": "^4.2.0",
+        "mime-types": "^2.1.27",
+        "neo-async": "^2.6.2",
+        "schema-utils": "^3.2.0",
+        "tapable": "^2.1.1",
+        "terser-webpack-plugin": "^5.3.10",
+        "watchpack": "^2.4.1",
+        "webpack-sources": "^3.2.3"
+      },
+      "bin": {
+        "webpack": "bin/webpack.js"
+      },
+      "engines": {
+        "node": ">=10.13.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/webpack"
+      },
+      "peerDependenciesMeta": {
+        "webpack-cli": {
+          "optional": true
+        }
+      }
+    },
+    "node_modules/webpack-sources": {
+      "version": "3.2.3",
+      "resolved": "https://registry.npmjs.org/webpack-sources/-/webpack-sources-3.2.3.tgz",
+      "integrity": "sha512-/DyMEOrDgLKKIG0fmvtz+4dUX/3Ghozwgm6iPp8KRhvn+eQf9+Q7GWxVNMk3+uCPWfdXYC4ExGBckIXdFEfH1w==",
+      "dev": true,
+      "peer": true,
+      "engines": {
+        "node": ">=10.13.0"
+      }
+    },
+    "node_modules/webpack/node_modules/schema-utils": {
+      "version": "3.3.0",
+      "resolved": "https://registry.npmjs.org/schema-utils/-/schema-utils-3.3.0.tgz",
+      "integrity": "sha512-pN/yOAvcC+5rQ5nERGuwrjLlYvLTbCibnZ1I7B1LaiAz9BRBlE9GMgE/eqV30P7aJQUf7Ddimy/RsbYO/GrVGg==",
+      "dev": true,
+      "peer": true,
+      "dependencies": {
+        "@types/json-schema": "^7.0.8",
+        "ajv": "^6.12.5",
+        "ajv-keywords": "^3.5.2"
+      },
+      "engines": {
+        "node": ">= 10.13.0"
+      },
+      "funding": {
+        "type": "opencollective",
+        "url": "https://opencollective.com/webpack"
+      }
+    },
     "node_modules/yallist": {
       "version": "3.1.1",
       "resolved": "https://registry.npmjs.org/yallist/-/yallist-3.1.1.tgz",

extensions/react-widget/package.json

@@ -1,6 +1,6 @@
 {
   "name": "docsgpt",
-  "version": "0.4.7",
+  "version": "0.4.9",
   "private": false,
   "description": "DocsGPT 🦖 is an innovative open-source tool designed to simplify the retrieval of information from project documentation using advanced GPT models 🤖.",
   "source": "./src/index.html",
@@ -30,9 +30,10 @@
     "styled-components": "^5"
   },
   "scripts": {
-    "build": "parcel build src/main.tsx --public-url ./",
+    "build": "parcel build src/browser.tsx --public-url ./",
     "build:react": "parcel build src/index.ts",
-    "dev": "parcel src/index.html -p 3000",
+    "serve": "parcel serve -p 3000",
+    "dev": "parcel -p 3000",
     "test": "jest",
     "lint": "eslint",
     "check": "tsc --noEmit",

extensions/react-widget/publish.sh

@@ -1,43 +1,85 @@
 #!/bin/bash
-## chmod +x publish.sh - to upgrade ownership
 set -e
-cat package.json >> package_copy.json
-cat package-lock.json >> package-lock_copy.json
+
+# Create backup of original files
+cp package.json package_original.json
+cp package-lock.json package-lock_original.json
+
+# Store the latest version after publishing
+LATEST_VERSION=""
+
 publish_package() {
-  PACKAGE_NAME=$1
-  BUILD_COMMAND=$2
-  # Update package name in package.json
-  jq --arg name "$PACKAGE_NAME" '.name=$name' package.json > temp.json && mv temp.json package.json
+    PACKAGE_NAME=$1
+    BUILD_COMMAND=$2
+    IS_REACT=$3
 
-  # Remove 'target' key if the package name is 'docsgpt-react'
-  if [ "$PACKAGE_NAME" = "docsgpt-react" ]; then
-    jq 'del(.targets)' package.json > temp.json && mv temp.json package.json
-  fi
+    echo "Preparing to publish ${PACKAGE_NAME}..."
+    
+    # Restore original package.json state before each publish
+    cp package_original.json package.json
+    cp package-lock_original.json package-lock.json
 
-  if [ -d "dist" ]; then
-    echo "Deleting existing dist directory..."
-    rm -rf dist
-  fi
+    # Update package name in package.json
+    jq --arg name "$PACKAGE_NAME" '.name=$name' package.json > temp.json && mv temp.json package.json
 
-  npm version patch
+    # Handle targets based on package type
+    if [ "$IS_REACT" = "true" ]; then
+        echo "Removing targets for React library build..."
+        jq 'del(.targets)' package.json > temp.json && mv temp.json package.json
+    fi
 
-  npm run "$BUILD_COMMAND"
+    # Clean dist directory
+    if [ -d "dist" ]; then
+        echo "Cleaning dist directory..."
+        rm -rf dist
+    fi
 
-  # Publish to npm
-  npm publish
-  # Clean up
-  mv package_copy.json package.json
-  mv package-lock_copy.json package-lock.json
-  echo "Published ${PACKAGE_NAME}"
+    # update version and store it
+    LATEST_VERSION=$(npm version patch)
+    echo "New version: ${LATEST_VERSION}"
+
+    # Build package
+    npm run "$BUILD_COMMAND"
+
+    # Replace npm publish with npm pack for testing
+    npm publish
+
+    echo "Successfully packaged ${PACKAGE_NAME}"
+
+    # Log the bundle size
+    TARBALL="${PACKAGE_NAME}-${LATEST_VERSION#v}.tgz"
+    if [ -f "$TARBALL" ]; then
+        BUNDLE_SIZE=$(du -h "$TARBALL" | cut -f1)
+        echo "Bundle size for ${PACKAGE_NAME}: ${BUNDLE_SIZE}"
+    else
+        echo "Error: ${TARBALL} not found."
+        exit 1
+    fi
 }
 
-# Publish docsgpt package
-publish_package "docsgpt" "build"
+# First publish docsgpt (HTML bundle)
+publish_package "docsgpt" "build" "false"
 
-# Publish docsgpt-react package
-publish_package "docsgpt-react" "build:react"
+# Then publish docsgpt-react (React library)
+publish_package "docsgpt-react" "build:react" "true"
 
+# Restore original state but keep the updated version
+cp package_original.json package.json
+cp package-lock_original.json package-lock.json
 
-rm -rf package_copy.json
-rm -rf package-lock_copy.json
-echo "---Process completed---"
+# Update the version in the final package.json
+jq --arg version "${LATEST_VERSION#v}" '.version=$version' package.json > temp.json && mv temp.json package.json
+
+# Run npm install to update package-lock.json with the new version
+npm install --package-lock-only
+
+# Cleanup backup files
+rm -f package_original.json
+rm -f package-lock_original.json
+rm -f temp.json
+
+echo "---Process completed---"
+echo "Final version in package.json: $(jq -r '.version' package.json)"
+echo "Final version in package-lock.json: $(jq -r '.version' package-lock.json)"
+echo "Generated test packages:"
+ls *.tgz

extensions/react-widget/src/App.tsx

@@ -1,11 +1,11 @@
 import React from "react"
 import {DocsGPTWidget} from "./components/DocsGPTWidget"
-const App = () => {
+import {SearchBar} from "./components/SearchBar"
+export const App = () => {
   return (
     <div>
+      <SearchBar/>
       <DocsGPTWidget/>
     </div>
   )
-}
-
-export default App
+}

extensions/react-widget/src/browser.tsx

@@ -0,0 +1,22 @@
+//exports browser ready methods
+
+import { createRoot } from "react-dom/client";
+
+import { DocsGPTWidget } from './components/DocsGPTWidget';
+import { SearchBar } from './components/SearchBar';
+import React from "react";
+if (typeof window !== 'undefined') {
+  const renderWidget = (elementId: string, props = {}) => {
+    const root = createRoot(document.getElementById(elementId) as HTMLElement);
+    root.render(<DocsGPTWidget {...props} />);
+  };
+  const renderSearchBar = (elementId: string, props = {}) => {
+    const root = createRoot(document.getElementById(elementId) as HTMLElement);
+    root.render(<SearchBar {...props} />);
+  };
+  (window as any).renderDocsGPTWidget = renderWidget;
+
+  (window as any).renderSearchBar = renderSearchBar;
+}
+
+export { DocsGPTWidget, SearchBar };

extensions/react-widget/src/components/DocsGPTWidget.tsx

@@ -1,9 +1,9 @@
 "use client";
 import React, { useRef } from 'react'
 import DOMPurify from 'dompurify';
-import styled, { keyframes, createGlobalStyle } from 'styled-components';
+import styled, { keyframes, css } from 'styled-components';
 import { PaperPlaneIcon, RocketIcon, ExclamationTriangleIcon, Cross2Icon } from '@radix-ui/react-icons';
-import { FEEDBACK, MESSAGE_TYPE, Query, Status, WidgetProps } from '../types/index';
+import { FEEDBACK, MESSAGE_TYPE, Query, Status, WidgetCoreProps, WidgetProps } from '../types/index';
 import { fetchAnswerStreaming, sendFeedback } from '../requests/streamingApi';
 import { ThemeProvider } from 'styled-components';
 import Like from "../assets/like.svg"
@@ -49,7 +49,86 @@ const sizesConfig = {
     maxHeight: custom.maxHeight || '70vh',
   }),
 };
+const createBox = keyframes`
+   0% {
+        transform: scale(0.6);
+      }
+      90% {
+        transform: scale(1.02);
+      }
+      100% {
+        transform: scale(1);
+      }
+`
+const closeBox = keyframes`
+  0% {
+        transform: scale(1); 
+      }
+      10% {
+        transform: scale(1.02); 
+      }
+      100% {
+        transform: scale(0.6);
+      }
+`
 
+const openContainer = keyframes`
+      0% {
+        width: 200px;
+        height: 100px;
+      }
+      100% {
+        width: ${(props) => props.theme.dimensions.width};
+        height: ${(props) => props.theme.dimensions.height};
+        border-radius: 12px;
+      }`
+const closeContainer = keyframes`
+  0% {
+        width: ${(props) => props.theme.dimensions.width};
+        height: ${(props) => props.theme.dimensions.height};
+        border-radius: 12px;
+      }
+      100% {
+        width: 200px;
+        height: 100px;
+      }
+`
+const fadeIn = keyframes`
+  from {
+        opacity: 0;
+        width: ${(props) => props.theme.dimensions.width};
+        height: ${(props) => props.theme.dimensions.height};
+        transform: scale(0.9);
+      }
+      to {
+        opacity: 1;
+        transform: scale(1);
+        width: ${(props) => props.theme.dimensions.width};
+        height: ${(props) => props.theme.dimensions.height};
+      }
+`
+
+const fadeOut = keyframes`
+  from {
+        opacity: 1;
+        width: ${(props) => props.theme.dimensions.width};
+        height: ${(props) => props.theme.dimensions.height};
+      }
+      to {
+        opacity: 0;
+        transform: scale(0.9);
+        width: ${(props) => props.theme.dimensions.width};
+        height: ${(props) => props.theme.dimensions.height};
+      }
+`
+const scaleAnimation = keyframes`
+  from {
+      transform: scale(1.2);
+      }
+      to {
+      transform: scale(1);
+      }
+`
 const Overlay = styled.div`
   position: fixed;
   top: 0;
@@ -60,53 +139,35 @@ const Overlay = styled.div`
   z-index: 999;
   transition: opacity 0.5s;
 `
+
+
 const WidgetContainer = styled.div<{ modal?: boolean, isOpen?: boolean }>`
     all: initial;
     position: fixed;
     right: ${props => props.modal ? '50%' : '10px'};
     bottom: ${props => props.modal ? '50%' : '10px'};
-    z-index: 1000;
-    display: none;
+    z-index: 1001;
     transform-origin:100% 100%;
+    display: block;
+    &.modal{
+      transform : translate(50%,50%);
+    }
     &.open {
-        animation: createBox 250ms cubic-bezier(0.25, 0.1, 0.25, 1) forwards;
+        animation: css ${createBox} 250ms cubic-bezier(0.25, 0.1, 0.25, 1) forwards;
     }
     &.close {
-      animation: closeBox 250ms cubic-bezier(0.25, 0.1, 0.25, 1) forwards;
+      animation: css ${closeBox} 250ms cubic-bezier(0.25, 0.1, 0.25, 1) forwards;
     }
-    ${props => props.modal &&
-    "transform : translate(50%,50%);"
-  }
     align-items: center;
     text-align: left;
-    @keyframes createBox {
-      0% {
-        transform: scale(0.6);
-      }
-      90% {
-        transform: scale(1.02);
-      }
-      100% {
-        transform: scale(1);
-      }
-    }
-
-    @keyframes closeBox {
-      0% {
-        transform: scale(1); 
-      }
-      10% {
-        transform: scale(1.02); 
-      }
-      100% {
-        transform: scale(0.6);
-      }
-    }
 `;
+
 const StyledContainer = styled.div<{ isOpen: boolean }>`
     all: initial;
     max-height: ${(props) => props.theme.dimensions.maxHeight};
     max-width: ${(props) => props.theme.dimensions.maxWidth};
+    width: ${(props) => props.theme.dimensions.width};
+    height: ${(props) => props.theme.dimensions.height} ;
     position: relative;
     flex-direction: column;
     justify-content: space-between;
@@ -119,68 +180,20 @@ const StyledContainer = styled.div<{ isOpen: boolean }>`
     box-shadow: 0 1px 2px rgba(0, 0, 0, 0.05), 0 2px 4px rgba(0, 0, 0, 0.1);
     padding: 26px 26px 0px 26px;
     animation: ${({ isOpen, theme }) =>
-      theme.dimensions.size === 'large'
-        ? isOpen
-          ? 'fadeIn 150ms ease-in forwards'
-          : 'fadeOut 150ms ease-in forwards'
-        : isOpen
-        ? 'openContainer 150ms ease-in forwards'
-        : 'closeContainer 250ms ease-in forwards'};
-    @keyframes openContainer {
-      0% {
-        width: 200px;
-        height: 100px;
-      }
-      100% {
-        width: ${(props) => props.theme.dimensions.width};
-        height: ${(props) => props.theme.dimensions.height};
-        border-radius: 12px;
-      }
-    }
-    @keyframes closeContainer {
-      0% {
-        width: ${(props) => props.theme.dimensions.width};
-        height: ${(props) => props.theme.dimensions.height};
-        border-radius: 12px;
-      }
-      100% {
-        width: 200px;
-        height: 100px;
-      }
-    }
-    @keyframes fadeIn {
-      from {
-        opacity: 0;
-        width: ${(props) => props.theme.dimensions.width};
-        height: ${(props) => props.theme.dimensions.height};
-        transform: scale(0.9);
-      }
-      to {
-        opacity: 1;
-        transform: scale(1);
-        width: ${(props) => props.theme.dimensions.width};
-        height: ${(props) => props.theme.dimensions.height};
-      }
-    }
-    @keyframes fadeOut {
-      from {
-        opacity: 1;
-        width: ${(props) => props.theme.dimensions.width};
-        height: ${(props) => props.theme.dimensions.height};
-      }
-      to {
-        opacity: 0;
-        transform: scale(0.9);
-        width: ${(props) => props.theme.dimensions.width};
-        height: ${(props) => props.theme.dimensions.height};
-      }
-    }
+    theme.dimensions.size === 'large'
+      ? isOpen
+        ? css`${fadeIn} 150ms ease-in forwards`
+        : css` ${fadeOut} 150ms ease-in forwards`
+      : isOpen
+        ? css`${openContainer} 150ms ease-in forwards`
+        : css`${closeContainer} 250ms ease-in forwards`};
     @media only screen and (max-width: 768px) {
       max-height: 100vh;
       max-width: 80vw;
       overflow: auto;
     }
 `;
+
 const FloatingButton = styled.div<{ bgcolor: string, hidden: boolean, isAnimatingButton: boolean }>`
     position: fixed;
     display: ${props => props.hidden ? "none" : "flex"};
@@ -198,7 +211,7 @@ const FloatingButton = styled.div<{ bgcolor: string, hidden: boolean, isAnimatin
     background: ${props => props.bgcolor};
     box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
     cursor: pointer;
-    animation: ${props => props.isAnimatingButton ? 'scaleAnimation 200ms forwards' : 'none'};
+    animation: ${props => props.isAnimatingButton ? css`${scaleAnimation} 200ms forwards` : 'none'};
     &:hover {
       transform: scale(1.1);
       transition: transform 0.2s ease-in-out;
@@ -206,15 +219,6 @@ const FloatingButton = styled.div<{ bgcolor: string, hidden: boolean, isAnimatin
     &:not(:hover) {
       transition: transform 0.2s ease-in-out;
     }
-
-    @keyframes scaleAnimation {
-      from {
-      transform: scale(1.2);
-      }
-      to {
-      transform: scale(1);
-      }
-    }
 `;
 const CancelButton = styled.button`
     cursor: pointer;
@@ -478,7 +482,47 @@ const Hero = ({ title, description, theme }: { title: string, description: strin
     </HeroContainer>
   );
 };
-export const DocsGPTWidget = ({
+export const DocsGPTWidget = (props: WidgetProps) => {
+
+  const {
+    buttonIcon = 'https://d3dg1063dc54p9.cloudfront.net/widget/chat.svg',
+    buttonText = 'Ask a question',
+    buttonBg = 'linear-gradient(to bottom right, #5AF0EC, #E80D9D)',
+    defaultOpen = false,
+    ...coreProps
+  } = props
+
+  const [open, setOpen] = React.useState<boolean>(defaultOpen);
+  const [isAnimatingButton, setIsAnimatingButton] = React.useState(false);
+  const [isFloatingButtonVisible, setIsFloatingButtonVisible] = React.useState(true);
+
+  React.useEffect(() => {
+    if (isFloatingButtonVisible)
+      setTimeout(() => setIsAnimatingButton(true), 250);
+    return () => {
+      setIsAnimatingButton(false)
+    }
+  }, [isFloatingButtonVisible])
+
+  const handleClose = () => {
+    setIsFloatingButtonVisible(true);
+    setOpen(false);
+  };
+  const handleOpen = () => {
+    setOpen(true);
+    setIsFloatingButtonVisible(false);
+  }
+  return (
+    <>
+      <FloatingButton bgcolor={buttonBg} onClick={handleOpen} hidden={!isFloatingButtonVisible} isAnimatingButton={isAnimatingButton}>
+        <img width={24} src={buttonIcon} />
+        <span>{buttonText}</span>
+      </FloatingButton>
+      <WidgetCore isOpen={open} handleClose={handleClose} {...coreProps} />
+    </>
+  )
+}
+export const WidgetCore = ({
   apiHost = 'https://gptcloud.arc53.com',
   apiKey = '82962c9a-aa77-4152-94e5-a4f84fd44c6a',
   avatar = 'https://d3dg1063dc54p9.cloudfront.net/cute-docsgpt.png',
@@ -488,25 +532,37 @@ export const DocsGPTWidget = ({
   heroDescription = 'This chatbot is built with DocsGPT and utilises GenAI, please review important information using sources.',
   size = 'small',
   theme = 'dark',
-  buttonIcon = 'https://d3dg1063dc54p9.cloudfront.net/widget/chat.svg',
-  buttonText = 'Ask a question',
-  buttonBg = 'linear-gradient(to bottom right, #5AF0EC, #E80D9D)',
   collectFeedback = true,
-  deafultOpen = false
-}: WidgetProps) => {
-  const [prompt, setPrompt] = React.useState('');
+  isOpen = false,
+  prefilledQuery = "",
+  handleClose
+}: WidgetCoreProps) => {
+  const [prompt, setPrompt] = React.useState<string>("");
+  const [mounted, setMounted] = React.useState(false);
   const [status, setStatus] = React.useState<Status>('idle');
-  const [queries, setQueries] = React.useState<Query[]>([])
-  const [conversationId, setConversationId] = React.useState<string | null>(null)
-  const [open, setOpen] = React.useState<boolean>(deafultOpen)
+  const [queries, setQueries] = React.useState<Query[]>([]);
+  const [conversationId, setConversationId] = React.useState<string | null>(null);
   const [eventInterrupt, setEventInterrupt] = React.useState<boolean>(false); //click or scroll by user while autoScrolling
-  const [isAnimatingButton, setIsAnimatingButton] = React.useState(false);
-  const [isFloatingButtonVisible, setIsFloatingButtonVisible] = React.useState(true);
-  const isBubbleHovered = useRef<boolean>(false)
-  const widgetRef = useRef<HTMLDivElement>(null)
+
+  const isBubbleHovered = useRef<boolean>(false);
   const endMessageRef = React.useRef<HTMLDivElement | null>(null);
   const md = new MarkdownIt();
 
+  React.useEffect(() => {
+    if (isOpen) {
+      setMounted(true); // Mount the component
+      appendQuery(prefilledQuery)
+    } else {
+      // Wait for animations before unmounting
+      const timeout = setTimeout(() => {
+        setMounted(false)
+      }, 250);
+      return () => clearTimeout(timeout);
+    }
+  }, [isOpen]);
+
+
+
   const handleUserInterrupt = () => {
     (status === 'loading') && setEventInterrupt(true);
   }
@@ -606,144 +662,138 @@ export const DocsGPTWidget = ({
   }
   // submit handler
   const handleSubmit = async (e: React.FormEvent<HTMLFormElement>) => {
-    e.preventDefault()
+    e.preventDefault();
+    await appendQuery(prompt)
+  }
+
+  const appendQuery = async (userQuery:string) => {
+    console.log(userQuery)
+    if(!userQuery)
+      return;
+
     setEventInterrupt(false);
-    queries.push({ prompt })
-    setPrompt('')
-    await stream(prompt)
+    queries.push({ prompt:userQuery});
+    setPrompt('');
+    await stream(userQuery);
   }
   const handleImageError = (event: React.SyntheticEvent<HTMLImageElement, Event>) => {
     event.currentTarget.src = "https://d3dg1063dc54p9.cloudfront.net/cute-docsgpt.png";
   };
-  const handleClose = () => {
-    setOpen(false);
-    setTimeout(() => {
-      if (widgetRef.current) widgetRef.current.style.display = "none";
-      setIsFloatingButtonVisible(true);
-      setIsAnimatingButton(true);
-      setTimeout(() => setIsAnimatingButton(false), 200);
-    }, 250)
-  };
-  const handleOpen = () => {
-    setOpen(true);
-    setIsFloatingButtonVisible(false);
-    if (widgetRef.current)
-      widgetRef.current.style.display = 'block'
-  }
+
   const dimensions =
     typeof size === 'object' && 'custom' in size
       ? sizesConfig.getCustom(size.custom)
       : sizesConfig[size];
-
+  if (!mounted) return null;
   return (
     <ThemeProvider theme={{ ...themes[theme], dimensions }}>
-      {open && size === 'large' &&
+      {isOpen && size === 'large' &&
         <Overlay onClick={handleClose} />
       }
-      <FloatingButton bgcolor={buttonBg} onClick={handleOpen} hidden={!isFloatingButtonVisible} isAnimatingButton={isAnimatingButton}>
-        <img width={24} src={buttonIcon} />
-        <span>{buttonText}</span>
-      </FloatingButton>
-      <WidgetContainer ref={widgetRef} className={`${size != "large" && (open ? "open" : "close")}`} modal={size == 'large'}>
-        {<StyledContainer isOpen={open}>
-          <div>
-            <CancelButton onClick={handleClose}>
-              <Cross2Icon width={24} height={24} color={theme === 'light' ? 'black' : 'white'} />
-            </CancelButton>
-            <Header>
-              <img style={{ transform: 'translateY(-5px)', maxWidth: "42px", maxHeight: "42px" }} onError={handleImageError} src={avatar} alt='docs-gpt' />
-              <ContentWrapper>
-                <Title>{title}</Title>
-                <Description>{description}</Description>
-              </ContentWrapper>
-            </Header>
-          </div>
-          <Conversation onWheel={handleUserInterrupt} onTouchMove={handleUserInterrupt}>
-            {
-              queries.length > 0 ? queries?.map((query, index) => {
-                return (
-                  <React.Fragment key={index}>
-                    {
-                      query.prompt && <MessageBubble type='QUESTION'>
-                        <Message
-                          type='QUESTION'
-                          ref={(!(query.response || query.error) && index === queries.length - 1) ? endMessageRef : null}>
-                          {query.prompt}
-                        </Message>
-                      </MessageBubble>
-                    }
-                    {
-                      query.response ? <MessageBubble onMouseOver={() => { isBubbleHovered.current = true }} type='ANSWER'>
-                        <Message
-                          type='ANSWER'
-                          ref={(index === queries.length - 1) ? endMessageRef : null}
-                        >
-                          <Markdown
-                            dangerouslySetInnerHTML={{ __html: DOMPurify.sanitize(md.render(query.response)) }}
-                          />
-                        </Message>
+      {(
+        <WidgetContainer className={`${size !== 'large' ? (isOpen ? "open" : "close") : "modal"}`} modal={size === 'large'}>
+          <StyledContainer isOpen={isOpen}>
+            <div>
+              <CancelButton onClick={handleClose}>
+                <Cross2Icon width={24} height={24} color={theme === 'light' ? 'black' : 'white'} />
+              </CancelButton>
+              <Header>
+                <img style={{ transform: 'translateY(-5px)', maxWidth: "42px", maxHeight: "42px" }} onError={handleImageError} src={avatar} alt='docs-gpt' />
+                <ContentWrapper>
+                  <Title>{title}</Title>
+                  <Description>{description}</Description>
+                </ContentWrapper>
+              </Header>
+            </div>
+            <Conversation onWheel={handleUserInterrupt} onTouchMove={handleUserInterrupt}>
+              {
+                queries.length > 0 ? queries?.map((query, index) => {
+                  return (
+                    <React.Fragment key={index}>
+                      {
+                        query.prompt && <MessageBubble type='QUESTION'>
+                          <Message
+                            type='QUESTION'
+                            ref={(!(query.response || query.error) && index === queries.length - 1) ? endMessageRef : null}>
+                            {query.prompt}
+                          </Message>
+                        </MessageBubble>
+                      }
+                      {
+                        query.response ? <MessageBubble onMouseOver={() => { isBubbleHovered.current = true }} type='ANSWER'>
+                          <Message
+                            type='ANSWER'
+                            ref={(index === queries.length - 1) ? endMessageRef : null}
+                          >
+                            <Markdown
+                              dangerouslySetInnerHTML={{ __html: DOMPurify.sanitize(md.render(query.response)) }}
+                            />
+                          </Message>
 
-                        {collectFeedback &&
-                          <Feedback>
-                            <Like
-                              style={{
-                                stroke: query.feedback == 'LIKE' ? '#8860DB' : '#c0c0c0',
-                                visibility: query.feedback == 'LIKE' ? 'visible' : 'hidden'
-                              }}
-                              fill='none'
-                              onClick={() => handleFeedback("LIKE", index)} />
-                            <Dislike
-                              style={{
-                                stroke: query.feedback == 'DISLIKE' ? '#ed8085' : '#c0c0c0',
-                                visibility: query.feedback == 'DISLIKE' ? 'visible' : 'hidden'
-                              }}
-                              fill='none'
-                              onClick={() => handleFeedback("DISLIKE", index)} />
-                          </Feedback>}
-                      </MessageBubble>
-                        : <div>
-                          {
-                            query.error ? <ErrorAlert>
+                          {collectFeedback &&
+                            <Feedback>
+                              <Like
+                                style={{
+                                  stroke: query.feedback == 'LIKE' ? '#8860DB' : '#c0c0c0',
+                                  visibility: query.feedback == 'LIKE' ? 'visible' : 'hidden'
+                                }}
+                                fill='none'
+                                onClick={() => handleFeedback("LIKE", index)} />
+                              <Dislike
+                                style={{
+                                  stroke: query.feedback == 'DISLIKE' ? '#ed8085' : '#c0c0c0',
+                                  visibility: query.feedback == 'DISLIKE' ? 'visible' : 'hidden'
+                                }}
+                                fill='none'
+                                onClick={() => handleFeedback("DISLIKE", index)} />
+                            </Feedback>}
+                        </MessageBubble>
+                          : <div>
+                            {
+                              query.error ? <ErrorAlert>
 
-                              <ExclamationTriangleIcon width={22} height={22} color='#b91c1c' />
-                              <div>
-                                <h5 style={{ margin: 2 }}>Network Error</h5>
-                                <span style={{ margin: 2, fontSize: '13px' }}>{query.error}</span>
-                              </div>
-                            </ErrorAlert>
-                              : <MessageBubble type='ANSWER'>
-                                <Message type='ANSWER' style={{ fontWeight: 600 }}>
-                                  <DotAnimation>.</DotAnimation>
-                                  <Delay delay={200}>.</Delay>
-                                  <Delay delay={400}>.</Delay>
-                                </Message>
-                              </MessageBubble>
-                          }
-                        </div>
-                    }
-                  </React.Fragment>)
-              })
-                : <Hero title={heroTitle} description={heroDescription} theme={theme} />
-            }
-          </Conversation>
-          <div>
-            <PromptContainer
-              onSubmit={handleSubmit}>
-              <StyledInput
-                value={prompt} onChange={(event) => setPrompt(event.target.value)}
-                type='text' placeholder="Ask your question" />
-              <StyledButton
-                disabled={prompt.trim().length == 0 || status !== 'idle'}>
-                <PaperPlaneIcon width={18} height={18} color='white' />
-              </StyledButton>
-            </PromptContainer>
-            <Tagline>
-              Powered by&nbsp;
-              <Hyperlink target='_blank' href='https://www.docsgpt.cloud/'>DocsGPT</Hyperlink>
-            </Tagline>
-          </div>
-        </StyledContainer>}
-      </WidgetContainer>
+                                <ExclamationTriangleIcon width={22} height={22} color='#b91c1c' />
+                                <div>
+                                  <h5 style={{ margin: 2 }}>Network Error</h5>
+                                  <span style={{ margin: 2, fontSize: '13px' }}>{query.error}</span>
+                                </div>
+                              </ErrorAlert>
+                                : <MessageBubble type='ANSWER'>
+                                  <Message type='ANSWER' style={{ fontWeight: 600 }}>
+                                    <DotAnimation>.</DotAnimation>
+                                    <Delay delay={200}>.</Delay>
+                                    <Delay delay={400}>.</Delay>
+                                  </Message>
+                                </MessageBubble>
+                            }
+                          </div>
+                      }
+                    </React.Fragment>)
+                })
+                  : <Hero title={heroTitle} description={heroDescription} theme={theme} />
+              }
+            </Conversation>
+            <div>
+              <PromptContainer
+                onSubmit={handleSubmit}>
+                <StyledInput
+                  autoFocus
+                  value={prompt} onChange={(event) => setPrompt(event.target.value)}
+                  type='text' placeholder="Ask your question" />
+                <StyledButton
+                  disabled={prompt.trim().length == 0 || status !== 'idle'}>
+                  <PaperPlaneIcon width={18} height={18} color='white' />
+                </StyledButton>
+              </PromptContainer>
+              <Tagline>
+                Powered by&nbsp;
+                <Hyperlink target='_blank' href='https://www.docsgpt.cloud/'>DocsGPT</Hyperlink>
+              </Tagline>
+            </div>
+          </StyledContainer>
+        </WidgetContainer>
+      )
+      }
     </ThemeProvider>
   )
 }

extensions/react-widget/src/components/SearchBar.tsx

@@ -0,0 +1,572 @@
+import React from 'react';
+import styled, { ThemeProvider, createGlobalStyle } from 'styled-components';
+import { WidgetCore } from './DocsGPTWidget';
+import { SearchBarProps } from '@/types';
+import { getSearchResults } from '../requests/searchAPI';
+import { Result } from '@/types';
+import MarkdownIt from 'markdown-it';
+import { getOS, processMarkdownString } from '../utils/helper';
+import DOMPurify from 'dompurify';
+import { 
+    CodeIcon, 
+    TextAlignLeftIcon,
+    HeadingIcon,
+    ReaderIcon, 
+    ListBulletIcon, 
+    QuoteIcon 
+} from '@radix-ui/react-icons';
+const themes = {
+    dark: {
+        bg: '#202124',
+        text: '#EDEDED',
+        primary: {
+            text: "#FAFAFA",
+            bg: '#111111'
+        },
+        secondary: {
+            text: "#A1A1AA",
+            bg: "#38383b"
+        }
+    },
+    light: {
+        bg: '#EAEAEA',
+        text: '#171717',
+        primary: {
+            text: "#222327",
+            bg: "#fff"
+        },
+        secondary: {
+            text: "#A1A1AA",
+            bg: "#F6F6F6"
+        }
+    }
+}
+
+const GlobalStyle = createGlobalStyle`
+  .highlight {
+    color:#007EE6;
+  }
+`;
+
+const loadGeistFont = () => {
+  const link = document.createElement('link');
+  link.href = 'https://fonts.googleapis.com/css2?family=Geist:wght@100..900&display=swap'; 
+  link.rel = 'stylesheet';
+  document.head.appendChild(link);
+};
+
+const Main = styled.div`
+    all: initial;
+    font-family: 'Geist', sans-serif;
+`
+const SearchButton = styled.button<{ inputWidth: string }>`
+    padding: 6px 6px;
+    font-family: inherit;
+    width: ${({ inputWidth }) => inputWidth};
+    border-radius: 8px;
+    display: inline;
+    color: ${props => props.theme.secondary.text};
+    outline: none;
+    border: none;
+    background-color: ${props => props.theme.secondary.bg};
+    -webkit-appearance: none;
+    -moz-appearance: none;
+    appearance: none;
+    transition: background-color 128ms linear;
+    text-align: left;
+    cursor: pointer;
+`
+
+const Container = styled.div`
+    position: relative;
+    display: inline-block;
+`
+const SearchResults = styled.div`
+    position: fixed;
+    display: flex;
+    flex-direction: column;
+    background-color: ${props => props.theme.primary.bg};
+    border: 1px solid ${props => props.theme.bg};
+    border-radius: 15px;
+    padding: 8px 0px 8px 0px;
+    width: 792px;
+    max-width: 90vw;
+    height: 396px;
+    z-index: 100;
+    left: 50%;
+    top: 50%;
+    transform: translate(-50%, -50%);
+    color: ${props => props.theme.primary.text};
+    box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1), 0 2px 4px rgba(0, 0, 0, 0.1);
+    backdrop-filter: blur(16px);
+    box-sizing: border-box;
+
+    @media only screen and (max-width: 768px) {
+        height: 80vh;
+        width: 90vw;
+    }
+`;
+
+const SearchResultsScroll = styled.div`
+    flex: 1;
+    overflow-y: auto;
+    overflow-x: hidden;
+    scrollbar-gutter: stable;
+    scrollbar-width: thin;
+    scrollbar-color: #383838 transparent;
+    padding: 0 16px;
+`;
+
+const IconTitleWrapper = styled.div`
+    display: flex;
+    align-items: center;
+    gap: 8px;
+
+    .element-icon{
+        margin: 4px;
+    }
+`;
+
+const Title = styled.h3`
+    font-size: 15px;
+    font-weight: 400;
+    color: ${props => props.theme.primary.text};
+    margin: 0;
+    overflow-wrap: break-word;
+    white-space: normal;
+    overflow: hidden;
+    text-overflow: ellipsis;
+`;
+const ContentWrapper = styled.div`
+    display: flex;
+    flex-direction: column;
+    gap: 12px; 
+`;
+const Content = styled.div`
+    display: flex;
+    margin-left: 8px;
+    flex-direction: column;
+    gap: 8px;
+    padding: 4px 0px 0px 12px;
+    font-size: 15px;
+    color: ${props => props.theme.primary.text};
+    line-height: 1.6;
+    border-left: 2px solid #585858;
+    overflow: hidden;
+`
+const ContentSegment = styled.div`
+    display: flex;
+    align-items: flex-start;
+    gap: 8px;
+    padding-right: 16px;
+    overflow-wrap: break-word;
+    white-space: normal;
+    overflow: hidden; 
+    text-overflow: ellipsis;
+`
+
+const ResultWrapper = styled.div`
+    display: flex;
+    align-items: flex-start;
+    width: 100%;
+    box-sizing: border-box;
+    padding: 8px 16px;
+    cursor: pointer;
+    background-color: ${props => props.theme.primary.bg};
+    font-family: 'Geist', sans-serif;
+    transition: background-color 0.2s;
+    border-radius: 8px;
+
+    word-wrap: break-word;
+    overflow-wrap: break-word;
+    word-break: break-word;
+    white-space: normal;
+    overflow: hidden;
+    text-overflow: ellipsis;
+
+    &:hover {
+        background-color: ${props => props.theme.bg};
+    }
+`
+const Markdown = styled.div`
+line-height:18px;
+font-size: 11px;
+white-space: pre-wrap;
+ pre {
+      padding: 8px;
+      width: 90%;
+      font-size: 11px;
+      border-radius: 6px;
+      overflow-x: auto;
+      background-color: #1B1C1F;
+      color: #fff ;
+    }
+
+    h1,h2 {
+      font-size: 14px;
+      font-weight: 600;
+      color: ${(props) => props.theme.text};
+      opacity: 0.8;
+    }
+
+
+    h3 {
+      font-size: 12px;
+    }
+
+    p {
+      margin: 0px;
+      line-height: 1.35rem;
+      font-size: 11px;
+    }
+
+    code:not(pre code) {
+      border-radius: 6px;
+      padding: 2px 2px;
+      margin: 2px;
+      font-size: 9px;
+      display: inline;
+      background-color: #646464;
+      color: #fff ;
+    }
+    img{
+        max-width: 50%;
+    }
+    code {
+      overflow-x: auto;
+    }
+    a{
+        color: #007ee6;
+    }
+`
+const Toolkit = styled.kbd`
+    position: absolute;
+    right: 4px;
+    top: 50%;
+    transform: translateY(-50%);
+    background-color: ${(props) => props.theme.primary.bg};
+    color: ${(props) => props.theme.secondary.text};
+    font-weight: 600;
+    font-size: 10px;
+    padding: 3px 6px;
+    border: 1px solid ${(props) => props.theme.secondary.text};
+    border-radius: 4px;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    z-index: 1;
+    pointer-events: none;
+`
+const Loader = styled.div`
+  margin: 2rem auto;
+  border: 4px solid ${props => props.theme.secondary.text};
+  border-top: 4px solid ${props => props.theme.primary.bg};
+  border-radius: 50%;
+  width: 12px;
+  height: 12px;
+  animation: spin 1s linear infinite;
+
+  @keyframes spin {
+    0% {
+      transform: rotate(0deg);
+    }
+    100% {
+      transform: rotate(360deg);
+    }
+  }
+`;
+
+const NoResults = styled.div`
+  margin-top: 2rem;
+  text-align: center;
+  font-size: 14px;
+  color: #888;
+`;
+const AskAIButton = styled.button`
+    display: flex;
+    align-items: center;
+    justify-content: flex-start;
+    gap: 12px;
+    width: calc(100% - 32px);
+    margin: 0 16px 16px 16px;
+    box-sizing: border-box;
+    height: 50px;
+    padding: 8px 24px;
+    border: none;
+    border-radius: 6px;
+    background-color: ${props => props.theme.bg};
+    color: ${props => props.theme.text}; 
+    cursor: pointer;
+    transition: background-color 0.2s, box-shadow 0.2s;
+    font-size: 16px;
+
+    &:hover {
+        opacity: 0.8;
+    }
+`
+const SearchHeader = styled.div`
+    display: flex;
+    align-items: center;
+    gap: 8px;
+    margin-bottom: 12px;
+    padding-bottom: 12px;
+    border-bottom: 1px solid ${props => props.theme.bg};
+`
+
+const TextField = styled.input`
+    width: calc(100% - 32px);
+    margin: 0 16px;
+    padding: 12px 16px;
+    border: none;
+    background-color: transparent;
+    color: ${props => props.theme.text}; 
+    font-size: 20px;
+    font-weight: 400; 
+    outline: none;
+    
+    &:focus {
+        border-color: none;
+    }
+`
+
+const EscapeInstruction = styled.kbd`
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    margin: 12px 16px 0;
+    padding: 4px 8px;
+    border-radius: 4px;
+    background-color: transparent;
+    border: 1px solid ${props => props.theme.secondary.text};
+    color: ${props => props.theme.text};
+    font-size: 12px;
+    font-family: 'Geist', sans-serif;
+    white-space: nowrap;
+    cursor: pointer;
+    width: fit-content;
+    &:hover {
+        background-color: rgba(255, 255, 255, 0.1);
+    }
+`
+export const SearchBar = ({
+    apiKey = "74039c6d-bff7-44ce-ae55-2973cbf13837",
+    apiHost = "https://gptcloud.arc53.com",
+    theme = "dark",
+    placeholder = "Search or Ask AI...",
+    width = "256px",
+    buttonText = "Search here"
+}: SearchBarProps) => {
+    const [input, setInput] = React.useState<string>("");
+    const [loading, setLoading] = React.useState<boolean>(false);
+    const [isWidgetOpen, setIsWidgetOpen] = React.useState<boolean>(false);
+    const inputRef = React.useRef<HTMLInputElement>(null);
+    const containerRef = React.useRef<HTMLInputElement>(null);
+    const [isResultVisible, setIsResultVisible] = React.useState<boolean>(false);
+    const [results, setResults] = React.useState<Result[]>([]);
+    const debounceTimeout = React.useRef<ReturnType<typeof setTimeout> | null>(null);
+    const abortControllerRef = React.useRef<AbortController | null>(null);
+    const browserOS = getOS();
+    const isTouch = 'ontouchstart' in window;
+    
+    const getKeyboardInstruction = () => {
+        if (isResultVisible) return "Enter";
+        return browserOS === 'mac' ? '⌘ + K' : 'Ctrl + K';
+    };
+
+    React.useEffect(() => {
+        loadGeistFont()
+        const handleClickOutside = (event: MouseEvent) => {
+            if (containerRef.current && !containerRef.current.contains(event.target as Node)) {
+                setIsResultVisible(false);
+            }
+        };
+
+        const handleKeyDown = (event: KeyboardEvent) => {
+            if (
+                ((browserOS === 'win' || browserOS === 'linux') && event.ctrlKey && event.key === 'k') ||
+                (browserOS === 'mac' && event.metaKey && event.key === 'k')
+            ) {
+                event.preventDefault();
+                inputRef.current?.focus();
+                setIsResultVisible(true);
+            } else if (event.key === 'Escape') {
+                setIsResultVisible(false);
+            }
+        };
+
+     
+        document.addEventListener('mousedown', handleClickOutside);
+        document.addEventListener('keydown', handleKeyDown);
+        return () => {
+            document.removeEventListener('mousedown', handleClickOutside);
+            document.removeEventListener('keydown', handleKeyDown);
+        };
+    }, []);
+
+    React.useEffect(() => {
+        if (!input) {
+            setResults([]);
+            return;
+        }
+        setLoading(true);
+        if (debounceTimeout.current) {
+            clearTimeout(debounceTimeout.current);
+        }
+
+        if (abortControllerRef.current) {
+            abortControllerRef.current.abort();
+        }
+        const abortController = new AbortController();
+        abortControllerRef.current = abortController;
+
+        debounceTimeout.current = setTimeout(() => {
+            getSearchResults(input, apiKey, apiHost, abortController.signal)
+                .then((data) => setResults(data))
+                .catch((err) => !abortController.signal.aborted && console.log(err))
+                .finally(() => setLoading(false));
+        }, 500);
+
+        return () => {
+            abortController.abort();
+            clearTimeout(debounceTimeout.current ?? undefined);
+        };
+    }, [input])
+
+    const handleKeyDown = (event: React.KeyboardEvent<HTMLInputElement>) => {
+        if (event.key === 'Enter') {
+            event.preventDefault();
+            openWidget();
+        }
+    };
+
+    const openWidget = () => {
+        setIsWidgetOpen(true);
+        setIsResultVisible(false);
+    };
+
+    const handleClose = () => {
+        setIsWidgetOpen(false);
+        setIsResultVisible(true);
+    };
+
+    return (
+        <ThemeProvider theme={{ ...themes[theme] }}>
+            <Main>
+                <GlobalStyle />
+                <Container ref={containerRef}>
+                    <SearchButton
+                        onClick={() => setIsResultVisible(true)}
+                        inputWidth={width}
+                    >
+                        {buttonText}
+                    </SearchButton>
+                    {
+                        isResultVisible && (
+                            <SearchResults>
+                                <SearchHeader>
+                                    <TextField
+                                        ref={inputRef}
+                                        value={input}
+                                        onChange={(e) => setInput(e.target.value)}
+                                        onKeyDown={(e) => handleKeyDown(e)}
+                                        placeholder={placeholder}
+                                        autoFocus
+                                    />
+                                    <EscapeInstruction onClick={() => setIsResultVisible(false)}>
+                                        Esc
+                                    </EscapeInstruction>
+                                </SearchHeader>
+                                <AskAIButton onClick={openWidget}>
+                                    <img 
+                                        src="https://d3dg1063dc54p9.cloudfront.net/cute-docsgpt.png" 
+                                        alt="DocsGPT"
+                                        width={24}
+                                        height={24}
+                                    />
+                                    <span>Ask the AI</span>
+                                </AskAIButton>
+                                <SearchResultsScroll>
+                                    {!loading ? (
+                                        results.length > 0 ? (
+                                            results.map((res, key) => {
+                                                const containsSource = res.source !== 'local';
+                                                const processedResults = processMarkdownString(res.text, input);
+                                                if (processedResults)
+                                                    return (
+                                                        <ResultWrapper
+                                                            key={key}
+                                                            onClick={() => {
+                                                                if (!containsSource) return;
+                                                                window.open(res.source, '_blank', 'noopener, noreferrer');
+                                                            }}
+                                                        >
+                                                            <div style={{ flex: 1 }}>
+                                                                <ContentWrapper>
+                                                                    <IconTitleWrapper>
+                                                                        <ReaderIcon className="title-icon" />
+                                                                        <Title>{res.title}</Title>
+                                                                    </IconTitleWrapper>
+                                                                    <Content>
+                                                                        {processedResults.map((element, index) => (
+                                                                            <ContentSegment key={index}>
+                                                                                <IconTitleWrapper>
+                                                                                    {element.tag === 'code' && <CodeIcon className="element-icon" />}
+                                                                                    {(element.tag === 'bulletList' || element.tag === 'numberedList') && <ListBulletIcon className="element-icon" />}
+                                                                                    {element.tag === 'text' && <TextAlignLeftIcon className="element-icon" />}
+                                                                                    {element.tag === 'heading' && <HeadingIcon className="element-icon" />}
+                                                                                    {element.tag === 'blockquote' && <QuoteIcon className="element-icon" />}
+                                                                                </IconTitleWrapper>
+                                                                                <div
+                                                                                    style={{ flex: 1 }}
+                                                                                    dangerouslySetInnerHTML={{
+                                                                                        __html: DOMPurify.sanitize(element.content),
+                                                                                    }}
+                                                                                />
+                                                                            </ContentSegment>
+                                                                        ))}
+                                                                    </Content>
+                                                                </ContentWrapper>
+                                                            </div>
+                                                        </ResultWrapper>
+                                                    );
+                                                return null;
+                                            })
+                                        ) : (
+                                            <NoResults>No results found</NoResults>
+                                        )
+                                    ) : (
+                                        <Loader />
+                                    )}
+                                </SearchResultsScroll>
+                            </SearchResults>
+                        )
+                    }
+                    {
+                        isTouch ?
+
+                            <Toolkit
+                                onClick={() => {
+                                    setIsWidgetOpen(true)
+                                }}
+                                title={"Tap to Ask the AI"}>
+                                Tap
+                            </Toolkit>
+                            :
+                            <Toolkit
+                                title={getKeyboardInstruction() === "Enter" ? "Press Enter to Ask AI" : ""}>
+                                {getKeyboardInstruction()}
+                            </Toolkit>
+                    }
+                </Container>
+                <WidgetCore
+                    theme={theme}
+                    apiHost={apiHost}
+                    apiKey={apiKey}
+                    prefilledQuery={input}
+                    isOpen={isWidgetOpen}
+                    handleClose={handleClose} size={"large"}
+                />
+            </Main>
+        </ThemeProvider>
+    )
+}

extensions/react-widget/src/index.html

@@ -9,11 +9,11 @@
   <body>
     <div id="app"></div>
     <script type="module" src="main.tsx"></script>
-    <script type="module" src="../dist/main.js"></script>
-    <script type="module">
+   <!--  <script type="module">
       window.onload = function() {
         renderDocsGPTWidget('app');
+        renderSearchBar('app')
       }
-    </script>
+    </script> -->
   </body>
 </html>

extensions/react-widget/src/index.ts

@@ -1 +1,3 @@
-export { DocsGPTWidget } from "./components/DocsGPTWidget";
+//exports methods for React
+export {SearchBar} from "./components/SearchBar"
+export { DocsGPTWidget } from "./components/DocsGPTWidget";

extensions/react-widget/src/main.tsx

@@ -1,12 +1,7 @@
-import React from 'react';
-import { createRoot } from 'react-dom/client';
-import { DocsGPTWidget } from './components/DocsGPTWidget';
-
-if (typeof window !== 'undefined') {
-  const renderWidget = (elementId: string, props = {}) => {
-    const root = createRoot(document.getElementById(elementId) as HTMLElement);
-    root.render(<DocsGPTWidget {...props} />);
-  };
-  (window as any).renderDocsGPTWidget = renderWidget;
-}
-export { DocsGPTWidget };
+//development
+import { createRoot } from "react-dom/client";
+import { App } from "./App";
+import React from "react";
+const container = document.getElementById("app") as HTMLElement;
+const root = createRoot(container)
+root.render(<App />);

extensions/react-widget/src/requests/searchAPI.ts

@@ -0,0 +1,37 @@
+import { Result } from "@/types";
+
+async function getSearchResults(question: string, apiKey: string, apiHost: string, signal: AbortSignal): Promise<Result[]> {
+
+  const payload = {
+    question,
+    api_key: apiKey
+  };
+
+  try {
+    const response = await fetch(`${apiHost}/api/search`, {
+      method: "POST",
+      headers: {
+        "Content-Type": "application/json",
+      },
+      body: JSON.stringify(payload),
+      signal: signal
+    });
+
+    if (!response.ok) {
+      throw new Error(`Error: ${response.status}`);
+    }
+
+    const data: Result[] = await response.json();
+    return data;
+
+  } catch (error) {
+    if (!(error instanceof DOMException && error.name == "AbortError")) {
+      console.error("Failed to fetch documents:", error);
+    }
+    throw error;
+  }
+}
+
+export {
+  getSearchResults
+}

extensions/react-widget/src/types/index.ts

@@ -32,5 +32,26 @@ export interface WidgetProps {
   buttonText?:string;
   buttonBg?:string;
   collectFeedback?:boolean;
-  deafultOpen?: boolean;
-}
+  defaultOpen?: boolean;
+}
+export interface WidgetCoreProps extends WidgetProps { 
+  widgetRef?:React.RefObject<HTMLDivElement> | null;
+  handleClose?:React.MouseEventHandler | undefined;
+  isOpen:boolean;
+  prefilledQuery?: string;
+}
+
+export interface SearchBarProps { 
+  apiHost?: string;
+  apiKey?: string;
+  theme?: THEME;
+  placeholder?: string;
+  width?: string;
+  buttonText?: string;
+}
+
+export interface Result {
+  text:string;
+  title:string;
+  source:string;
+}

extensions/react-widget/src/utils/helper.ts

@@ -0,0 +1,151 @@
+export const getOS = () => {
+  const platform = window.navigator.platform;
+  const userAgent = window.navigator.userAgent || window.navigator.vendor;
+
+  if (/Mac/i.test(platform)) {
+    return 'mac';
+  }
+
+  if (/Win/i.test(platform)) {
+    return 'win';
+  }
+
+  if (/Linux/i.test(platform) && !/Android/i.test(userAgent)) {
+    return 'linux';
+  }
+
+  if (/Android/i.test(userAgent)) {
+    return 'android';
+  }
+
+  if (/iPhone|iPad|iPod/i.test(userAgent)) {
+    return 'ios';
+  }
+
+  return 'other';
+};
+
+interface ParsedElement {
+  content: string;
+  tag: string;
+}
+
+export const processMarkdownString = (markdown: string, keyword?: string): ParsedElement[] => {
+  const lines = markdown.trim().split('\n');
+  const keywordLower = keyword?.toLowerCase();
+
+  const escapeRegExp = (str: string) => str.replace(/[-\/\\^$*+?.()|[\]{}]/g, '\\$&');
+  const escapedKeyword = keyword ? escapeRegExp(keyword) : '';
+  const keywordRegex = keyword ? new RegExp(`(${escapedKeyword})`, 'gi') : null;
+
+  let isInCodeBlock = false;
+  let codeBlockContent: string[] = [];
+  let matchingLines: ParsedElement[] = [];
+  let firstLine: ParsedElement | null = null;
+
+  for (let i = 0; i < lines.length; i++) {
+    const trimmedLine = lines[i].trim();
+    if (!trimmedLine) continue;
+
+    if (trimmedLine.startsWith('```')) {
+      if (!isInCodeBlock) {
+        isInCodeBlock = true;
+        codeBlockContent = [];
+      } else {
+        isInCodeBlock = false;
+        const codeContent = codeBlockContent.join('\n');
+        const parsedElement: ParsedElement = {
+          content: codeContent,
+          tag: 'code'
+        };
+
+        if (!firstLine) {
+          firstLine = parsedElement;
+        }
+
+        if (keywordLower && codeContent.toLowerCase().includes(keywordLower)) {
+          parsedElement.content = parsedElement.content.replace(keywordRegex!, '<span class="highlight">$1</span>');
+          matchingLines.push(parsedElement);
+        }
+      }
+      continue;
+    }
+
+    if (isInCodeBlock) {
+      codeBlockContent.push(trimmedLine);
+      continue;
+    }
+
+    let parsedElement: ParsedElement | null = null;
+
+    const headingMatch = trimmedLine.match(/^(#{1,6})\s+(.+)$/);
+    const bulletMatch = trimmedLine.match(/^[-*]\s+(.+)$/);
+    const numberedMatch = trimmedLine.match(/^\d+\.\s+(.+)$/);
+    const blockquoteMatch = trimmedLine.match(/^>+\s*(.+)$/);
+
+    let content = trimmedLine;
+
+    if (headingMatch) {
+      content = headingMatch[2];
+      parsedElement = {
+        content: content,
+        tag: 'heading'
+      };
+    } else if (bulletMatch) {
+      content = bulletMatch[1];
+      parsedElement = {
+        content: content,
+        tag: 'bulletList'
+      };
+    } else if (numberedMatch) {
+      content = numberedMatch[1];
+      parsedElement = {
+        content: content,
+        tag: 'numberedList'
+      };
+    } else if (blockquoteMatch) {
+      content = blockquoteMatch[1];
+      parsedElement = {
+        content: content,
+        tag: 'blockquote'
+      };
+    } else {
+      parsedElement = {
+        content: content,
+        tag: 'text'
+      };
+    }
+
+    if (!firstLine) {
+      firstLine = parsedElement;
+    }
+
+    if (keywordLower && parsedElement.content.toLowerCase().includes(keywordLower)) {
+      parsedElement.content = parsedElement.content.replace(keywordRegex!, '<span class="highlight">$1</span>');
+      matchingLines.push(parsedElement);
+    }
+  }
+
+  if (isInCodeBlock && codeBlockContent.length > 0) {
+    const codeContent = codeBlockContent.join('\n');
+    const parsedElement: ParsedElement = {
+      content: codeContent,
+      tag: 'code'
+    };
+
+    if (!firstLine) {
+      firstLine = parsedElement;
+    }
+
+    if (keywordLower && codeContent.toLowerCase().includes(keywordLower)) {
+      parsedElement.content = parsedElement.content.replace(keywordRegex!, '<span class="highlight">$1</span>');
+      matchingLines.push(parsedElement);
+    }
+  }
+
+  if (keywordLower && matchingLines.length > 0) {
+    return matchingLines;
+  }
+
+  return firstLine ? [firstLine] : [];
+};

extensions/react-widget/tsconfig.json

@@ -21,7 +21,7 @@
       /* Linting */
       "strict": true,
       "noUnusedLocals": false,
-      "noUnusedParameters": true,
+      "noUnusedParameters": false,
       "noFallthroughCasesInSwitch": true,
       /* The "typeRoots" configuration specifies the locations where 
          TypeScript looks for type definitions (.d.ts files) to 

frontend/index.html

@@ -5,7 +5,7 @@
   <meta charset="UTF-8" />
   <meta name="viewport" content="width=device-width, initial-scale=1.0,viewport-fit=cover" />
   <meta name="apple-mobile-web-app-capable" content="yes">
-  <title>DocsGPT 🦖</title>
+  <title>DocsGPT</title>
   <link rel="shortcut icon" type="image/x-icon" href="/favicon.ico" />
 </head>
 

frontend/package.json

@@ -19,20 +19,21 @@
     ]
   },
   "dependencies": {
-    "@reduxjs/toolkit": "^2.2.7",
+    "@reduxjs/toolkit": "^2.5.1",
     "chart.js": "^4.4.4",
-    "i18next": "^23.15.1",
-    "i18next-browser-languagedetector": "^8.0.0",
+    "i18next": "^24.2.0",
+    "i18next-browser-languagedetector": "^8.0.2",
     "prop-types": "^15.8.1",
     "react": "^18.2.0",
-    "react-chartjs-2": "^5.2.0",
+    "react-chartjs-2": "^5.3.0",
     "react-copy-to-clipboard": "^5.1.0",
     "react-dom": "^18.3.1",
-    "react-dropzone": "^14.2.3",
-    "react-i18next": "^15.0.2",
+    "react-helmet": "^6.1.0",
+    "react-dropzone": "^14.3.5",
+    "react-i18next": "^15.4.0",
     "react-markdown": "^9.0.1",
     "react-redux": "^8.0.5",
-    "react-router-dom": "^6.8.1",
+    "react-router-dom": "^7.1.1",
     "react-syntax-highlighter": "^15.5.0",
     "rehype-katex": "^7.0.1",
     "remark-gfm": "^4.0.0",
@@ -41,28 +42,29 @@
   "devDependencies": {
     "@types/react": "^18.0.27",
     "@types/react-dom": "^18.3.0",
+    "@types/react-helmet": "^6.1.11",
     "@types/react-syntax-highlighter": "^15.5.13",
     "@typescript-eslint/eslint-plugin": "^5.51.0",
     "@typescript-eslint/parser": "^5.62.0",
-    "@vitejs/plugin-react": "^4.3.1",
+    "@vitejs/plugin-react": "^4.3.4",
     "autoprefixer": "^10.4.13",
     "eslint": "^8.57.1",
     "eslint-config-prettier": "^9.1.0",
     "eslint-config-standard-with-typescript": "^34.0.0",
-    "eslint-plugin-import": "^2.30.0",
+    "eslint-plugin-import": "^2.31.0",
     "eslint-plugin-n": "^15.7.0",
     "eslint-plugin-prettier": "^5.2.1",
     "eslint-plugin-promise": "^6.6.0",
-    "eslint-plugin-react": "^7.35.0",
+    "eslint-plugin-react": "^7.37.3",
     "eslint-plugin-unused-imports": "^4.1.4",
     "husky": "^8.0.0",
-    "lint-staged": "^15.2.10",
-    "postcss": "^8.4.41",
-    "prettier": "^3.3.3",
-    "prettier-plugin-tailwindcss": "^0.6.8",
-    "tailwindcss": "^3.4.11",
-    "typescript": "^5.6.2",
-    "vite": "^5.4.6",
+    "lint-staged": "^15.3.0",
+    "postcss": "^8.4.49",
+    "prettier": "^3.4.2",
+    "prettier-plugin-tailwindcss": "^0.6.9",
+    "tailwindcss": "^3.4.17",
+    "typescript": "^5.7.2",
+    "vite": "^5.4.14",
     "vite-plugin-svgr": "^4.2.0"
   }
 }

frontend/public/toolIcons/tool_api_tool.svg

@@ -0,0 +1,6 @@
+<?xml version="1.0" encoding="utf-8"?>
+<svg viewBox="1 6 38 28" xmlns="http://www.w3.org/2000/svg">
+  <path d="M3,33.5c-0.827,0-1.5-0.673-1.5-1.5V8c0-0.827,0.673-1.5,1.5-1.5h34c0.827,0,1.5,0.673,1.5,1.5v24 c0,0.827-0.673,1.5-1.5,1.5H3z" style="fill: rgb(7, 106, 255);"/>
+  <path d="M37,7c0.551,0,1,0.449,1,1v24c0,0.551-0.449,1-1,1H3c-0.551,0-1-0.449-1-1V8c0-0.551,0.449-1,1-1 H37 M37,6H3C1.895,6,1,6.895,1,8v24c0,1.105,0.895,2,2,2h34c1.105,0,2-0.895,2-2V8C39,6.895,38.105,6,37,6L37,6z" style="fill: rgb(7, 106, 255);"/>
+  <path d="M 19.296 13.226 C 20.066 13.06 21.108 12.955 22.147 12.955 C 23.772 12.955 25.153 13.185 26.047 14.038 C 26.88 14.766 27.255 15.931 27.255 17.118 C 27.255 18.638 26.798 19.718 26.07 20.489 C 25.196 21.426 23.801 21.842 22.656 21.842 C 22.47 21.842 22.302 21.842 22.115 21.821 L 22.115 27.045 L 19.297 27.045 L 19.297 13.226 L 19.296 13.226 Z M 22.114 19.616 C 22.259 19.637 22.405 19.637 22.571 19.637 C 23.945 19.637 24.55 18.657 24.55 17.347 C 24.55 16.119 24.049 15.162 22.78 15.162 C 22.532 15.162 22.281 15.203 22.114 15.266 L 22.114 19.616 Z M 29.158 12.955 L 31.976 12.955 L 31.976 27.045 L 29.158 27.045 L 29.158 12.955 Z M 15.001 27.045 L 17.887 27.045 L 14.91 12.955 L 11.342 12.955 L 8.024 27.045 L 10.91 27.045 L 11.524 24.227 L 14.408 24.227 L 15.001 27.045 Z M 13 15.547 L 13.068 15.547 C 13.205 16.467 13.409 17.888 13.568 18.745 L 14.021 21.409 L 11.942 21.409 L 12.457 18.746 C 12.614 17.93 12.841 16.488 13 15.547 Z" style="fill: rgb(255, 255, 255);"/>
+</svg>

frontend/public/toolIcons/tool_cryptoprice.svg

@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 122.88 122.88"><path d="M17.89 0h88.9c8.85 0 16.1 7.24 16.1 16.1v90.68c0 8.85-7.24 16.1-16.1 16.1H16.1c-8.85 0-16.1-7.24-16.1-16.1v-88.9C0 8.05 8.05 0 17.89 0zm57.04 66.96l16.46 4.96c-1.1 4.61-2.84 8.47-5.23 11.56-2.38 3.1-5.32 5.43-8.85 7-3.52 1.57-8.01 2.36-13.45 2.36-6.62 0-12.01-.96-16.21-2.87-4.19-1.92-7.79-5.3-10.83-10.13-3.04-4.82-4.57-11.02-4.57-18.54 0-10.04 2.67-17.76 8.02-23.17 5.36-5.39 12.93-8.09 22.71-8.09 7.65 0 13.68 1.54 18.06 4.64 4.37 3.1 7.64 7.85 9.76 14.27l-16.55 3.66c-.58-1.84-1.19-3.18-1.82-4.03-1.06-1.43-2.35-2.53-3.86-3.3-1.53-.78-3.22-1.16-5.11-1.16-4.27 0-7.54 1.71-9.8 5.12-1.71 2.53-2.57 6.52-2.57 11.94 0 6.73 1.02 11.33 3.07 13.83 2.05 2.49 4.92 3.73 8.63 3.73 3.59 0 6.31-1 8.15-3.03 1.83-1.99 3.16-4.92 3.99-8.75z" fill-rule="evenodd" clip-rule="evenodd"/></svg>

frontend/public/toolIcons/tool_postgres.svg

@@ -0,0 +1,29 @@
+<?xml version="1.0" encoding="utf-8"?><!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools -->
+<svg width="800px" height="800px" viewBox="-8.78 0 70 70" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns="http://www.w3.org/2000/svg">
+    <metadata>
+        <rdf:RDF>
+            <cc:Work>
+                <dc:subject>
+                    Data
+                </dc:subject>
+                <dc:identifier>
+                    sql-database-generic
+                </dc:identifier>
+                <dc:title>
+                    SQL Database (Generic)
+                </dc:title>
+                <dc:format>
+                    image/svg+xml
+                </dc:format>
+                <dc:publisher>
+                    Amido Limited
+                </dc:publisher>
+                <dc:creator>
+                    Richard Slater
+                </dc:creator>
+                <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+            </cc:Work>
+        </rdf:RDF>
+    </metadata>
+    <path d="m 852.97077,1013.9363 c -6.55238,-0.4723 -13.02857,-2.1216 -17.00034,-4.3296 -2.26232,-1.2576 -3.98589,-2.8032 -4.66223,-4.1807 l -0.4024,-0.8196 0,-25.70807 0,-25.7081 0.31843,-0.6465 c 1.42297,-2.889 5.96432,-5.4935 12.30378,-7.0562 2.15195,-0.5305 5.2586,-1.0588 7.79304,-1.3252 2.58797,-0.2721 9.44765,-0.2307 12.02919,0.073 6.86123,0.8061 12.69967,2.6108 16.29768,5.0377 1.38756,0.9359 2.81137,2.4334 3.29371,3.4642 l 0.41358,0.8838 -0.0354,25.6303 -0.0354,25.63047 -0.33195,0.6744 c -0.18257,0.3709 -0.73406,1.1007 -1.22553,1.6216 -2.99181,3.1715 -9.40919,5.5176 -17.8267,6.5172 -1.71567,0.2038 -9.16916,0.3686 -10.92937,0.2417 z m 12.07501,-22.02839 c -0.0252,-0.0657 -1.00472,-0.93831 -2.17671,-1.93922 -1.17199,-1.00091 -2.18138,-1.86687 -2.24309,-1.92436 -0.0617,-0.0575 0.15481,-0.26106 0.48117,-0.45237 0.32635,-0.19131 0.95163,-0.7235 1.3895,-1.18265 1.2805,-1.34272 1.88466,-3.00131 1.88466,-5.17388 0,-2.1388 -0.65162,-3.8645 -1.95671,-5.1818 -1.31533,-1.3278 -2.82554,-1.8983 -5.02486,-1.8983 -3.39007,0 -5.99368,1.9781 -6.82468,5.1851 -0.28586,1.1031 -0.28432,3.33211 0.003,4.31023 0.74941,2.55136 2.79044,4.40434 5.33062,4.83946 0.8596,0.14724 0.97605,0.21071 1.5621,0.85144 0.34829,0.38078 1.06301,1.14085 1.58827,1.68904 l 0.95501,0.9967 2.53878,0 c 1.39633,0 2.51816,-0.0537 2.49296,-0.11939 z m -8.70653,-7.10848 c -0.61119,-0.31868 -0.84225,-0.56599 -1.19079,-1.27453 -0.26919,-0.54724 -0.31522,-0.85851 -0.31824,-2.15197 -0.003,-1.3143 0.0388,-1.5983 0.31987,-2.169 0.45985,-0.9339 1.09355,-1.376 2.07384,-1.4469 1.36454,-0.099 2.15217,0.5707 2.56498,2.1801 0.50612,1.97321 -0.0504,4.07107 -1.26471,4.76729 -0.63707,0.36527 -1.58737,0.40659 -2.18495,0.095 z m -11.25315,3.66269 c 2.66179,-0.5048 4.1728,-2.0528 4.1728,-4.27495 0,-1.97137 -0.97548,-3.12004 -3.6716,-4.32364 -1.54338,-0.689 -2.10241,-1.1215 -2.10241,-1.6268 0,-0.4188 0.53052,-0.8777 1.14813,-0.993 0.60302,-0.1126 2.20237,0.1652 3.14683,0.5467 l 0.79167,0.3198 0,-1.7524 0,-1.7525 -0.85923,-0.1906 c -0.53103,-0.1178 -1.64689,-0.1885 -2.92137,-0.1849 -1.80528,0 -2.15881,0.044 -2.83818,0.3138 -1.98445,0.7878 -2.92613,2.1298 -2.91107,4.1485 0.0141,1.8898 1.01108,3.06864 3.49227,4.12912 1.46399,0.62572 2.05076,1.10218 2.05076,1.66522 0,1.1965 -1.99362,1.34375 -4.10437,0.30315 -0.57805,-0.28498 -1.09739,-0.54137 -1.1541,-0.56976 -0.0567,-0.0284 -0.10311,0.79023 -0.10311,1.81917 0,1.86239 0.002,1.87137 0.33919,1.99974 1.26979,0.48278 4.07626,0.69787 5.52379,0.42335 z m 30.4308,-1.72766 0,-1.58098 -2.40584,0 -2.40583,0 0,-5.43035 0,-5.4303 -2.13089,0 -2.13088,0 0,7.0113 0,7.01131 4.53672,0 4.53672,0 0,-1.58098 z m -14.84745,-27.70503 c 4.23447,-0.2937 7.4086,-0.8482 10.20178,-1.7821 2.78264,-0.9304 4.42643,-2.0562 4.79413,-3.2834 0.14166,-0.4729 0.13146,-0.6523 -0.0665,-1.1708 -0.88775,-2.3245 -5.84694,-4.1104 -13.42493,-4.8345 -3.24154,-0.3098 -9.13671,-0.2094 -12.22745,0.2081 -4.71604,0.6372 -8.54333,1.8208 -10.2451,3.1683 -3.44251,2.726 0.19793,5.7242 8.66397,7.1354 3.67084,0.6119 8.42674,0.828 12.30414,0.559 z" fill="#00bcf2" transform="translate(-830.906 -943.981)"/>
+</svg>

frontend/public/toolIcons/tool_telegram.svg

@@ -0,0 +1,10 @@
+<svg width="24" height="25" viewBox="0 0 24 25" fill="none" xmlns="http://www.w3.org/2000/svg">
+<path d="M12 0.5C8.81812 0.5 5.76375 1.76506 3.51562 4.01469C1.2652 6.26522 0.000643966 9.31734 0 12.5C0 15.6813 1.26562 18.7357 3.51562 20.9853C5.76375 23.2349 8.81812 24.5 12 24.5C15.1819 24.5 18.2362 23.2349 20.4844 20.9853C22.7344 18.7357 24 15.6813 24 12.5C24 9.31869 22.7344 6.26431 20.4844 4.01469C18.2362 1.76506 15.1819 0.5 12 0.5Z" fill="url(#paint0_linear_5586_9958)"/>
+<path d="M5.43282 12.373C8.93157 10.849 11.2641 9.8443 12.4303 9.3588C15.7641 7.97261 16.4559 7.73186 16.9078 7.7237C17.0072 7.72211 17.2284 7.74667 17.3728 7.86339C17.4928 7.96183 17.5266 8.09495 17.5434 8.18842C17.5584 8.2818 17.5791 8.49461 17.5622 8.66074C17.3822 10.5582 16.6003 15.1629 16.2028 17.2882C16.0359 18.1874 15.7041 18.4889 15.3834 18.5184C14.6859 18.5825 14.1572 18.0579 13.4822 17.6155C12.4266 16.9231 11.8303 16.4922 10.8047 15.8167C9.6197 15.0359 10.3884 14.6067 11.0634 13.9055C11.2397 13.7219 14.3109 10.9291 14.3691 10.6758C14.3766 10.6441 14.3841 10.526 14.3128 10.4637C14.2434 10.4013 14.1403 10.4227 14.0653 10.4395C13.9584 10.4635 12.2728 11.5788 9.00282 13.7851C8.52469 14.114 8.09157 14.2743 7.70157 14.2659C7.27407 14.2567 6.44907 14.0236 5.83595 13.8245C5.08595 13.5802 4.48782 13.451 4.54032 13.036C4.56657 12.82 4.8647 12.599 5.43282 12.373Z" fill="white"/>
+<defs>
+<linearGradient id="paint0_linear_5586_9958" x1="1200" y1="0.5" x2="1200" y2="2400.5" gradientUnits="userSpaceOnUse">
+<stop stop-color="#2AABEE"/>
+<stop offset="1" stop-color="#229ED9"/>
+</linearGradient>
+</defs>
+</svg>

frontend/src/Hero.tsx

@@ -37,12 +37,14 @@ export default function Hero({
               <Fragment key={key}>
                 <button
                   onClick={() => handleQuestion({ question: demo.query })}
-                  className="w-full rounded-full border border-silver px-6 py-4 text-left hover:border-gray-4000 dark:hover:border-gray-3000 xl:min-w-[24vw]"
+                  className="w-full rounded-full border border-silver px-6 py-4 text-left hover:border-gray-4000 dark:hover:border-gray-3000 xl:min-w-[24vw] bg-white dark:bg-raisin-black focus:outline-none focus:ring-2 focus:ring-purple-taupe"
                 >
-                  <p className="mb-1 font-semibold text-black dark:text-silver">
+                  <p className="mb-1 font-semibold text-black-1000 dark:text-bright-gray">
                     {demo.header}
                   </p>
-                  <span className="text-gray-400">{demo.query}</span>
+                  <span className="text-gray-700 dark:text-gray-300">
+                    {demo.query}
+                  </span>
                 </button>
               </Fragment>
             ),

frontend/src/Navigation.tsx

@@ -21,11 +21,10 @@ import {
   handleAbort,
 } from './conversation/conversationSlice';
 import ConversationTile from './conversation/ConversationTile';
-import { useDarkTheme, useMediaQuery, useOutsideAlerter } from './hooks';
+import { useDarkTheme, useMediaQuery } from './hooks';
 import useDefaultDocument from './hooks/useDefaultDocument';
 import DeleteConvModal from './modals/DeleteConvModal';
 import { ActiveState, Doc } from './models/misc';
-import APIKeyModal from './preferences/APIKeyModal';
 import { getConversations, getDocs } from './preferences/preferenceApi';
 import {
   selectApiKeyStatus,
@@ -51,21 +50,7 @@ interface NavigationProps {
   navOpen: boolean;
   setNavOpen: React.Dispatch<React.SetStateAction<boolean>>;
 }
-/* const NavImage: React.FC<{
-  Light: string | undefined;
-  Dark: string | undefined;
-}> = ({ Light, Dark }) => {
-  return (
-    <>
-      <img src={Dark} alt="icon" className="ml-2 hidden w-5 dark:block " />
-      <img src={Light} alt="icon" className="ml-2 w-5 dark:hidden filter dark:invert" />
-    </>
-  );
-};
-NavImage.propTypes = {
-  Light: PropTypes.string,
-  Dark: PropTypes.string,
-}; */
+
 export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
   const dispatch = useDispatch();
   const queries = useSelector(selectQueries);
@@ -82,8 +67,6 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
   const [isDocsListOpen, setIsDocsListOpen] = useState(false);
   const { t } = useTranslation();
   const isApiKeySet = useSelector(selectApiKeyStatus);
-  const [apiKeyModalState, setApiKeyModalState] =
-    useState<ActiveState>('INACTIVE');
 
   const [uploadModalState, setUploadModalState] =
     useState<ActiveState>('INACTIVE');
@@ -206,12 +189,6 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
         console.error(err);
       });
   }
-  useOutsideAlerter(navRef, () => {
-    if (isMobile && navOpen && apiKeyModalState === 'INACTIVE') {
-      setNavOpen(false);
-      setIsDocsListOpen(false);
-    }
-  }, [navOpen, isDocsListOpen, apiKeyModalState]);
 
   /*
     Needed to fix bug where if mobile nav was closed and then window was resized to desktop, nav would still be closed but the button to open would be gone, as per #1 on issue #146
@@ -234,7 +211,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
             >
               <img
                 src={Expand}
-                alt="menu toggle"
+                alt="Toggle navigation menu"
                 className={`${
                   !navOpen ? 'rotate-180' : 'rotate-0'
                 } m-auto transition-all duration-200`}
@@ -248,7 +225,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
               >
                 <img
                   src={openNewChat}
-                  alt="open new chat icon"
+                  alt="Start new chat"
                   className="cursor-pointer"
                 />
               </button>
@@ -277,7 +254,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
             }}
           >
             <a href="/" className="flex gap-1.5">
-              <img className="mb-2 h-10" src={DocsGPT3} alt="" />
+              <img className="mb-2 h-10" src={DocsGPT3} alt="DocsGPT Logo" />
               <p className="my-auto text-2xl font-semibold">DocsGPT</p>
             </a>
           </div>
@@ -289,7 +266,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
           >
             <img
               src={Expand}
-              alt="menu toggle"
+              alt="Toggle navigation menu"
               className={`${
                 !navOpen ? 'rotate-180' : 'rotate-0'
               } m-auto transition-all duration-200`}
@@ -312,7 +289,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
         >
           <img
             src={Add}
-            alt="new"
+            alt="Create new chat"
             className="opacity-80 group-hover:opacity-100"
           />
           <p className=" text-sm text-dove-gray group-hover:text-neutral-600 dark:text-chinese-silver dark:group-hover:text-bright-gray">
@@ -328,7 +305,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
               <img
                 src={isDarkTheme ? SpinnerDark : Spinner}
                 className="animate-spin cursor-pointer bg-transparent"
-                alt="Loading..."
+                alt="Loading conversations"
               />
             </div>
           )}
@@ -379,6 +356,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
               <img
                 className="mt-2 h-9 w-9 hover:cursor-pointer"
                 src={UploadIcon}
+                alt="Upload document"
                 onClick={() => {
                   setUploadModalState('ACTIVE');
                   if (isMobile) {
@@ -406,7 +384,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
             >
               <img
                 src={SettingGear}
-                alt="icon"
+                alt="Settings"
                 className="ml-2 w-5 filter dark:invert"
               />
               <p className="my-auto text-sm text-eerie-black  dark:text-white">
@@ -428,7 +406,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
                 >
                   <img
                     src={Discord}
-                    alt="discord"
+                    alt="Join Discord community"
                     className="m-2 w-6 self-center filter dark:invert"
                   />
                 </NavLink>
@@ -441,7 +419,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
                 >
                   <img
                     src={Twitter}
-                    alt="x"
+                    alt="Follow us on Twitter"
                     className="m-2 w-5 self-center filter dark:invert"
                   />
                 </NavLink>
@@ -454,7 +432,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
                 >
                   <img
                     src={Github}
-                    alt="github"
+                    alt="View on GitHub"
                     className="m-2 w-6 self-center filter dark:invert"
                   />
                 </NavLink>
@@ -471,18 +449,13 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
           >
             <img
               src={Hamburger}
-              alt="menu toggle"
+              alt="Toggle mobile menu"
               className="w-7 filter dark:invert"
             />
           </button>
           <div className="text-[#949494] font-medium text-[20px]">DocsGPT</div>
         </div>
       </div>
-      <APIKeyModal
-        modalState={apiKeyModalState}
-        setModalState={setApiKeyModalState}
-        isCancellable={isApiKeySet}
-      />
       <DeleteConvModal
         modalState={modalStateDeleteConv}
         setModalState={setModalStateDeleteConv}
@@ -490,8 +463,10 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
       />
       {uploadModalState === 'ACTIVE' && (
         <Upload
+          receivedFile={[]}
           setModalState={setUploadModalState}
           isOnboarding={false}
+          renderTab={null}
           close={() => setUploadModalState('INACTIVE')}
         ></Upload>
       )}

frontend/src/api/endpoints.ts

@@ -18,6 +18,12 @@ const endpoints = {
     FEEDBACK_ANALYTICS: '/api/get_feedback_analytics',
     LOGS: `/api/get_user_logs`,
     MANAGE_SYNC: '/api/manage_sync',
+    GET_AVAILABLE_TOOLS: '/api/available_tools',
+    GET_USER_TOOLS: '/api/get_tools',
+    CREATE_TOOL: '/api/create_tool',
+    UPDATE_TOOL_STATUS: '/api/update_tool_status',
+    UPDATE_TOOL: '/api/update_tool',
+    DELETE_TOOL: '/api/delete_tool',
   },
   CONVERSATION: {
     ANSWER: '/api/answer',

frontend/src/api/services/userService.ts

@@ -35,6 +35,18 @@ const userService = {
     apiClient.post(endpoints.USER.LOGS, data),
   manageSync: (data: any): Promise<any> =>
     apiClient.post(endpoints.USER.MANAGE_SYNC, data),
+  getAvailableTools: (): Promise<any> =>
+    apiClient.get(endpoints.USER.GET_AVAILABLE_TOOLS),
+  getUserTools: (): Promise<any> =>
+    apiClient.get(endpoints.USER.GET_USER_TOOLS),
+  createTool: (data: any): Promise<any> =>
+    apiClient.post(endpoints.USER.CREATE_TOOL, data),
+  updateToolStatus: (data: any): Promise<any> =>
+    apiClient.post(endpoints.USER.UPDATE_TOOL_STATUS, data),
+  updateTool: (data: any): Promise<any> =>
+    apiClient.post(endpoints.USER.UPDATE_TOOL, data),
+  deleteTool: (data: any): Promise<any> =>
+    apiClient.post(endpoints.USER.DELETE_TOOL, data),
 };
 
 export default userService;

frontend/src/assets/DragFileUpload.svg

@@ -0,0 +1,4 @@
+<svg width="72" height="72" viewBox="0 0 72 72" fill="current" xmlns="http://www.w3.org/2000/svg">
+<path d="M33.8788 9.87836C34.4413 9.31595 35.2043 9 35.9998 9C36.7952 9 37.5582 9.31595 38.1208 9.87836L50.1208 21.8784C50.6672 22.4442 50.9696 23.202 50.9628 23.9886C50.9559 24.7752 50.6404 25.5276 50.0842 26.0838C49.528 26.64 48.7755 26.9555 47.989 26.9624C47.2024 26.9692 46.4446 26.6668 45.8788 26.1204L38.9998 19.2414V47.9994C38.9998 48.795 38.6837 49.5581 38.1211 50.1207C37.5585 50.6833 36.7954 50.9994 35.9998 50.9994C35.2041 50.9994 34.441 50.6833 33.8784 50.1207C33.3158 49.5581 32.9998 48.795 32.9998 47.9994V19.2414L26.1208 26.1204C25.555 26.6668 24.7971 26.9692 24.0106 26.9624C23.224 26.9555 22.4715 26.64 21.9153 26.0838C21.3591 25.5276 21.0436 24.7752 21.0367 23.9886C21.0299 23.202 21.3323 22.4442 21.8788 21.8784L33.8788 9.87836Z" fill="#313131"/>
+<path d="M18 51C18 50.2044 17.6839 49.4413 17.1213 48.8787C16.5587 48.3161 15.7956 48 15 48C14.2044 48 13.4413 48.3161 12.8787 48.8787C12.3161 49.4413 12 50.2044 12 51V52.8C12 58.446 16.554 63 22.2 63H49.8C55.446 63 60 58.446 60 52.8V51C60 50.2044 59.6839 49.4413 59.1213 48.8787C58.5587 48.3161 57.7957 48 57 48C56.2043 48 55.4413 48.3161 54.8787 48.8787C54.3161 49.4413 54 50.2044 54 51V52.8C54 55.134 52.134 57 49.8 57H22.2C19.866 57 18 55.134 18 52.8V51Z" fill="#313131"/>
+</svg>

frontend/src/assets/circle-check.svg

@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="rgb(34 197 94)" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-circle-check"><circle cx="12" cy="12" r="10"/><path d="m9 12 2 2 4-4"/></svg>

frontend/src/assets/circle-x.svg

@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="rgb(239 68 68)" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-circle-x"><circle cx="12" cy="12" r="10"/><path d="m15 9-6 6"/><path d="m9 9 6 6"/></svg>

frontend/src/assets/cogwheel.svg

@@ -0,0 +1,3 @@
+<svg width="16" height="17" viewBox="0 0 16 17" fill="none" xmlns="http://www.w3.org/2000/svg">
+<path d="M8.00182 11.2502C7.23838 11.2502 6.50621 10.9552 5.96637 10.4301C5.42654 9.90499 5.12327 9.1928 5.12327 8.4502C5.12327 7.70759 5.42654 6.9954 5.96637 6.4703C6.50621 5.9452 7.23838 5.6502 8.00182 5.6502C8.76525 5.6502 9.49743 5.9452 10.0373 6.4703C10.5771 6.9954 10.8804 7.70759 10.8804 8.4502C10.8804 9.1928 10.5771 9.90499 10.0373 10.4301C9.49743 10.9552 8.76525 11.2502 8.00182 11.2502ZM14.1126 9.2262C14.1455 8.9702 14.1701 8.7142 14.1701 8.4502C14.1701 8.1862 14.1455 7.9222 14.1126 7.6502L15.8479 6.3462C16.0042 6.2262 16.0453 6.0102 15.9466 5.8342L14.3017 3.0662C14.203 2.8902 13.981 2.8182 13.8 2.8902L11.7522 3.6902C11.3245 3.3782 10.8804 3.1062 10.3622 2.9062L10.0579 0.786197C10.0412 0.69197 9.99076 0.606538 9.91549 0.545038C9.84022 0.483538 9.745 0.449939 9.6467 0.450197H6.35693C6.15132 0.450197 5.97861 0.594197 5.94571 0.786197L5.64141 2.9062C5.12327 3.1062 4.67915 3.3782 4.25148 3.6902L2.2036 2.8902C2.02266 2.8182 1.8006 2.8902 1.70191 3.0662L0.0570212 5.8342C-0.0498963 6.0102 -0.00054964 6.2262 0.155714 6.3462L1.89107 7.6502C1.85817 7.9222 1.8335 8.1862 1.8335 8.4502C1.8335 8.7142 1.85817 8.9702 1.89107 9.2262L0.155714 10.5542C-0.00054964 10.6742 -0.0498963 10.8902 0.0570212 11.0662L1.70191 13.8342C1.8006 14.0102 2.02266 14.0742 2.2036 14.0102L4.25148 13.2022C4.67915 13.5222 5.12327 13.7942 5.64141 13.9942L5.94571 16.1142C5.97861 16.3062 6.15132 16.4502 6.35693 16.4502H9.6467C9.85231 16.4502 10.025 16.3062 10.0579 16.1142L10.3622 13.9942C10.8804 13.7862 11.3245 13.5222 11.7522 13.2022L13.8 14.0102C13.981 14.0742 14.203 14.0102 14.3017 13.8342L15.9466 11.0662C16.0453 10.8902 16.0042 10.6742 15.8479 10.5542L14.1126 9.2262Z" fill="#747474"/>
+</svg>

frontend/src/assets/no-files-dark.svg

@@ -0,0 +1,15 @@
+<svg width="113" height="124" viewBox="0 0 113 124" fill="none" xmlns="http://www.w3.org/2000/svg">
+<circle cx="55.5" cy="71" r="53" fill="#2D2E33"/>
+<rect x="-0.599797" y="0.654564" width="43.9445" height="61.5222" rx="4.39444" transform="matrix(-0.999048 0.0436194 0.0436194 0.999048 68.9873 43.3176)" fill="#45464D" stroke="#5F6167" stroke-width="1.25556"/>
+<rect x="0.704349" y="-0.540466" width="46.4556" height="64.0333" rx="5.65" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 96.3673 40.893)" fill="#3F4147" stroke="#5F6167" stroke-width="1.25556"/>
+<path d="M94.3796 45.7849C94.7417 43.0349 92.8059 40.5122 90.0559 40.1501L55.2011 35.5614C52.4511 35.1994 49.9284 37.1352 49.5663 39.8851L48.3372 49.2212L93.1505 55.121L94.3796 45.7849Z" fill="#54555B"/>
+<rect width="1.88333" height="6.27778" rx="0.941667" transform="matrix(-0.130526 0.991445 0.991445 0.130526 40.4766 36.7888)" fill="#5F6167"/>
+<rect width="1.88333" height="6.27778" rx="0.941667" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 57.6758 26.3892)" fill="#5F6167"/>
+<rect width="1.88333" height="6.27778" rx="0.941667" transform="matrix(-0.793353 0.608761 0.608761 0.793353 46.6406 28.1023)" fill="#5F6167"/>
+<rect width="36.4111" height="2.51111" rx="1.25556" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 88.4668 57.0371)" fill="#5F6167"/>
+<rect width="36.4111" height="2.51111" rx="1.25556" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 86.8281 69.4851)" fill="#5F6167"/>
+<rect width="36.4111" height="2.51111" rx="1.25556" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 85.1895 81.9333)" fill="#5F6167"/>
+<rect width="13.1833" height="2.51111" rx="1.25556" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 87.8105 62.0164)" fill="#5F6167"/>
+<rect width="13.1833" height="2.51111" rx="1.25556" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 86.1719 74.4644)" fill="#5F6167"/>
+<rect width="13.1833" height="2.51111" rx="1.25556" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 84.5332 86.9126)" fill="#5F6167"/>
+</svg>

frontend/src/assets/no-files.svg

@@ -0,0 +1,15 @@
+<svg width="113" height="124" viewBox="0 0 113 124" fill="none" xmlns="http://www.w3.org/2000/svg">
+<circle cx="55.5" cy="71" r="53" fill="#F1F1F1" fill-opacity="0.5"/>
+<rect x="-0.599797" y="0.654564" width="43.9445" height="61.5222" rx="4.39444" transform="matrix(-0.999048 0.0436194 0.0436194 0.999048 68.9873 43.3176)" fill="#EEEEEE" stroke="#999999" stroke-width="1.25556"/>
+<rect x="0.704349" y="-0.540466" width="46.4556" height="64.0333" rx="5.65" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 96.3673 40.893)" fill="#FAFAFA" stroke="#999999" stroke-width="1.25556"/>
+<path d="M94.3796 45.7849C94.7417 43.0349 92.8059 40.5122 90.0559 40.1501L55.2011 35.5614C52.4511 35.1994 49.9284 37.1352 49.5663 39.8851L48.3372 49.2212L93.1505 55.121L94.3796 45.7849Z" fill="#EEEEEE"/>
+<rect width="1.88333" height="6.27778" rx="0.941667" transform="matrix(-0.130526 0.991445 0.991445 0.130526 40.4766 36.7888)" fill="#999999"/>
+<rect width="1.88333" height="6.27778" rx="0.941667" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 57.6758 26.3892)" fill="#999999"/>
+<rect width="1.88333" height="6.27778" rx="0.941667" transform="matrix(-0.793353 0.608761 0.608761 0.793353 46.6406 28.1023)" fill="#999999"/>
+<rect width="36.4111" height="2.51111" rx="1.25556" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 88.4668 57.0371)" fill="#DCDCDC"/>
+<rect width="36.4111" height="2.51111" rx="1.25556" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 86.8281 69.4851)" fill="#DCDCDC"/>
+<rect width="36.4111" height="2.51111" rx="1.25556" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 85.1895 81.9333)" fill="#DCDCDC"/>
+<rect width="13.1833" height="2.51111" rx="1.25556" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 87.8105 62.0164)" fill="#EEEEEE"/>
+<rect width="13.1833" height="2.51111" rx="1.25556" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 86.1719 74.4644)" fill="#EEEEEE"/>
+<rect width="13.1833" height="2.51111" rx="1.25556" transform="matrix(-0.991445 -0.130526 -0.130526 0.991445 84.5332 86.9126)" fill="#EEEEEE"/>
+</svg>

frontend/src/components/Avatar.tsx

@@ -5,7 +5,7 @@ export default function Avatar({
   size,
   className,
 }: {
-  avatar: string | ReactNode;
+  avatar: ReactNode;
   size?: 'SMALL' | 'MEDIUM' | 'LARGE';
   className: string;
 }) {

frontend/src/components/DocumentPagination.tsx

@@ -1,4 +1,5 @@
 import React, { useState } from 'react';
+import { useTranslation } from 'react-i18next';
 import SingleArrowLeft from '../assets/single-left-arrow.svg';
 import SingleArrowRight from '../assets/single-right-arrow.svg';
 import DoubleArrowLeft from '../assets/double-arrow-left.svg';
@@ -19,7 +20,11 @@ const Pagination: React.FC<PaginationProps> = ({
   onPageChange,
   onRowsPerPageChange,
 }) => {
-  const [rowsPerPageOptions] = useState([5, 10, 15, 20]);
+  const { t } = useTranslation();
+  const [isDropdownOpen, setIsDropdownOpen] = useState(false);
+  const rowsPerPageOptions = [5, 10, 20, 50];
+
+  const toggleDropdown = () => setIsDropdownOpen((prev) => !prev);
 
   const handlePreviousPage = () => {
     if (currentPage > 1) {
@@ -41,31 +46,53 @@ const Pagination: React.FC<PaginationProps> = ({
     onPageChange(totalPages);
   };
 
+  const handleSelectRowsPerPage = (rows: number) => {
+    setIsDropdownOpen(false);
+    onRowsPerPageChange(rows);
+  };
+
   return (
     <div className="flex items-center text-xs justify-end gap-4 mt-2 p-2 border-gray-200">
-      <div className="flex items-center gap-2 ">
-        <span className="text-gray-900 dark:text-gray-50">Rows per page:</span>
-        <select
-          value={rowsPerPage}
-          onChange={(e) => onRowsPerPageChange(Number(e.target.value))}
-          className="border border-gray-300 rounded px-2 py-1 dark:bg-dark-charcoal dark:text-gray-50"
-        >
-          {rowsPerPageOptions.map((option) => (
-            <option
-              className="bg-white dark:bg-dark-charcoal dark:text-gray-50"
-              key={option}
-              value={option}
-            >
-              {option}
-            </option>
-          ))}
-        </select>
+      {/* Rows per page dropdown */}
+      <div className="flex items-center gap-2 relative">
+        <span className="text-gray-900 dark:text-gray-50">
+          {t('pagination.rowsPerPage')}:
+        </span>
+        <div className="relative">
+          <button
+            onClick={toggleDropdown}
+            className="px-3 py-1 border rounded dark:bg-dark-charcoal dark:text-light-gray hover:bg-gray-200 dark:hover:bg-neutral-700"
+          >
+            {rowsPerPage}
+          </button>
+          <div
+            className={`absolute z-50 right-0 mt-1 w-28 transform  bg-white dark:bg-dark-charcoal shadow-lg ring-1 ring-black ring-opacity-5 transition-all duration-200 ease-in-out ${
+              isDropdownOpen
+                ? 'scale-100 opacity-100 block'
+                : 'scale-95 opacity-0 hidden'
+            }`}
+          >
+            {rowsPerPageOptions.map((option) => (
+              <div
+                key={option}
+                onClick={() => handleSelectRowsPerPage(option)}
+                className={`cursor-pointer px-4 py-2 text-xs hover:bg-gray-100 dark:hover:bg-neutral-700  ${
+                  rowsPerPage === option
+                    ? 'bg-gray-100 dark:bg-neutral-700 dark:text-light-gray'
+                    : 'bg-white dark:bg-dark-charcoal dark:text-light-gray'
+                }`}
+              >
+                {option}
+              </div>
+            ))}
+          </div>
+        </div>
       </div>
 
+      {/* Pagination controls */}
       <div className="text-gray-900 dark:text-gray-50">
-        Page {currentPage} of {totalPages}
+        {t('pagination.pageOf', { currentPage, totalPages })}
       </div>
-
       <div className="flex items-center gap-2 text-gray-900 dark:text-gray-50">
         <button
           onClick={handleFirstPage}
@@ -74,7 +101,7 @@ const Pagination: React.FC<PaginationProps> = ({
         >
           <img
             src={DoubleArrowLeft}
-            alt="arrow"
+            alt={t('pagination.firstPage')}
             className="dark:invert dark:sepia dark:brightness-200"
           />
         </button>
@@ -85,7 +112,7 @@ const Pagination: React.FC<PaginationProps> = ({
         >
           <img
             src={SingleArrowLeft}
-            alt="arrow"
+            alt={t('pagination.previousPage')}
             className="dark:invert dark:sepia dark:brightness-200"
           />
         </button>
@@ -96,7 +123,7 @@ const Pagination: React.FC<PaginationProps> = ({
         >
           <img
             src={SingleArrowRight}
-            alt="arrow"
+            alt={t('pagination.nextPage')}
             className="dark:invert dark:sepia dark:brightness-200"
           />
         </button>
@@ -107,7 +134,7 @@ const Pagination: React.FC<PaginationProps> = ({
         >
           <img
             src={DoubleArrowRight}
-            alt="arrow"
+            alt={t('pagination.lastPage')}
             className="dark:invert dark:sepia dark:brightness-200"
           />
         </button>

frontend/src/components/DropdownMenu.tsx

@@ -48,7 +48,7 @@ export default function DropdownMenu({
     <div className="static inline-block text-left" ref={dropdownRef}>
       <button
         onClick={handleToggle}
-        className="flex w-20 cursor-pointer flex-row items-center gap-px rounded-3xl border-purple-30/25 bg-purple-30 p-2 text-xs text-white hover:bg-[#6F3FD1] focus:outline-none"
+        className="flex w-20 cursor-pointer flex-row  gap-1 rounded-3xl border-purple-30/25 bg-purple-30 p-2 text-xs text-white hover:bg-[#6F3FD1] focus:outline-none"
       >
         {icon && <img src={icon} alt="OptionIcon" className="h-4 w-4" />}
         {selectedOption.value !== 'never' ? selectedOption.label : name}

frontend/src/components/Input.tsx

@@ -7,6 +7,7 @@ const Input = ({
   value,
   isAutoFocused = false,
   placeholder,
+  label,
   maxLength,
   className,
   colorVariant = 'silver',
@@ -26,21 +27,30 @@ const Input = ({
     thick: 'border-2',
   };
   return (
-    <input
-      className={`h-[42px] w-full rounded-full px-3 py-1 outline-none dark:bg-transparent dark:text-white ${className} ${colorStyles[colorVariant]} ${borderStyles[borderVariant]}`}
-      type={type}
-      id={id}
-      name={name}
-      autoFocus={isAutoFocused}
-      placeholder={placeholder}
-      maxLength={maxLength}
-      value={value}
-      onChange={onChange}
-      onPaste={onPaste}
-      onKeyDown={onKeyDown}
-    >
-      {children}
-    </input>
+    <div className="relative">
+      <input
+        className={`h-[42px] w-full rounded-full px-3 py-1 outline-none dark:bg-transparent dark:text-white ${className} ${colorStyles[colorVariant]} ${borderStyles[borderVariant]}`}
+        type={type}
+        id={id}
+        name={name}
+        autoFocus={isAutoFocused}
+        placeholder={placeholder}
+        maxLength={maxLength}
+        value={value}
+        onChange={onChange}
+        onPaste={onPaste}
+        onKeyDown={onKeyDown}
+      >
+        {children}
+      </input>
+      {label && (
+        <div className="absolute -top-2 left-2">
+          <span className="bg-white px-2 text-xs text-gray-4000 dark:bg-outer-space dark:text-silver">
+            {label}
+          </span>
+        </div>
+      )}
+    </div>
   );
 };
 

frontend/src/components/SettingsBar.tsx

@@ -13,6 +13,7 @@ const useTabs = () => {
     t('settings.apiKeys.label'),
     t('settings.analytics.label'),
     t('settings.logs.label'),
+    t('settings.tools.label'),
   ];
   return tabs;
 };
@@ -58,7 +59,8 @@ const SettingsBar = ({ setActiveTab, activeTab }: SettingsBarProps) => {
       <div className="md:hidden z-10">
         <button
           onClick={() => scrollTabs(-1)}
-          className="flex h-6 w-6 items-center rounded-full justify-center transition-all hover:bg-gray-100"
+          className="flex h-6 w-6 items-center rounded-full justify-center transition-all hover:bg-gray-200 dark:hover:bg-gray-700"
+          aria-label="Scroll tabs left"
         >
           <img src={ArrowLeft} alt="left-arrow" className="h-3" />
         </button>
@@ -66,16 +68,22 @@ const SettingsBar = ({ setActiveTab, activeTab }: SettingsBarProps) => {
       <div
         ref={containerRef}
         className="flex flex-nowrap overflow-x-auto no-scrollbar md:space-x-4 scroll-smooth snap-x"
+        role="tablist"
+        aria-label="Settings tabs"
       >
         {tabs.map((tab, index) => (
           <button
             key={index}
             onClick={() => setActiveTab(tab)}
-            className={`snap-start h-9 rounded-3xl px-4 font-bold hover:text-neutral-600 dark:hover:text-white/60 ${
+            className={`snap-start h-9 rounded-3xl px-4 font-bold transition-colors ${
               activeTab === tab
-                ? 'bg-neutral-100 text-neutral-600  dark:bg-dark-charcoal dark:text-white/60'
-                : 'text-gray-6000'
+                ? 'bg-neutral-200 text-neutral-900 dark:bg-dark-charcoal dark:text-white'
+                : 'text-neutral-700 hover:text-neutral-900 dark:text-neutral-400 dark:hover:text-white'
             }`}
+            role="tab"
+            aria-selected={activeTab === tab}
+            aria-controls={`${tab.toLowerCase()}-panel`}
+            id={`${tab.toLowerCase()}-tab`}
           >
             {tab}
           </button>
@@ -84,7 +92,8 @@ const SettingsBar = ({ setActiveTab, activeTab }: SettingsBarProps) => {
       <div className="md:hidden z-10">
         <button
           onClick={() => scrollTabs(1)}
-          className="flex h-6 w-6 rounded-full items-center justify-center hover:bg-gray-100"
+          className="flex h-6 w-6 rounded-full items-center justify-center hover:bg-gray-200 dark:hover:bg-gray-700"
+          aria-label="Scroll tabs right"
         >
           <img src={ArrowRight} alt="right-arrow" className="h-3" />
         </button>

frontend/src/components/ToggleSwitch.tsx

@@ -0,0 +1,58 @@
+import React from 'react';
+
+type ToggleSwitchProps = {
+  checked: boolean;
+  onChange: (checked: boolean) => void;
+  className?: string;
+  label?: string;
+  disabled?: boolean;
+  activeColor?: string;
+  inactiveColor?: string;
+  id?: string;
+};
+
+const ToggleSwitch: React.FC<ToggleSwitchProps> = ({
+  checked,
+  onChange,
+  className = '',
+  label,
+  disabled = false,
+  activeColor = 'bg-purple-30',
+  inactiveColor = 'bg-transparent',
+  id,
+}) => {
+  return (
+    <label
+      className={`cursor-pointer select-none justify-between flex flex-row items-center ${disabled ? 'opacity-50 cursor-not-allowed' : ''} ${className}`}
+      htmlFor={id}
+    >
+      {label && (
+        <span className="mr-2 text-eerie-black dark:text-white">{label}</span>
+      )}
+      <div className="relative">
+        <input
+          type="checkbox"
+          checked={checked}
+          onChange={(e) => onChange(e.target.checked)}
+          className="sr-only"
+          disabled={disabled}
+          id={id}
+        />
+        <div
+          className={`box block h-8 w-14 rounded-full border border-purple-30 ${
+            checked
+              ? `${activeColor} dark:${activeColor}`
+              : `${inactiveColor} dark:${inactiveColor}`
+          }`}
+        ></div>
+        <div
+          className={`absolute left-1 top-1 flex h-6 w-6 items-center justify-center rounded-full transition ${
+            checked ? 'translate-x-full bg-silver' : 'bg-purple-30'
+          }`}
+        ></div>
+      </div>
+    </label>
+  );
+};
+
+export default ToggleSwitch;

frontend/src/components/types/index.ts

@@ -8,6 +8,7 @@ export type InputProps = {
   maxLength?: number;
   name?: string;
   placeholder?: string;
+  label?: string;
   className?: string;
   children?: React.ReactElement;
   onChange: (

frontend/src/conversation/Conversation.tsx

@@ -1,8 +1,10 @@
-import { Fragment, useEffect, useRef, useState } from 'react';
+import { Fragment, useCallback, useEffect, useRef, useState } from 'react';
 import { useTranslation } from 'react-i18next';
 import { useDispatch, useSelector } from 'react-redux';
 import { useNavigate } from 'react-router-dom';
 import Hero from '../Hero';
+import { useDropzone } from 'react-dropzone';
+import DragFileUpload from '../assets/DragFileUpload.svg';
 import ArrowDown from '../assets/arrow-down.svg';
 import newChatIcon from '../assets/openNewChat.svg';
 import Send from '../assets/send.svg';
@@ -21,12 +23,15 @@ import { FEEDBACK, Query } from './conversationModels';
 import {
   addQuery,
   fetchAnswer,
+  resendQuery,
   selectQueries,
   selectStatus,
   setConversation,
   updateConversationId,
   updateQuery,
 } from './conversationSlice';
+import Upload from '../upload/Upload';
+import { ActiveState } from '../models/misc';
 
 export default function Conversation() {
   const queries = useSelector(selectQueries);
@@ -44,6 +49,47 @@ export default function Conversation() {
   const [isShareModalOpen, setShareModalState] = useState<boolean>(false);
   const { t } = useTranslation();
   const { isMobile } = useMediaQuery();
+  const [uploadModalState, setUploadModalState] =
+    useState<ActiveState>('INACTIVE');
+  const [files, setFiles] = useState<File[]>([]);
+  const [handleDragActive, setHandleDragActive] = useState<boolean>(false);
+
+  const onDrop = useCallback((acceptedFiles: File[]) => {
+    setUploadModalState('ACTIVE');
+    setFiles(acceptedFiles);
+    setHandleDragActive(false);
+  }, []);
+
+  const { getRootProps, getInputProps } = useDropzone({
+    onDrop,
+    noClick: true,
+    multiple: true,
+    onDragEnter: () => {
+      setHandleDragActive(true);
+    },
+    onDragLeave: () => {
+      setHandleDragActive(false);
+    },
+    maxSize: 25000000,
+    accept: {
+      'application/pdf': ['.pdf'],
+      'text/plain': ['.txt'],
+      'text/x-rst': ['.rst'],
+      'text/x-markdown': ['.md'],
+      'application/zip': ['.zip'],
+      'application/vnd.openxmlformats-officedocument.wordprocessingml.document':
+        ['.docx'],
+      'application/json': ['.json'],
+      'text/csv': ['.csv'],
+      'text/html': ['.html'],
+      'application/epub+zip': ['.epub'],
+      'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet': [
+        '.xlsx',
+      ],
+      'application/vnd.openxmlformats-officedocument.presentationml.presentation':
+        ['.pptx'],
+    },
+  });
 
   const handleUserInterruption = () => {
     if (!eventInterrupt && status === 'loading') setEventInterrupt(true);
@@ -85,27 +131,57 @@ export default function Conversation() {
   const handleQuestion = ({
     question,
     isRetry = false,
+    updated = null,
+    indx = undefined,
   }: {
     question: string;
     isRetry?: boolean;
+    updated?: boolean | null;
+    indx?: number;
   }) => {
-    question = question.trim();
-    if (question === '') return;
-    setEventInterrupt(false);
-    !isRetry && dispatch(addQuery({ prompt: question })); //dispatch only new queries
-    fetchStream.current = dispatch(fetchAnswer({ question }));
+    if (updated === true) {
+      !isRetry &&
+        dispatch(resendQuery({ index: indx as number, prompt: question })); //dispatch only new queries
+      fetchStream.current = dispatch(fetchAnswer({ question, indx }));
+    } else {
+      question = question.trim();
+      if (question === '') return;
+      setEventInterrupt(false);
+      !isRetry && dispatch(addQuery({ prompt: question })); //dispatch only new queries
+      fetchStream.current = dispatch(fetchAnswer({ question }));
+    }
   };
 
   const handleFeedback = (query: Query, feedback: FEEDBACK, index: number) => {
     const prevFeedback = query.feedback;
     dispatch(updateQuery({ index, query: { feedback } }));
-    handleSendFeedback(query.prompt, query.response!, feedback).catch(() =>
-      dispatch(updateQuery({ index, query: { feedback: prevFeedback } })),
+    handleSendFeedback(
+      query.prompt,
+      query.response!,
+      feedback,
+      conversationId as string,
+      index,
+    ).catch(() =>
+      handleSendFeedback(
+        query.prompt,
+        query.response!,
+        feedback,
+        conversationId as string,
+        index,
+      ).catch(() =>
+        dispatch(updateQuery({ index, query: { feedback: prevFeedback } })),
+      ),
     );
   };
 
-  const handleQuestionSubmission = () => {
-    if (inputRef.current?.value && status !== 'loading') {
+  const handleQuestionSubmission = (
+    updatedQuestion?: string,
+    updated?: boolean,
+    indx?: number,
+  ) => {
+    if (updated === true) {
+      handleQuestion({ question: updatedQuestion as string, updated, indx });
+    } else if (inputRef.current?.value && status !== 'loading') {
       if (lastQueryReturnedErr) {
         // update last failed query with new prompt
         dispatch(
@@ -290,6 +366,8 @@ export default function Conversation() {
                     key={`${index}QUESTION`}
                     message={query.prompt}
                     type="QUESTION"
+                    handleUpdatedQuestionSubmission={handleQuestionSubmission}
+                    questionNumber={index}
                     sources={query.sources}
                   ></ConversationBubble>
 
@@ -303,14 +381,24 @@ export default function Conversation() {
         )}
       </div>
 
-      <div className="flex w-11/12 flex-col items-end self-center rounded-2xl bg-opacity-0 z-3 sm:w-[62%] h-auto">
-        <div className="flex w-full items-center rounded-[40px] border border-silver bg-white py-1 dark:bg-raisin-black">
+      <div className="flex w-11/12 flex-col items-end self-center rounded-2xl bg-opacity-0 z-3 sm:w-[62%] h-auto py-1">
+        <div
+          {...getRootProps()}
+          className="flex w-full items-center rounded-[40px] border border-silver bg-white dark:bg-raisin-black"
+        >
+          <label htmlFor="file-upload" className="sr-only">
+            {t('modals.uploadDoc.label')}
+          </label>
+          <input {...getInputProps()} id="file-upload" />
+          <label htmlFor="message-input" className="sr-only">
+            {t('inputPlaceholder')}
+          </label>
           <textarea
-            id="inputbox"
+            id="message-input"
             ref={inputRef}
             tabIndex={1}
             placeholder={t('inputPlaceholder')}
-            className={`inputbox-style w-full overflow-y-auto overflow-x-hidden whitespace-pre-wrap rounded-full bg-transparent py-5 text-base leading-tight opacity-100 focus:outline-none dark:bg-transparent dark:text-bright-gray`}
+            className={`inputbox-style w-full overflow-y-auto overflow-x-hidden whitespace-pre-wrap rounded-full bg-transparent py-5 text-base leading-tight opacity-100 focus:outline-none dark:bg-transparent dark:text-bright-gray dark:placeholder-bright-gray dark:placeholder-opacity-50`}
             onInput={handleInput}
             onKeyDown={(e) => {
               if (e.key === 'Enter' && !e.shiftKey) {
@@ -318,19 +406,27 @@ export default function Conversation() {
                 handleQuestionSubmission();
               }
             }}
+            aria-label={t('inputPlaceholder')}
           ></textarea>
           {status === 'loading' ? (
             <img
               src={isDarkTheme ? SpinnerDark : Spinner}
               className="relative right-[38px] bottom-[24px] -mr-[30px] animate-spin cursor-pointer self-end bg-transparent"
-            ></img>
+              alt={t('loading')}
+            />
           ) : (
             <div className="mx-1 cursor-pointer rounded-full p-3 text-center hover:bg-gray-3000 dark:hover:bg-dark-charcoal">
-              <img
-                className="ml-[4px] h-6 w-6 text-white "
-                onClick={handleQuestionSubmission}
-                src={isDarkTheme ? SendDark : Send}
-              ></img>
+              <button
+                onClick={() => handleQuestionSubmission()}
+                aria-label={t('send')}
+                className="flex items-center justify-center"
+              >
+                <img
+                  className="ml-[4px] h-6 w-6 text-white"
+                  src={isDarkTheme ? SendDark : Send}
+                  alt={t('send')}
+                />
+              </button>
             </div>
           )}
         </div>
@@ -339,6 +435,26 @@ export default function Conversation() {
           {t('tagline')}
         </p>
       </div>
+      {handleDragActive && (
+        <div className="pointer-events-none fixed top-0 left-0 z-30 flex flex-col size-full items-center justify-center bg-opacity-50 bg-white dark:bg-gray-alpha">
+          <img className="filter dark:invert" src={DragFileUpload} />
+          <span className="px-2 text-2xl font-bold text-outer-space dark:text-silver">
+            {t('modals.uploadDoc.drag.title')}
+          </span>
+          <span className="p-2 text-s w-48 text-center text-outer-space dark:text-silver">
+            {t('modals.uploadDoc.drag.description')}
+          </span>
+        </div>
+      )}
+      {uploadModalState === 'ACTIVE' && (
+        <Upload
+          receivedFile={files}
+          setModalState={setUploadModalState}
+          isOnboarding={false}
+          renderTab={'file'}
+          close={() => setUploadModalState('INACTIVE')}
+        ></Upload>
+      )}
     </div>
   );
 }

frontend/src/conversation/ConversationBubble.tsx

@@ -1,6 +1,6 @@
 import 'katex/dist/katex.min.css';
 
-import { forwardRef, useState } from 'react';
+import { forwardRef, useRef, useState } from 'react';
 import ReactMarkdown from 'react-markdown';
 import { useSelector } from 'react-redux';
 import { Prism as SyntaxHighlighter } from 'react-syntax-highlighter';
@@ -8,6 +8,7 @@ import { vscDarkPlus } from 'react-syntax-highlighter/dist/cjs/styles/prism';
 import rehypeKatex from 'rehype-katex';
 import remarkGfm from 'remark-gfm';
 import remarkMath from 'remark-math';
+import { useTranslation } from 'react-i18next';
 
 import DocsGPT3 from '../assets/cute_docsgpt3.svg';
 import Dislike from '../assets/dislike.svg?react';
@@ -15,6 +16,8 @@ import Document from '../assets/document.svg';
 import Like from '../assets/like.svg?react';
 import Link from '../assets/link.svg';
 import Sources from '../assets/sources.svg';
+import Edit from '../assets/edit.svg';
+import UserIcon from '../assets/user.png';
 import Avatar from '../components/Avatar';
 import CopyButton from '../components/CopyButton';
 import Sidebar from '../components/Sidebar';
@@ -25,6 +28,7 @@ import {
 } from '../preferences/preferenceSlice';
 import classes from './ConversationBubble.module.css';
 import { FEEDBACK, MESSAGE_TYPE } from './conversationModels';
+import { useOutsideAlerter } from '../hooks';
 
 const DisableSourceFE = import.meta.env.VITE_DISABLE_SOURCE_FE || false;
 
@@ -38,36 +42,114 @@ const ConversationBubble = forwardRef<
     handleFeedback?: (feedback: FEEDBACK) => void;
     sources?: { title: string; text: string; source: string }[];
     retryBtn?: React.ReactElement;
+    questionNumber?: number;
+    handleUpdatedQuestionSubmission?: (
+      updatedquestion?: string,
+      updated?: boolean,
+      index?: number,
+    ) => void;
   }
 >(function ConversationBubble(
-  { message, type, className, feedback, handleFeedback, sources, retryBtn },
+  {
+    message,
+    type,
+    className,
+    feedback,
+    handleFeedback,
+    sources,
+    retryBtn,
+    questionNumber,
+    handleUpdatedQuestionSubmission,
+  },
   ref,
 ) {
+  const { t } = useTranslation();
   // const bubbleRef = useRef<HTMLDivElement | null>(null);
   const chunks = useSelector(selectChunks);
   const selectedDocs = useSelector(selectSelectedDocs);
   const [isLikeHovered, setIsLikeHovered] = useState(false);
+  const [isEditClicked, setIsEditClicked] = useState(false);
   const [isDislikeHovered, setIsDislikeHovered] = useState(false);
+  const [isQuestionHovered, setIsQuestionHovered] = useState(false);
+  const [editInputBox, setEditInputBox] = useState<string>('');
+
   const [isLikeClicked, setIsLikeClicked] = useState(false);
   const [isDislikeClicked, setIsDislikeClicked] = useState(false);
   const [activeTooltip, setActiveTooltip] = useState<number | null>(null);
   const [isSidebarOpen, setIsSidebarOpen] = useState<boolean>(false);
+  const editableQueryRef = useRef<HTMLDivElement | null>(null);
 
+  useOutsideAlerter(editableQueryRef, () => setIsEditClicked(false), [], true);
+  const handleEditClick = () => {
+    setIsEditClicked(false);
+    handleUpdatedQuestionSubmission?.(editInputBox, true, questionNumber);
+  };
   let bubble;
   if (type === 'QUESTION') {
     bubble = (
       <div
-        ref={ref}
-        className={`flex flex-row-reverse self-end flex-wrap ${className}`}
+        onMouseEnter={() => setIsQuestionHovered(true)}
+        onMouseLeave={() => setIsQuestionHovered(false)}
       >
-        <Avatar className="mt-2 text-2xl" avatar="🧑‍💻"></Avatar>
         <div
-          style={{
-            wordBreak: 'break-word',
-          }}
-          className="ml-10 mr-2 flex items-center rounded-[28px] bg-purple-30 py-[14px] px-[19px] text-white max-w-full whitespace-pre-wrap leading-normal"
+          ref={ref}
+          className={`flex flex-row-reverse self-end flex-wrap ${className}`}
         >
-          {message}
+          <Avatar
+            size="SMALL"
+            className="mt-2 text-2xl"
+            avatar={
+              <img className="rounded-full mr-1" width={30} src={UserIcon} />
+            }
+          />
+          {!isEditClicked && (
+            <div
+              style={{
+                wordBreak: 'break-word',
+              }}
+              className="text-sm sm:text-base ml-2 mr-2 flex items-center rounded-[28px] bg-purple-30 py-[14px] px-[19px] text-white max-w-full whitespace-pre-wrap leading-normal"
+            >
+              {message}
+            </div>
+          )}
+          {isEditClicked && (
+            <div ref={editableQueryRef} className="w-[75%] flex flex-col">
+              <textarea
+                placeholder={t('conversation.edit.placeholder')}
+                onChange={(e) => {
+                  setEditInputBox(e.target.value);
+                }}
+                rows={1}
+                value={editInputBox}
+                className="ml-2 mr-12 text-[15px] resize-y h-12 min-h-max rounded-3xl p-3 no-scrollbar leading-relaxed dark:border-[0.5px] dark:border-white dark:bg-raisin-black dark:text-white px-[18px] border-[1.5px] border-black"
+              />
+              <div
+                className={`flex flex-row-reverse justify-end gap-1 mt-3 text-sm font-medium`}
+              >
+                <button
+                  className="rounded-full bg-[#CDB5FF] hover:bg-[#E1D3FF] py-[10px] px-[15px] text-purple-30 max-w-full whitespace-pre-wrap leading-none"
+                  onClick={() => handleEditClick()}
+                >
+                  {t('conversation.edit.update')}
+                </button>
+                <button
+                  className="py-[10px] px-[15px]  no-underline hover:underline  text-purple-30 max-w-full whitespace-pre-wrap leading-normal"
+                  onClick={() => setIsEditClicked(false)}
+                >
+                  {t('conversation.edit.cancel')}
+                </button>
+              </div>
+            </div>
+          )}
+          <button
+            onClick={() => {
+              setIsEditClicked(true);
+              setEditInputBox(message);
+            }}
+            className={`h-fit mt-3 p-2 cursor-pointer rounded-full hover:bg-[#35363B] flex items-center ${isQuestionHovered || isEditClicked ? 'visible' : 'invisible'}`}
+          >
+            <img src={Edit} alt="Edit" className="cursor-pointer" />
+          </button>
         </div>
       </div>
     );
@@ -105,12 +187,14 @@ const ConversationBubble = forwardRef<
                 avatar={
                   <img
                     src={Sources}
-                    alt="Sources"
+                    alt={t('conversation.sources.title')}
                     className="h-full w-full object-fill"
                   />
                 }
               />
-              <p className="text-base font-semibold">Sources</p>
+              <p className="text-base font-semibold">
+                {t('conversation.sources.title')}
+              </p>
             </div>
             <div className="grid grid-cols-2 gap-2 lg:grid-cols-4">
               {Array.from({ length: 4 }).map((_, index) => (
@@ -137,12 +221,14 @@ const ConversationBubble = forwardRef<
                   avatar={
                     <img
                       src={Sources}
-                      alt="Sources"
+                      alt={t('conversation.sources.title')}
                       className="h-full w-full object-fill"
                     />
                   }
                 />
-                <p className="text-base font-semibold">Sources</p>
+                <p className="text-base font-semibold">
+                  {t('conversation.sources.title')}
+                </p>
               </div>
               <div className="fade-in ml-3 mr-5 max-w-[90vw] md:max-w-[70vw] lg:max-w-[50vw]">
                 <div className="grid grid-cols-2 gap-2 lg:grid-cols-4">
@@ -209,9 +295,11 @@ const ConversationBubble = forwardRef<
                       className="flex h-28 cursor-pointer flex-col-reverse rounded-[20px] bg-gray-1000 p-4 text-purple-30 hover:bg-[#F1F1F1] hover:text-[#6D3ECC] dark:bg-gun-metal dark:hover:bg-[#2C2E3C] dark:hover:text-[#8C67D7]"
                       onClick={() => setIsSidebarOpen(true)}
                     >
-                      <p className="ellipsis-text h-22 text-xs">{`View ${
-                        sources?.length ? sources.length - 3 : 0
-                      } more`}</p>
+                      <p className="ellipsis-text h-22 text-xs">
+                        {t('conversation.sources.view_more', {
+                          count: sources?.length ? sources.length - 3 : 0,
+                        })}
+                      </p>
                     </div>
                   )}
                 </div>
@@ -226,12 +314,14 @@ const ConversationBubble = forwardRef<
               avatar={
                 <img
                   src={DocsGPT3}
-                  alt="DocsGPT"
+                  alt={t('conversation.answer')}
                   className="h-full w-full object-cover"
                 />
               }
             />
-            <p className="text-base font-semibold">Answer</p>
+            <p className="text-base font-semibold">
+              {t('conversation.answer')}
+            </p>
           </div>
           <div
             className={`fade-in-bubble ml-2 mr-5 flex max-w-[90vw] rounded-[28px] bg-gray-1000 py-[14px] px-7 dark:bg-gun-metal md:max-w-[70vw] lg:max-w-[50vw] ${
@@ -241,7 +331,7 @@ const ConversationBubble = forwardRef<
             }`}
           >
             <ReactMarkdown
-              className="fade-in whitespace-pre-wrap break-normal leading-normal"
+              className="fade-in whitespace-pre-wrap break-words leading-normal"
               remarkPlugins={[remarkGfm, remarkMath]}
               rehypePlugins={[rehypeKatex]}
               components={{
@@ -269,12 +359,7 @@ const ConversationBubble = forwardRef<
                       </div>
                     </div>
                   ) : (
-                    <code
-                      className={className ? className : 'whitespace-pre-line'}
-                      {...props}
-                    >
-                      {children}
-                    </code>
+                    <code className="whitespace-pre-line">{children}</code>
                   );
                 },
                 ul({ children }) {
@@ -344,7 +429,7 @@ const ConversationBubble = forwardRef<
             ${type !== 'ERROR' ? 'group-hover:lg:visible' : 'hidden'}`}
           >
             <div>
-              <SpeakButton text={message} /> {/* Add SpeakButton here */}
+              <SpeakButton text={message} />
             </div>
           </div>
           {type === 'ERROR' && (
@@ -361,7 +446,8 @@ const ConversationBubble = forwardRef<
                   feedback === 'LIKE' || type !== 'ERROR'
                     ? 'group-hover:lg:visible'
                     : ''
-                }`}
+                }
+                ${feedback === 'DISLIKE' && type !== 'ERROR' ? 'hidden' : ''}`}
               >
                 <div>
                   <div
@@ -379,9 +465,15 @@ const ConversationBubble = forwardRef<
                       : 'fill-none  stroke-gray-4000'
                   }`}
                       onClick={() => {
-                        handleFeedback?.('LIKE');
-                        setIsLikeClicked(true);
-                        setIsDislikeClicked(false);
+                        if (feedback === undefined || feedback === null) {
+                          handleFeedback?.('LIKE');
+                          setIsLikeClicked(true);
+                          setIsDislikeClicked(false);
+                        } else if (feedback === 'LIKE') {
+                          handleFeedback?.(null);
+                          setIsLikeClicked(false);
+                          setIsDislikeClicked(false);
+                        }
                       }}
                       onMouseEnter={() => setIsLikeHovered(true)}
                       onMouseLeave={() => setIsLikeHovered(false)}
@@ -396,7 +488,7 @@ const ConversationBubble = forwardRef<
                   feedback === 'DISLIKE' || type !== 'ERROR'
                     ? 'group-hover:lg:visible'
                     : ''
-                }`}
+                } ${feedback === 'LIKE' && type !== 'ERROR' ? ' hidden' : ''} `}
               >
                 <div>
                   <div
@@ -413,9 +505,15 @@ const ConversationBubble = forwardRef<
                           : 'fill-none  stroke-gray-4000'
                       }`}
                       onClick={() => {
-                        handleFeedback?.('DISLIKE');
-                        setIsDislikeClicked(true);
-                        setIsLikeClicked(false);
+                        if (feedback === undefined || feedback === null) {
+                          handleFeedback?.('DISLIKE');
+                          setIsDislikeClicked(true);
+                          setIsLikeClicked(false);
+                        } else if (feedback === 'DISLIKE') {
+                          handleFeedback?.(null);
+                          setIsLikeClicked(false);
+                          setIsDislikeClicked(false);
+                        }
                       }}
                       onMouseEnter={() => setIsDislikeHovered(true)}
                       onMouseLeave={() => setIsDislikeHovered(false)}
@@ -469,7 +567,7 @@ function AllSources(sources: AllSourcesProps) {
               {source.source && source.source !== 'local' ? (
                 <img
                   src={Link}
-                  alt="Link"
+                  alt={'Link'}
                   className="h-3 w-3 cursor-pointer object-fill"
                   onClick={() =>
                     window.open(source.source, '_blank', 'noopener, noreferrer')

frontend/src/conversation/ConversationTile.tsx

@@ -115,6 +115,19 @@ export default function ConversationTile({
     setConversationsName(conversation.name);
     setIsEdit(false);
   }
+
+  const handleRenameKeyDown = (e: React.KeyboardEvent<HTMLInputElement>) => {
+    e.stopPropagation();
+    if (e.key === 'Enter') {
+      handleSaveConversation({
+        id: conversation.id,
+        name: conversationName,
+      });
+    } else if (e.key === 'Escape') {
+      onClear();
+    }
+  };
+
   return (
     <>
       <div
@@ -144,6 +157,7 @@ export default function ConversationTile({
               className="h-6 w-full bg-transparent px-1 text-sm font-normal leading-6 focus:outline-[#0075FF]"
               value={conversationName}
               onChange={(e) => setConversationsName(e.target.value)}
+              onKeyDown={handleRenameKeyDown}
             />
           ) : (
             <p className="my-auto overflow-hidden overflow-ellipsis whitespace-nowrap text-sm font-normal leading-6 text-eerie-black dark:text-white">
@@ -239,7 +253,7 @@ export default function ConversationTile({
                 >
                   <img
                     src={Trash}
-                    alt="Edit"
+                    alt="Delete"
                     width={24}
                     height={24}
                     className="cursor-pointer hover:opacity-50"

frontend/src/conversation/SharedConversation.tsx

@@ -26,6 +26,7 @@ import {
   selectQueries,
 } from './sharedConversationSlice';
 import { useSelector } from 'react-redux';
+import { Helmet } from 'react-helmet';
 
 export const SharedConversation = () => {
   const navigate = useNavigate();
@@ -176,94 +177,111 @@ export const SharedConversation = () => {
   }, []);
 
   return (
-    <div className="flex h-full flex-col items-center justify-between gap-2 overflow-y-hidden dark:bg-raisin-black">
-      <div
-        ref={sharedConversationRef}
-        onWheel={handleUserInterruption}
-        onTouchMove={handleUserInterruption}
-        className="flex w-full justify-center overflow-auto"
-      >
-        <div className="mt-0 w-11/12 md:w-10/12 lg:w-6/12">
-          <div className="mb-2 w-full border-b pb-2 dark:border-b-silver">
-            <h1 className="font-semi-bold text-4xl text-chinese-black dark:text-chinese-silver">
-              {title}
-            </h1>
-            <h2 className="font-semi-bold text-base text-chinese-black dark:text-chinese-silver">
-              {t('sharedConv.subtitle')}{' '}
-              <a href="/" className="text-[#007DFF]">
-                DocsGPT
-              </a>
-            </h2>
-            <h2 className="font-semi-bold text-base text-chinese-black dark:text-chinese-silver">
-              {date}
-            </h2>
-          </div>
-          <div className="">
-            {queries?.map((query, index) => {
-              return (
-                <Fragment key={index}>
-                  <ConversationBubble
-                    ref={endMessageRef}
-                    className={'mb-1 last:mb-28 md:mb-7'}
-                    key={`${index}QUESTION`}
-                    message={query.prompt}
-                    type="QUESTION"
-                    sources={query.sources}
-                  ></ConversationBubble>
+    <>
+      <Helmet>
+        <title>{`DocsGPT | ${title}`}</title>
+        <meta name="description" content="Shared conversations with DocsGPT" />
+        <meta property="og:title" content={title} />
+        <meta
+          property="og:description"
+          content="Shared conversations with DocsGPT"
+        />
+        <meta name="twitter:card" content="summary_large_image" />
+        <meta name="twitter:title" content={title} />
+        <meta
+          name="twitter:description"
+          content="Shared conversations with DocsGPT"
+        />
+      </Helmet>
+      <div className="flex h-full flex-col items-center justify-between gap-2 overflow-y-hidden dark:bg-raisin-black">
+        <div
+          ref={sharedConversationRef}
+          onWheel={handleUserInterruption}
+          onTouchMove={handleUserInterruption}
+          className="flex w-full justify-center overflow-auto"
+        >
+          <div className="mt-0 w-11/12 md:w-10/12 lg:w-6/12">
+            <div className="mb-2 w-full border-b pb-2 dark:border-b-silver">
+              <h1 className="font-semi-bold text-4xl text-chinese-black dark:text-chinese-silver">
+                {title}
+              </h1>
+              <h2 className="font-semi-bold text-base text-chinese-black dark:text-chinese-silver">
+                {t('sharedConv.subtitle')}{' '}
+                <a href="/" className="text-[#007DFF]">
+                  DocsGPT
+                </a>
+              </h2>
+              <h2 className="font-semi-bold text-base text-chinese-black dark:text-chinese-silver">
+                {date}
+              </h2>
+            </div>
+            <div className="">
+              {queries?.map((query, index) => {
+                return (
+                  <Fragment key={index}>
+                    <ConversationBubble
+                      ref={endMessageRef}
+                      className={'mb-1 last:mb-28 md:mb-7'}
+                      key={`${index}QUESTION`}
+                      message={query.prompt}
+                      type="QUESTION"
+                      sources={query.sources}
+                    ></ConversationBubble>
 
-                  {prepResponseView(query, index)}
-                </Fragment>
-              );
-            })}
+                    {prepResponseView(query, index)}
+                  </Fragment>
+                );
+              })}
+            </div>
           </div>
         </div>
-      </div>
 
-      <div className=" flex w-11/12 flex-col items-center gap-4 pb-2 md:w-10/12 lg:w-6/12">
-        {apiKey ? (
-          <div className="flex h-full w-full items-center rounded-[40px] border border-silver bg-white py-1 dark:bg-raisin-black">
-            <div
-              id="inputbox"
-              ref={inputRef}
-              tabIndex={1}
-              onPaste={handlePaste}
-              placeholder={t('inputPlaceholder')}
-              contentEditable
-              className={`inputbox-style max-h-24 w-full overflow-y-auto overflow-x-hidden whitespace-pre-wrap rounded-full bg-white pt-5 pb-[22px] text-base leading-tight opacity-100 focus:outline-none dark:bg-raisin-black dark:text-bright-gray`}
-              onKeyDown={(e) => {
-                if (e.key === 'Enter' && !e.shiftKey) {
-                  e.preventDefault();
-                  handleQuestionSubmission();
-                }
-              }}
-            ></div>
-            {status === 'loading' ? (
-              <img
-                src={Spinner}
-                className="relative right-[38px] bottom-[24px] -mr-[30px] animate-spin cursor-pointer self-end bg-transparent filter dark:invert"
-              ></img>
-            ) : (
-              <div className="mx-1 cursor-pointer rounded-full p-3 text-center hover:bg-gray-3000 dark:hover:bg-dark-charcoal">
+        <div className="flex w-11/12 flex-col items-center gap-4 pb-2 md:w-10/12 lg:w-6/12">
+          {apiKey ? (
+            <div className="flex h-full w-full items-center rounded-[40px] border border-silver bg-white py-1 dark:bg-raisin-black">
+              <div
+                id="inputbox"
+                ref={inputRef}
+                tabIndex={1}
+                onPaste={handlePaste}
+                placeholder={t('inputPlaceholder')}
+                contentEditable
+                className={`inputbox-style max-h-24 w-full overflow-y-auto overflow-x-hidden whitespace-pre-wrap rounded-full bg-white pt-5 pb-[22px] text-base leading-tight opacity-100 focus:outline-none dark:bg-raisin-black dark:text-bright-gray`}
+                onKeyDown={(e) => {
+                  if (e.key === 'Enter' && !e.shiftKey) {
+                    e.preventDefault();
+                    handleQuestionSubmission();
+                  }
+                }}
+              ></div>
+              {status === 'loading' ? (
                 <img
-                  onClick={handleQuestionSubmission}
-                  className="ml-[4px] h-6 w-6 text-white filter dark:invert"
-                  src={Send}
+                  src={Spinner}
+                  className="relative right-[38px] bottom-[24px] -mr-[30px] animate-spin cursor-pointer self-end bg-transparent filter dark:invert"
                 ></img>
-              </div>
-            )}
-          </div>
-        ) : (
-          <button
-            onClick={() => navigate('/')}
-            className="w-fit rounded-full bg-purple-30 p-4 text-white shadow-xl transition-colors duration-200 hover:bg-purple-taupe"
-          >
-            {t('sharedConv.button')}
-          </button>
-        )}
-        <span className="mb-2 hidden text-xs text-dark-charcoal dark:text-silver sm:inline">
-          {t('sharedConv.meta')}
-        </span>
+              ) : (
+                <div className="mx-1 cursor-pointer rounded-full p-3 text-center hover:bg-gray-3000 dark:hover:bg-dark-charcoal">
+                  <img
+                    onClick={handleQuestionSubmission}
+                    className="ml-[4px] h-6 w-6 text-white filter dark:invert"
+                    src={Send}
+                  ></img>
+                </div>
+              )}
+            </div>
+          ) : (
+            <button
+              onClick={() => navigate('/')}
+              className="w-fit rounded-full bg-purple-30 p-4 text-white shadow-xl transition-colors duration-200 hover:bg-purple-taupe mb-14 sm:mb-0"
+            >
+              {t('sharedConv.button')}
+            </button>
+          )}
+          <span className="mb-2 hidden text-xs text-dark-charcoal dark:text-silver sm:inline">
+            {t('sharedConv.meta')}
+          </span>
+        </div>
       </div>
-    </div>
+    </>
   );
 };