feat: enhance MemoryTool and NotesTool with tool_id management and directory renaming tests

* Add MongoDB-backed NotesTool with CRUD actions and unit tests

* NotesTool: enforce single note per user, require decoded_token['sub'] user_id, fix tests

* chore: remove accidentally committed results.txt and ignore it

* fix: remove results.txt, enforce single note per user, add tests

* refactor: update NotesTool actions and tests for clarity and consistency

* refactor: update NotesTool docstring for clarity

* refactor: simplify MCPTool docstring and remove redundant import in test_notes_tool

* lint: fix test

* refactor: remove unused import from test_notes_tool.py

---------

Co-authored-by: Alex <a@tushynski.me>

.vscode/launch.json

@@ -2,15 +2,11 @@
   "version": "0.2.0",
   "configurations": [
     {
-      "name": "Docker Debug Frontend",
+      "name": "Frontend Debug (npm)",
+      "type": "node-terminal",
       "request": "launch",
-      "type": "chrome",
-      "preLaunchTask": "docker-compose: debug:frontend",
-      "url": "http://127.0.0.1:5173",
-      "webRoot": "${workspaceFolder}/frontend",
-      "skipFiles": [
-        "<node_internals>/**"
-      ]
+      "command": "npm run dev",
+      "cwd": "${workspaceFolder}/frontend"
     },
     {
         "name": "Flask Debugger",
@@ -49,6 +45,27 @@
         "--pool=solo"
       ],
       "cwd": "${workspaceFolder}"
+    },
+    {
+      "name": "Dev Containers (Mongo + Redis)",
+      "type": "node-terminal",
+      "request": "launch",
+      "command": "docker compose -f deployment/docker-compose-dev.yaml up --build",
+      "cwd": "${workspaceFolder}"
+    }
+  ],
+  "compounds": [
+    {
+      "name": "DocsGPT: Full Stack",
+      "configurations": [
+        "Frontend Debug (npm)",
+        "Flask Debugger",
+        "Celery Debugger"
+      ],
+      "presentation": {
+        "group": "DocsGPT",
+        "order": 1
+      }
     }
   ]
 }

.vscode/tasks.json

@@ -1,21 +0,0 @@
-{
-  "version": "2.0.0",
-  "tasks": [
-    {
-      "type": "docker-compose",
-      "label": "docker-compose: debug:frontend",
-      "dockerCompose": {
-        "up": {
-          "detached": true,
-          "services": [
-            "frontend"
-          ],
-          "build": true
-        },
-        "files": [
-          "${workspaceFolder}/docker-compose.yaml"
-        ]
-      }
-    }
-  ]
-}

HACKTOBERFEST.md

@@ -0,0 +1,38 @@
+# **🎉 Join the Hacktoberfest with DocsGPT and win a Free T-shirt for a meaningful PR! 🎉**
+
+Welcome, contributors! We're excited to announce that DocsGPT is participating in Hacktoberfest. Get involved by submitting meaningful pull requests.
+
+All Meaningful contributors with accepted PRs that were created for issues with the `hacktoberfest` label (set by our maintainer team: dartpain, siiddhantt, pabik, ManishMadan2882) will receive a cool T-shirt! 🤩.
+
+Fill in [this form](https://forms.gle/Npaba4n9Epfyx56S8
+) after your PR was merged please 
+
+If you are in doubt don't hesitate to ping us on discord, ping me - Alex (dartpain).
+
+## 📜 Here's How to Contribute:
+```text
+🛠️ Code: This is the golden ticket! Make meaningful contributions through PRs.
+
+🧩 API extension: Build an app utilising DocsGPT API. We prefer submissions that showcase original ideas and turn the API into an AI agent.
+They can be a completely separate repos. 
+For example: 
+https://github.com/arc53/tg-bot-docsgpt-extenstion or 
+https://github.com/arc53/DocsGPT-cli
+
+Non-Code Contributions:
+
+📚 Wiki: Improve our documentation, create a guide.
+
+🖥️ Design: Improve the UI/UX or design a new feature.
+```
+
+### 📝 Guidelines for Pull Requests:
+- Familiarize yourself with the current contributions and our [Roadmap](https://github.com/orgs/arc53/projects/2).
+- Before contributing check existing [issues](https://github.com/arc53/DocsGPT/issues) or [create](https://github.com/arc53/DocsGPT/issues/new/choose) an issue and wait to get assigned.
+- Once you are finished with your contribution, please fill in this [form](https://forms.gle/Npaba4n9Epfyx56S8).
+- Refer to the [Documentation](https://docs.docsgpt.cloud/).
+- Feel free to join our [Discord](https://discord.gg/n5BX8dh8rU) server. We're here to help newcomers, so don't hesitate to jump in! Join us [here](https://discord.gg/n5BX8dh8rU).
+  
+Thank you very much for considering contributing to DocsGPT during Hacktoberfest! 🙏 Your contributions (not just simple typos) could earn you a stylish new t-shirt.
+
+We will publish a t-shirt design later into the October.

README.md

@@ -17,7 +17,7 @@
   <a href="https://github.com/arc53/DocsGPT/blob/main/LICENSE">![link to license file](https://img.shields.io/github/license/arc53/docsgpt)</a>
   <a href="https://www.bestpractices.dev/projects/9907"><img src="https://www.bestpractices.dev/projects/9907/badge"></a>
   <a href="https://discord.gg/n5BX8dh8rU">![link to discord](https://img.shields.io/discord/1070046503302877216)</a>
-  <a href="https://twitter.com/docsgptai">![X (formerly Twitter) URL](https://img.shields.io/twitter/follow/docsgptai)</a>
+  <a href="https://x.com/docsgptai">![X (formerly Twitter) URL](https://img.shields.io/twitter/follow/docsgptai)</a>
 
 <a href="https://docs.docsgpt.cloud/quickstart">⚡️ Quickstart</a> • <a href="https://app.docsgpt.cloud/">☁️ Cloud Version</a> • <a href="https://discord.gg/n5BX8dh8rU">💬 Discord</a>
 <br>
@@ -25,7 +25,17 @@
 <br>
 
 </div>
+
 <div align="center">
+  <br>
+🎃 <a href="https://github.com/arc53/DocsGPT/blob/main/HACKTOBERFEST.md"> Hacktoberfest Prizes, Rules & Q&A </a> 🎃
+  <br>
+  <br>
+</div>
+
+
+<div align="center">
+  <br>
 <img src="https://d3dg1063dc54p9.cloudfront.net/videos/demov7.gif" alt="video-example-of-docs-gpt" width="800" height="450">
 </div>
 <h3 align="left">
@@ -57,7 +67,7 @@
 - [x] Json Responses (August 2025)
 - [x] MCP support (August 2025)
 - [x] Google Drive integration (September 2025)
-- [ ] Add OAuth 2.0 authentication for MCP (September 2025)
+- [x] Add OAuth 2.0 authentication for MCP (September 2025)
 - [ ] Sharepoint integration (October 2025)
 - [ ] Deep Agents (October 2025)
 - [ ] Agent scheduling

application/agents/base.py

@@ -213,18 +213,24 @@ class BaseAgent(ABC):
                 ):
                     target_dict[param] = value
         tm = ToolManager(config={})
+
+        # Prepare tool_config and add tool_id for memory tools
+        if tool_data["name"] == "api_tool":
+            tool_config = {
+                "url": tool_data["config"]["actions"][action_name]["url"],
+                "method": tool_data["config"]["actions"][action_name]["method"],
+                "headers": headers,
+                "query_params": query_params,
+            }
+        else:
+            tool_config = tool_data["config"].copy() if tool_data["config"] else {}
+            # Add tool_id from MongoDB _id for tools that need instance isolation (like memory tool)
+            # Use MongoDB _id if available, otherwise fall back to enumerated tool_id
+            tool_config["tool_id"] = str(tool_data.get("_id", tool_id))
+
         tool = tm.load_tool(
             tool_data["name"],
-            tool_config=(
-                {
-                    "url": tool_data["config"]["actions"][action_name]["url"],
-                    "method": tool_data["config"]["actions"][action_name]["method"],
-                    "headers": headers,
-                    "query_params": query_params,
-                }
-                if tool_data["name"] == "api_tool"
-                else tool_data["config"]
-            ),
+            tool_config=tool_config,
             user_id=self.user,  # Pass user ID for MCP tools credential decryption
         )
         if tool_data["name"] == "api_tool":

application/agents/tools/mcp_tool.py

@@ -37,7 +37,7 @@ _mcp_clients_cache = {}
 class MCPTool(Tool):
     """
     MCP Tool
-    Connect to remote Model Context Protocol (MCP) servers to access dynamic tools and resources. Supports various authentication methods and provides secure access to external services through the MCP protocol.
+    Connect to remote Model Context Protocol (MCP) servers to access dynamic tools and resources.
     """
 
     def __init__(self, config: Dict[str, Any], user_id: Optional[str] = None):

application/agents/tools/memory.py

@@ -0,0 +1,546 @@
+from datetime import datetime
+from pathlib import Path
+from typing import Any, Dict, List, Optional
+import re
+import uuid
+
+from .base import Tool
+from application.core.mongo_db import MongoDB
+from application.core.settings import settings
+
+
+class MemoryTool(Tool):
+    """Memory
+
+    Stores and retrieves information across conversations through a memory file directory.
+    """
+
+    def __init__(self, tool_config: Optional[Dict[str, Any]] = None, user_id: Optional[str] = None) -> None:
+        """Initialize the tool.
+
+        Args:
+            tool_config: Optional tool configuration. Should include:
+                - tool_id: Unique identifier for this memory tool instance (from user_tools._id)
+                           This ensures each user's tool configuration has isolated memories
+            user_id: The authenticated user's id (should come from decoded_token["sub"]).
+        """
+        self.user_id: Optional[str] = user_id
+
+        # Get tool_id from configuration (passed from user_tools._id in production)
+        # In production, tool_id is the MongoDB ObjectId string from user_tools collection
+        if tool_config and "tool_id" in tool_config:
+            self.tool_id = tool_config["tool_id"]
+        elif user_id:
+            # Fallback for backward compatibility or testing
+            self.tool_id = f"default_{user_id}"
+        else:
+            # Last resort fallback (shouldn't happen in normal use)
+            self.tool_id = str(uuid.uuid4())
+
+        db = MongoDB.get_client()[settings.MONGO_DB_NAME]
+        self.collection = db["memories"]
+
+    # -----------------------------
+    # Action implementations
+    # -----------------------------
+    def execute_action(self, action_name: str, **kwargs: Any) -> str:
+        """Execute an action by name.
+
+        Args:
+            action_name: One of view, create, str_replace, insert, delete, rename.
+            **kwargs: Parameters for the action.
+
+        Returns:
+            A human-readable string result.
+        """
+        if not self.user_id:
+            return "Error: MemoryTool requires a valid user_id."
+
+        if action_name == "view":
+            return self._view(
+                kwargs.get("path", "/"),
+                kwargs.get("view_range")
+            )
+
+        if action_name == "create":
+            return self._create(
+                kwargs.get("path", ""),
+                kwargs.get("file_text", "")
+            )
+
+        if action_name == "str_replace":
+            return self._str_replace(
+                kwargs.get("path", ""),
+                kwargs.get("old_str", ""),
+                kwargs.get("new_str", "")
+            )
+
+        if action_name == "insert":
+            return self._insert(
+                kwargs.get("path", ""),
+                kwargs.get("insert_line", 1),
+                kwargs.get("insert_text", "")
+            )
+
+        if action_name == "delete":
+            return self._delete(kwargs.get("path", ""))
+
+        if action_name == "rename":
+            return self._rename(
+                kwargs.get("old_path", ""),
+                kwargs.get("new_path", "")
+            )
+
+        return f"Unknown action: {action_name}"
+
+    def get_actions_metadata(self) -> List[Dict[str, Any]]:
+        """Return JSON metadata describing supported actions for tool schemas."""
+        return [
+            {
+                "name": "view",
+                "description": "Shows directory contents or file contents with optional line ranges.",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "path": {
+                            "type": "string",
+                            "description": "Path to file or directory (e.g., /notes.txt or /project/ or /)."
+                        },
+                        "view_range": {
+                            "type": "array",
+                            "items": {"type": "integer"},
+                            "description": "Optional [start_line, end_line] to view specific lines (1-indexed)."
+                        }
+                    },
+                    "required": ["path"]
+                },
+            },
+            {
+                "name": "create",
+                "description": "Create or overwrite a file.",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "path": {
+                            "type": "string",
+                            "description": "File path to create (e.g., /notes.txt or /project/task.txt)."
+                        },
+                        "file_text": {
+                            "type": "string",
+                            "description": "Content to write to the file."
+                        }
+                    },
+                    "required": ["path", "file_text"]
+                },
+            },
+            {
+                "name": "str_replace",
+                "description": "Replace text in a file.",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "path": {
+                            "type": "string",
+                            "description": "File path (e.g., /notes.txt)."
+                        },
+                        "old_str": {
+                            "type": "string",
+                            "description": "String to find."
+                        },
+                        "new_str": {
+                            "type": "string",
+                            "description": "String to replace with."
+                        }
+                    },
+                    "required": ["path", "old_str", "new_str"]
+                },
+            },
+            {
+                "name": "insert",
+                "description": "Insert text at a specific line in a file.",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "path": {
+                            "type": "string",
+                            "description": "File path (e.g., /notes.txt)."
+                        },
+                        "insert_line": {
+                            "type": "integer",
+                            "description": "Line number to insert at (1-indexed)."
+                        },
+                        "insert_text": {
+                            "type": "string",
+                            "description": "Text to insert."
+                        }
+                    },
+                    "required": ["path", "insert_line", "insert_text"]
+                },
+            },
+            {
+                "name": "delete",
+                "description": "Delete a file or directory.",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "path": {
+                            "type": "string",
+                            "description": "Path to delete (e.g., /notes.txt or /project/)."
+                        }
+                    },
+                    "required": ["path"]
+                },
+            },
+            {
+                "name": "rename",
+                "description": "Rename or move a file/directory.",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "old_path": {
+                            "type": "string",
+                            "description": "Current path (e.g., /old.txt)."
+                        },
+                        "new_path": {
+                            "type": "string",
+                            "description": "New path (e.g., /new.txt)."
+                        }
+                    },
+                    "required": ["old_path", "new_path"]
+                },
+            },
+        ]
+
+    def get_config_requirements(self) -> Dict[str, Any]:
+        """Return configuration requirements."""
+        return {}
+
+    # -----------------------------
+    # Path validation
+    # -----------------------------
+    def _validate_path(self, path: str) -> Optional[str]:
+        """Validate and normalize path.
+
+        Args:
+            path: User-provided path.
+
+        Returns:
+            Normalized path or None if invalid.
+        """
+        if not path:
+            return None
+
+        # Remove any leading/trailing whitespace
+        path = path.strip()
+
+        # Preserve whether path ends with / (indicates directory)
+        is_directory = path.endswith("/")
+
+        # Ensure path starts with / for consistency
+        if not path.startswith("/"):
+            path = "/" + path
+
+        # Check for directory traversal patterns
+        if ".." in path or path.count("//") > 0:
+            return None
+
+        # Normalize the path
+        try:
+            # Convert to Path object and resolve to canonical form
+            normalized = str(Path(path).as_posix())
+
+            # Ensure it still starts with /
+            if not normalized.startswith("/"):
+                return None
+
+            # Preserve trailing slash for directories
+            if is_directory and not normalized.endswith("/") and normalized != "/":
+                normalized = normalized + "/"
+
+            return normalized
+        except Exception:
+            return None
+
+    # -----------------------------
+    # Internal helpers
+    # -----------------------------
+    def _view(self, path: str, view_range: Optional[List[int]] = None) -> str:
+        """View directory contents or file contents."""
+        validated_path = self._validate_path(path)
+        if not validated_path:
+            return "Error: Invalid path."
+
+        # Check if viewing directory (ends with / or is root)
+        if validated_path == "/" or validated_path.endswith("/"):
+            return self._view_directory(validated_path)
+
+        # Otherwise view file
+        return self._view_file(validated_path, view_range)
+
+    def _view_directory(self, path: str) -> str:
+        """List files in a directory."""
+        # Ensure path ends with / for proper prefix matching
+        search_path = path if path.endswith("/") else path + "/"
+
+        # Find all files that start with this directory path
+        query = {
+            "user_id": self.user_id,
+            "tool_id": self.tool_id,
+            "path": {"$regex": f"^{re.escape(search_path)}"}
+        }
+
+        docs = list(self.collection.find(query, {"path": 1}))
+
+        if not docs:
+            return f"Directory: {path}\n(empty)"
+
+        # Extract filenames relative to the directory
+        files = []
+        for doc in docs:
+            file_path = doc["path"]
+            # Remove the directory prefix
+            if file_path.startswith(search_path):
+                relative = file_path[len(search_path):]
+                if relative:
+                    files.append(relative)
+
+        files.sort()
+        file_list = "\n".join(f"- {f}" for f in files)
+        return f"Directory: {path}\n{file_list}"
+
+    def _view_file(self, path: str, view_range: Optional[List[int]] = None) -> str:
+        """View file contents with optional line range."""
+        doc = self.collection.find_one({"user_id": self.user_id, "tool_id": self.tool_id, "path": path})
+
+        if not doc or not doc.get("content"):
+            return f"Error: File not found: {path}"
+
+        content = str(doc["content"])
+
+        # Apply view_range if specified
+        if view_range and len(view_range) == 2:
+            lines = content.split("\n")
+            start, end = view_range
+            # Convert to 0-indexed
+            start_idx = max(0, start - 1)
+            end_idx = min(len(lines), end)
+
+            if start_idx >= len(lines):
+                return f"Error: Line range out of bounds. File has {len(lines)} lines."
+
+            selected_lines = lines[start_idx:end_idx]
+            # Add line numbers (enumerate with 1-based start)
+            numbered_lines = [f"{i}: {line}" for i, line in enumerate(selected_lines, start=start)]
+            return "\n".join(numbered_lines)
+
+        return content
+
+    def _create(self, path: str, file_text: str) -> str:
+        """Create or overwrite a file."""
+        validated_path = self._validate_path(path)
+        if not validated_path:
+            return "Error: Invalid path."
+
+        if validated_path == "/" or validated_path.endswith("/"):
+            return "Error: Cannot create a file at directory path."
+
+        self.collection.update_one(
+            {"user_id": self.user_id, "tool_id": self.tool_id, "path": validated_path},
+            {
+                "$set": {
+                    "content": file_text,
+                    "updated_at": datetime.now()
+                }
+            },
+            upsert=True
+        )
+
+        return f"File created: {validated_path}"
+
+    def _str_replace(self, path: str, old_str: str, new_str: str) -> str:
+        """Replace text in a file."""
+        validated_path = self._validate_path(path)
+        if not validated_path:
+            return "Error: Invalid path."
+
+        if not old_str:
+            return "Error: old_str is required."
+
+        doc = self.collection.find_one({"user_id": self.user_id, "tool_id": self.tool_id, "path": validated_path})
+
+        if not doc or not doc.get("content"):
+            return f"Error: File not found: {validated_path}"
+
+        current_content = str(doc["content"])
+
+        # Check if old_str exists (case-insensitive)
+        if old_str.lower() not in current_content.lower():
+            return f"Error: String '{old_str}' not found in file."
+
+        # Replace the string (case-insensitive)
+        import re as regex_module
+        updated_content = regex_module.sub(regex_module.escape(old_str), new_str, current_content, flags=regex_module.IGNORECASE)
+
+        self.collection.update_one(
+            {"user_id": self.user_id, "tool_id": self.tool_id, "path": validated_path},
+            {
+                "$set": {
+                    "content": updated_content,
+                    "updated_at": datetime.now()
+                }
+            }
+        )
+
+        return f"File updated: {validated_path}"
+
+    def _insert(self, path: str, insert_line: int, insert_text: str) -> str:
+        """Insert text at a specific line."""
+        validated_path = self._validate_path(path)
+        if not validated_path:
+            return "Error: Invalid path."
+
+        if not insert_text:
+            return "Error: insert_text is required."
+
+        doc = self.collection.find_one({"user_id": self.user_id, "tool_id": self.tool_id, "path": validated_path})
+
+        if not doc or not doc.get("content"):
+            return f"Error: File not found: {validated_path}"
+
+        current_content = str(doc["content"])
+        lines = current_content.split("\n")
+
+        # Convert to 0-indexed
+        index = insert_line - 1
+        if index < 0 or index > len(lines):
+            return f"Error: Invalid line number. File has {len(lines)} lines."
+
+        lines.insert(index, insert_text)
+        updated_content = "\n".join(lines)
+
+        self.collection.update_one(
+            {"user_id": self.user_id, "tool_id": self.tool_id, "path": validated_path},
+            {
+                "$set": {
+                    "content": updated_content,
+                    "updated_at": datetime.now()
+                }
+            }
+        )
+
+        return f"Text inserted at line {insert_line} in {validated_path}"
+
+    def _delete(self, path: str) -> str:
+        """Delete a file or directory."""
+        validated_path = self._validate_path(path)
+        if not validated_path:
+            return "Error: Invalid path."
+
+        if validated_path == "/":
+            # Delete all files for this user and tool
+            result = self.collection.delete_many({"user_id": self.user_id, "tool_id": self.tool_id})
+            return f"Deleted {result.deleted_count} file(s) from memory."
+
+        # Check if it's a directory (ends with /)
+        if validated_path.endswith("/"):
+            # Delete all files in directory
+            result = self.collection.delete_many({
+                "user_id": self.user_id,
+                "tool_id": self.tool_id,
+                "path": {"$regex": f"^{re.escape(validated_path)}"}
+            })
+            return f"Deleted directory and {result.deleted_count} file(s)."
+
+        # Try to delete as directory first (without trailing slash)
+        # Check if any files start with this path + /
+        search_path = validated_path + "/"
+        directory_result = self.collection.delete_many({
+            "user_id": self.user_id,
+            "tool_id": self.tool_id,
+            "path": {"$regex": f"^{re.escape(search_path)}"}
+        })
+
+        if directory_result.deleted_count > 0:
+            return f"Deleted directory and {directory_result.deleted_count} file(s)."
+
+        # Delete single file
+        result = self.collection.delete_one({
+            "user_id": self.user_id,
+            "tool_id": self.tool_id,
+            "path": validated_path
+        })
+
+        if result.deleted_count:
+            return f"Deleted: {validated_path}"
+        return f"Error: File not found: {validated_path}"
+
+    def _rename(self, old_path: str, new_path: str) -> str:
+        """Rename or move a file/directory."""
+        validated_old = self._validate_path(old_path)
+        validated_new = self._validate_path(new_path)
+
+        if not validated_old or not validated_new:
+            return "Error: Invalid path."
+
+        if validated_old == "/" or validated_new == "/":
+            return "Error: Cannot rename root directory."
+
+        # Check if renaming a directory
+        if validated_old.endswith("/"):
+            # Ensure validated_new also ends with / for proper path replacement
+            if not validated_new.endswith("/"):
+                validated_new = validated_new + "/"
+
+            # Find all files in the old directory
+            docs = list(self.collection.find({
+                "user_id": self.user_id,
+                "tool_id": self.tool_id,
+                "path": {"$regex": f"^{re.escape(validated_old)}"}
+            }))
+
+            if not docs:
+                return f"Error: Directory not found: {validated_old}"
+
+            # Update paths for all files
+            for doc in docs:
+                old_file_path = doc["path"]
+                new_file_path = old_file_path.replace(validated_old, validated_new, 1)
+
+                self.collection.update_one(
+                    {"_id": doc["_id"]},
+                    {"$set": {"path": new_file_path, "updated_at": datetime.now()}}
+                )
+
+            return f"Renamed directory: {validated_old} -> {validated_new} ({len(docs)} files)"
+
+        # Rename single file
+        doc = self.collection.find_one({
+            "user_id": self.user_id,
+            "tool_id": self.tool_id,
+            "path": validated_old
+        })
+
+        if not doc:
+            return f"Error: File not found: {validated_old}"
+
+        # Check if new path already exists
+        existing = self.collection.find_one({
+            "user_id": self.user_id,
+            "tool_id": self.tool_id,
+            "path": validated_new
+        })
+
+        if existing:
+            return f"Error: File already exists at {validated_new}"
+
+        # Delete the old document and create a new one with the new path
+        self.collection.delete_one({"user_id": self.user_id, "tool_id": self.tool_id, "path": validated_old})
+        self.collection.insert_one({
+            "user_id": self.user_id,
+            "tool_id": self.tool_id,
+            "path": validated_new,
+            "content": doc.get("content", ""),
+            "updated_at": datetime.now()
+        })
+
+        return f"Renamed: {validated_old} -> {validated_new}"

application/agents/tools/notes.py

@@ -0,0 +1,199 @@
+from datetime import datetime
+from typing import Any, Dict, List, Optional
+import uuid
+
+from .base import Tool
+from application.core.mongo_db import MongoDB
+from application.core.settings import settings
+
+
+class NotesTool(Tool):
+    """Notepad
+
+    Single note. Supports viewing, overwriting, string replacement.
+    """
+
+    def __init__(self, tool_config: Optional[Dict[str, Any]] = None, user_id: Optional[str] = None) -> None:
+        """Initialize the tool.
+
+        Args:
+            tool_config: Optional tool configuration. Should include:
+                - tool_id: Unique identifier for this notes tool instance (from user_tools._id)
+                           This ensures each user's tool configuration has isolated notes
+            user_id: The authenticated user's id (should come from decoded_token["sub"]).
+        """
+        self.user_id: Optional[str] = user_id
+
+        # Get tool_id from configuration (passed from user_tools._id in production)
+        # In production, tool_id is the MongoDB ObjectId string from user_tools collection
+        if tool_config and "tool_id" in tool_config:
+            self.tool_id = tool_config["tool_id"]
+        elif user_id:
+            # Fallback for backward compatibility or testing
+            self.tool_id = f"default_{user_id}"
+        else:
+            # Last resort fallback (shouldn't happen in normal use)
+            self.tool_id = str(uuid.uuid4())
+
+        db = MongoDB.get_client()[settings.MONGO_DB_NAME]
+        self.collection = db["notes"]
+
+    # -----------------------------
+    # Action implementations
+    # -----------------------------
+    def execute_action(self, action_name: str, **kwargs: Any) -> str:
+        """Execute an action by name.
+
+        Args:
+            action_name: One of view, overwrite, str_replace, insert, delete.
+            **kwargs: Parameters for the action.
+
+        Returns:
+            A human-readable string result.
+        """
+        if not self.user_id:
+             return "Error: NotesTool requires a valid user_id."
+
+        if action_name == "view":
+            return self._get_note()
+
+        if action_name == "overwrite":
+            return self._overwrite_note(kwargs.get("text", ""))
+
+        if action_name == "str_replace":
+            return self._str_replace(kwargs.get("old_str", ""), kwargs.get("new_str", ""))
+
+        if action_name == "insert":
+            return self._insert(kwargs.get("line_number", 1), kwargs.get("text", ""))
+
+        if action_name == "delete":
+            return self._delete_note()
+
+        return f"Unknown action: {action_name}"
+
+    def get_actions_metadata(self) -> List[Dict[str, Any]]:
+        """Return JSON metadata describing supported actions for tool schemas."""
+        return [
+            {
+                "name": "view",
+                "description": "Retrieve the user's note.",
+                "parameters": {"type": "object", "properties": {}},
+            },
+            {
+                "name": "overwrite",
+                "description": "Replace the entire note content (creates if doesn't exist).",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "text": {"type": "string", "description": "New note content."}
+                    },
+                    "required": ["text"],
+                },
+            },
+            {
+                "name": "str_replace",
+                "description": "Replace occurrences of old_str with new_str in the note.",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "old_str": {"type": "string", "description": "String to find."},
+                        "new_str": {"type": "string", "description": "String to replace with."}
+                    },
+                    "required": ["old_str", "new_str"],
+                },
+            },
+            {
+                "name": "insert",
+                "description": "Insert text at the specified line number (1-indexed).",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "line_number": {"type": "integer", "description": "Line number to insert at (1-indexed)."},
+                        "text": {"type": "string", "description": "Text to insert."}
+                    },
+                    "required": ["line_number", "text"],
+                },
+            },
+            {
+                "name": "delete",
+                "description": "Delete the user's note.",
+                "parameters": {"type": "object", "properties": {}},
+            },
+        ]
+
+    def get_config_requirements(self) -> Dict[str, Any]:
+        """Return configuration requirements (none for now)."""
+        return {}
+
+    # -----------------------------
+    # Internal helpers (single-note)
+    # -----------------------------
+    def _get_note(self) -> str:
+        doc = self.collection.find_one({"user_id": self.user_id, "tool_id": self.tool_id})
+        if not doc or not doc.get("note"):
+            return "No note found."
+        return str(doc["note"])
+
+    def _overwrite_note(self, content: str) -> str:
+        content = (content or "").strip()
+        if not content:
+            return "Note content required."
+        self.collection.update_one(
+            {"user_id": self.user_id, "tool_id": self.tool_id},
+            {"$set": {"note": content, "updated_at": datetime.utcnow()}},
+            upsert=True,  # ✅ create if missing
+        )
+        return "Note saved."
+
+    def _str_replace(self, old_str: str, new_str: str) -> str:
+        if not old_str:
+            return "old_str is required."
+
+        doc = self.collection.find_one({"user_id": self.user_id, "tool_id": self.tool_id})
+        if not doc or not doc.get("note"):
+            return "No note found."
+
+        current_note = str(doc["note"])
+
+        # Case-insensitive search
+        if old_str.lower() not in current_note.lower():
+            return f"String '{old_str}' not found in note."
+
+        # Case-insensitive replacement
+        import re
+        updated_note = re.sub(re.escape(old_str), new_str, current_note, flags=re.IGNORECASE)
+
+        self.collection.update_one(
+            {"user_id": self.user_id, "tool_id": self.tool_id},
+            {"$set": {"note": updated_note, "updated_at": datetime.utcnow()}},
+        )
+        return "Note updated."
+
+    def _insert(self, line_number: int, text: str) -> str:
+        if not text:
+            return "Text is required."
+
+        doc = self.collection.find_one({"user_id": self.user_id, "tool_id": self.tool_id})
+        if not doc or not doc.get("note"):
+            return "No note found."
+
+        current_note = str(doc["note"])
+        lines = current_note.split("\n")
+
+        # Convert to 0-indexed and validate
+        index = line_number - 1
+        if index < 0 or index > len(lines):
+            return f"Invalid line number. Note has {len(lines)} lines."
+
+        lines.insert(index, text)
+        updated_note = "\n".join(lines)
+
+        self.collection.update_one(
+            {"user_id": self.user_id, "tool_id": self.tool_id},
+            {"$set": {"note": updated_note, "updated_at": datetime.utcnow()}},
+        )
+        return "Text inserted."
+
+    def _delete_note(self) -> str:
+        res = self.collection.delete_one({"user_id": self.user_id, "tool_id": self.tool_id})
+        return "Note deleted." if res.deleted_count else "No note found to delete."

application/agents/tools/tool_manager.py

@@ -28,7 +28,7 @@ class ToolManager:
         module = importlib.import_module(f"application.agents.tools.{tool_name}")
         for member_name, obj in inspect.getmembers(module, inspect.isclass):
             if issubclass(obj, Tool) and obj is not Tool:
-                if tool_name == "mcp_tool" and user_id:
+                if tool_name in {"mcp_tool", "notes", "memory"} and user_id:
                     return obj(tool_config, user_id)
                 else:
                     return obj(tool_config)
@@ -36,7 +36,7 @@ class ToolManager:
     def execute_action(self, tool_name, action_name, user_id=None, **kwargs):
         if tool_name not in self.tools:
             raise ValueError(f"Tool '{tool_name}' not loaded")
-        if tool_name == "mcp_tool" and user_id:
+        if tool_name in {"mcp_tool", "memory"} and user_id:
             tool_config = self.config.get(tool_name, {})
             tool = self.load_tool(tool_name, tool_config, user_id)
             return tool.execute_action(action_name, **kwargs)

application/api/user/__init__.py

@@ -0,0 +1,5 @@
+"""User API module - provides all user-related API endpoints"""
+
+from .routes import user
+
+__all__ = ["user"]

application/api/user/agents/__init__.py

@@ -0,0 +1,7 @@
+"""Agents module."""
+
+from .routes import agents_ns
+from .sharing import agents_sharing_ns
+from .webhooks import agents_webhooks_ns
+
+__all__ = ["agents_ns", "agents_sharing_ns", "agents_webhooks_ns"]

application/api/user/agents/routes.py

@@ -0,0 +1,910 @@
+"""Agent management routes."""
+
+import datetime
+import json
+import uuid
+
+from bson.dbref import DBRef
+from bson.objectid import ObjectId
+from flask import current_app, jsonify, make_response, request
+from flask_restx import fields, Namespace, Resource
+
+from application.api import api
+from application.api.user.base import (
+    agents_collection,
+    db,
+    ensure_user_doc,
+    handle_image_upload,
+    resolve_tool_details,
+    storage,
+    users_collection,
+)
+from application.utils import (
+    check_required_fields,
+    generate_image_url,
+    validate_required_fields,
+)
+
+
+agents_ns = Namespace("agents", description="Agent management operations", path="/api")
+
+
+@agents_ns.route("/get_agent")
+class GetAgent(Resource):
+    @api.doc(params={"id": "Agent ID"}, description="Get agent by ID")
+    def get(self):
+        if not (decoded_token := request.decoded_token):
+            return {"success": False}, 401
+        if not (agent_id := request.args.get("id")):
+            return {"success": False, "message": "ID required"}, 400
+        try:
+            agent = agents_collection.find_one(
+                {"_id": ObjectId(agent_id), "user": decoded_token["sub"]}
+            )
+            if not agent:
+                return {"status": "Not found"}, 404
+            data = {
+                "id": str(agent["_id"]),
+                "name": agent["name"],
+                "description": agent.get("description", ""),
+                "image": (
+                    generate_image_url(agent["image"]) if agent.get("image") else ""
+                ),
+                "source": (
+                    str(source_doc["_id"])
+                    if isinstance(agent.get("source"), DBRef)
+                    and (source_doc := db.dereference(agent.get("source")))
+                    else ""
+                ),
+                "sources": [
+                    (
+                        str(db.dereference(source_ref)["_id"])
+                        if isinstance(source_ref, DBRef) and db.dereference(source_ref)
+                        else source_ref
+                    )
+                    for source_ref in agent.get("sources", [])
+                    if (isinstance(source_ref, DBRef) and db.dereference(source_ref))
+                    or source_ref == "default"
+                ],
+                "chunks": agent["chunks"],
+                "retriever": agent.get("retriever", ""),
+                "prompt_id": agent.get("prompt_id", ""),
+                "tools": agent.get("tools", []),
+                "tool_details": resolve_tool_details(agent.get("tools", [])),
+                "agent_type": agent.get("agent_type", ""),
+                "status": agent.get("status", ""),
+                "json_schema": agent.get("json_schema"),
+                "created_at": agent.get("createdAt", ""),
+                "updated_at": agent.get("updatedAt", ""),
+                "last_used_at": agent.get("lastUsedAt", ""),
+                "key": (
+                    f"{agent['key'][:4]}...{agent['key'][-4:]}"
+                    if "key" in agent
+                    else ""
+                ),
+                "pinned": agent.get("pinned", False),
+                "shared": agent.get("shared_publicly", False),
+                "shared_metadata": agent.get("shared_metadata", {}),
+                "shared_token": agent.get("shared_token", ""),
+            }
+            return make_response(jsonify(data), 200)
+        except Exception as e:
+            current_app.logger.error(f"Agent fetch error: {e}", exc_info=True)
+            return {"success": False}, 400
+
+
+@agents_ns.route("/get_agents")
+class GetAgents(Resource):
+    @api.doc(description="Retrieve agents for the user")
+    def get(self):
+        if not (decoded_token := request.decoded_token):
+            return {"success": False}, 401
+        user = decoded_token.get("sub")
+        try:
+            user_doc = ensure_user_doc(user)
+            pinned_ids = set(user_doc.get("agent_preferences", {}).get("pinned", []))
+
+            agents = agents_collection.find({"user": user})
+            list_agents = [
+                {
+                    "id": str(agent["_id"]),
+                    "name": agent["name"],
+                    "description": agent.get("description", ""),
+                    "image": (
+                        generate_image_url(agent["image"]) if agent.get("image") else ""
+                    ),
+                    "source": (
+                        str(source_doc["_id"])
+                        if isinstance(agent.get("source"), DBRef)
+                        and (source_doc := db.dereference(agent.get("source")))
+                        else (
+                            agent.get("source", "")
+                            if agent.get("source") == "default"
+                            else ""
+                        )
+                    ),
+                    "sources": [
+                        (
+                            source_ref
+                            if source_ref == "default"
+                            else str(db.dereference(source_ref)["_id"])
+                        )
+                        for source_ref in agent.get("sources", [])
+                        if source_ref == "default"
+                        or (
+                            isinstance(source_ref, DBRef) and db.dereference(source_ref)
+                        )
+                    ],
+                    "chunks": agent["chunks"],
+                    "retriever": agent.get("retriever", ""),
+                    "prompt_id": agent.get("prompt_id", ""),
+                    "tools": agent.get("tools", []),
+                    "tool_details": resolve_tool_details(agent.get("tools", [])),
+                    "agent_type": agent.get("agent_type", ""),
+                    "status": agent.get("status", ""),
+                    "json_schema": agent.get("json_schema"),
+                    "created_at": agent.get("createdAt", ""),
+                    "updated_at": agent.get("updatedAt", ""),
+                    "last_used_at": agent.get("lastUsedAt", ""),
+                    "key": (
+                        f"{agent['key'][:4]}...{agent['key'][-4:]}"
+                        if "key" in agent
+                        else ""
+                    ),
+                    "pinned": str(agent["_id"]) in pinned_ids,
+                    "shared": agent.get("shared_publicly", False),
+                    "shared_metadata": agent.get("shared_metadata", {}),
+                    "shared_token": agent.get("shared_token", ""),
+                }
+                for agent in agents
+                if "source" in agent or "retriever" in agent
+            ]
+        except Exception as err:
+            current_app.logger.error(f"Error retrieving agents: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify(list_agents), 200)
+
+
+@agents_ns.route("/create_agent")
+class CreateAgent(Resource):
+    create_agent_model = api.model(
+        "CreateAgentModel",
+        {
+            "name": fields.String(required=True, description="Name of the agent"),
+            "description": fields.String(
+                required=True, description="Description of the agent"
+            ),
+            "image": fields.Raw(
+                required=False, description="Image file upload", type="file"
+            ),
+            "source": fields.String(
+                required=False, description="Source ID (legacy single source)"
+            ),
+            "sources": fields.List(
+                fields.String,
+                required=False,
+                description="List of source identifiers for multiple sources",
+            ),
+            "chunks": fields.Integer(required=True, description="Chunks count"),
+            "retriever": fields.String(required=True, description="Retriever ID"),
+            "prompt_id": fields.String(required=True, description="Prompt ID"),
+            "tools": fields.List(
+                fields.String, required=False, description="List of tool identifiers"
+            ),
+            "agent_type": fields.String(required=True, description="Type of the agent"),
+            "status": fields.String(
+                required=True, description="Status of the agent (draft or published)"
+            ),
+            "json_schema": fields.Raw(
+                required=False,
+                description="JSON schema for enforcing structured output format",
+            ),
+        },
+    )
+
+    @api.expect(create_agent_model)
+    @api.doc(description="Create a new agent")
+    def post(self):
+        if not (decoded_token := request.decoded_token):
+            return {"success": False}, 401
+        user = decoded_token.get("sub")
+        if request.content_type == "application/json":
+            data = request.get_json()
+        else:
+            data = request.form.to_dict()
+            if "tools" in data:
+                try:
+                    data["tools"] = json.loads(data["tools"])
+                except json.JSONDecodeError:
+                    data["tools"] = []
+            if "sources" in data:
+                try:
+                    data["sources"] = json.loads(data["sources"])
+                except json.JSONDecodeError:
+                    data["sources"] = []
+            if "json_schema" in data:
+                try:
+                    data["json_schema"] = json.loads(data["json_schema"])
+                except json.JSONDecodeError:
+                    data["json_schema"] = None
+        print(f"Received data: {data}")
+
+        # Validate JSON schema if provided
+
+        if data.get("json_schema"):
+            try:
+                # Basic validation - ensure it's a valid JSON structure
+
+                json_schema = data.get("json_schema")
+                if not isinstance(json_schema, dict):
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": False,
+                                "message": "JSON schema must be a valid JSON object",
+                            }
+                        ),
+                        400,
+                    )
+                # Validate that it has either a 'schema' property or is itself a schema
+
+                if "schema" not in json_schema and "type" not in json_schema:
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": False,
+                                "message": "JSON schema must contain either a 'schema' property or be a valid JSON schema with 'type' property",
+                            }
+                        ),
+                        400,
+                    )
+            except Exception as e:
+                return make_response(
+                    jsonify(
+                        {"success": False, "message": f"Invalid JSON schema: {str(e)}"}
+                    ),
+                    400,
+                )
+        if data.get("status") not in ["draft", "published"]:
+            return make_response(
+                jsonify(
+                    {
+                        "success": False,
+                        "message": "Status must be either 'draft' or 'published'",
+                    }
+                ),
+                400,
+            )
+        if data.get("status") == "published":
+            required_fields = [
+                "name",
+                "description",
+                "chunks",
+                "retriever",
+                "prompt_id",
+                "agent_type",
+            ]
+            # Require either source or sources (but not both)
+
+            if not data.get("source") and not data.get("sources"):
+                return make_response(
+                    jsonify(
+                        {
+                            "success": False,
+                            "message": "Either 'source' or 'sources' field is required for published agents",
+                        }
+                    ),
+                    400,
+                )
+            validate_fields = ["name", "description", "prompt_id", "agent_type"]
+        else:
+            required_fields = ["name"]
+            validate_fields = []
+        missing_fields = check_required_fields(data, required_fields)
+        invalid_fields = validate_required_fields(data, validate_fields)
+        if missing_fields:
+            return missing_fields
+        if invalid_fields:
+            return invalid_fields
+        image_url, error = handle_image_upload(request, "", user, storage)
+        if error:
+            return make_response(
+                jsonify({"success": False, "message": "Image upload failed"}), 400
+            )
+        try:
+            key = str(uuid.uuid4()) if data.get("status") == "published" else ""
+
+            sources_list = []
+            if data.get("sources") and len(data.get("sources", [])) > 0:
+                for source_id in data.get("sources", []):
+                    if source_id == "default":
+                        sources_list.append("default")
+                    elif ObjectId.is_valid(source_id):
+                        sources_list.append(DBRef("sources", ObjectId(source_id)))
+                source_field = ""
+            else:
+                source_value = data.get("source", "")
+                if source_value == "default":
+                    source_field = "default"
+                elif ObjectId.is_valid(source_value):
+                    source_field = DBRef("sources", ObjectId(source_value))
+                else:
+                    source_field = ""
+            new_agent = {
+                "user": user,
+                "name": data.get("name"),
+                "description": data.get("description", ""),
+                "image": image_url,
+                "source": source_field,
+                "sources": sources_list,
+                "chunks": data.get("chunks", ""),
+                "retriever": data.get("retriever", ""),
+                "prompt_id": data.get("prompt_id", ""),
+                "tools": data.get("tools", []),
+                "agent_type": data.get("agent_type", ""),
+                "status": data.get("status"),
+                "json_schema": data.get("json_schema"),
+                "createdAt": datetime.datetime.now(datetime.timezone.utc),
+                "updatedAt": datetime.datetime.now(datetime.timezone.utc),
+                "lastUsedAt": None,
+                "key": key,
+            }
+            if new_agent["chunks"] == "":
+                new_agent["chunks"] = "2"
+            if (
+                new_agent["source"] == ""
+                and new_agent["retriever"] == ""
+                and not new_agent["sources"]
+            ):
+                new_agent["retriever"] = "classic"
+            resp = agents_collection.insert_one(new_agent)
+            new_id = str(resp.inserted_id)
+        except Exception as err:
+            current_app.logger.error(f"Error creating agent: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"id": new_id, "key": key}), 201)
+
+
+@agents_ns.route("/update_agent/<string:agent_id>")
+class UpdateAgent(Resource):
+    update_agent_model = api.model(
+        "UpdateAgentModel",
+        {
+            "name": fields.String(required=True, description="New name of the agent"),
+            "description": fields.String(
+                required=True, description="New description of the agent"
+            ),
+            "image": fields.String(
+                required=False, description="New image URL or identifier"
+            ),
+            "source": fields.String(
+                required=False, description="Source ID (legacy single source)"
+            ),
+            "sources": fields.List(
+                fields.String,
+                required=False,
+                description="List of source identifiers for multiple sources",
+            ),
+            "chunks": fields.Integer(required=True, description="Chunks count"),
+            "retriever": fields.String(required=True, description="Retriever ID"),
+            "prompt_id": fields.String(required=True, description="Prompt ID"),
+            "tools": fields.List(
+                fields.String, required=False, description="List of tool identifiers"
+            ),
+            "agent_type": fields.String(required=True, description="Type of the agent"),
+            "status": fields.String(
+                required=True, description="Status of the agent (draft or published)"
+            ),
+            "json_schema": fields.Raw(
+                required=False,
+                description="JSON schema for enforcing structured output format",
+            ),
+        },
+    )
+
+    @api.expect(update_agent_model)
+    @api.doc(description="Update an existing agent")
+    def put(self, agent_id):
+        if not (decoded_token := request.decoded_token):
+            return make_response(
+                jsonify({"success": False, "message": "Unauthorized"}), 401
+            )
+        user = decoded_token.get("sub")
+
+        if not ObjectId.is_valid(agent_id):
+            return make_response(
+                jsonify({"success": False, "message": "Invalid agent ID format"}), 400
+            )
+        oid = ObjectId(agent_id)
+
+        try:
+            if request.content_type and "application/json" in request.content_type:
+                data = request.get_json()
+            else:
+                data = request.form.to_dict()
+                json_fields = ["tools", "sources", "json_schema"]
+                for field in json_fields:
+                    if field in data and data[field]:
+                        try:
+                            data[field] = json.loads(data[field])
+                        except json.JSONDecodeError:
+                            return make_response(
+                                jsonify(
+                                    {
+                                        "success": False,
+                                        "message": f"Invalid JSON format for field: {field}",
+                                    }
+                                ),
+                                400,
+                            )
+        except Exception as err:
+            current_app.logger.error(
+                f"Error parsing request data: {err}", exc_info=True
+            )
+            return make_response(
+                jsonify({"success": False, "message": "Invalid request data"}), 400
+            )
+        try:
+            existing_agent = agents_collection.find_one({"_id": oid, "user": user})
+        except Exception as err:
+            current_app.logger.error(
+                f"Error finding agent {agent_id}: {err}", exc_info=True
+            )
+            return make_response(
+                jsonify({"success": False, "message": "Database error finding agent"}),
+                500,
+            )
+        if not existing_agent:
+            return make_response(
+                jsonify(
+                    {"success": False, "message": "Agent not found or not authorized"}
+                ),
+                404,
+            )
+        image_url, error = handle_image_upload(
+            request, existing_agent.get("image", ""), user, storage
+        )
+        if error:
+            current_app.logger.error(
+                f"Image upload error for agent {agent_id}: {error}"
+            )
+            return make_response(
+                jsonify({"success": False, "message": f"Image upload failed: {error}"}),
+                400,
+            )
+        update_fields = {}
+        allowed_fields = [
+            "name",
+            "description",
+            "image",
+            "source",
+            "sources",
+            "chunks",
+            "retriever",
+            "prompt_id",
+            "tools",
+            "agent_type",
+            "status",
+            "json_schema",
+        ]
+
+        for field in allowed_fields:
+            if field not in data:
+                continue
+            if field == "status":
+                new_status = data.get("status")
+                if new_status not in ["draft", "published"]:
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": False,
+                                "message": "Invalid status value. Must be 'draft' or 'published'",
+                            }
+                        ),
+                        400,
+                    )
+                update_fields[field] = new_status
+            elif field == "source":
+                source_id = data.get("source")
+                if source_id == "default":
+                    update_fields[field] = "default"
+                elif source_id and ObjectId.is_valid(source_id):
+                    update_fields[field] = DBRef("sources", ObjectId(source_id))
+                elif source_id:
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": False,
+                                "message": f"Invalid source ID format: {source_id}",
+                            }
+                        ),
+                        400,
+                    )
+                else:
+                    update_fields[field] = ""
+            elif field == "sources":
+                sources_list = data.get("sources", [])
+                if sources_list and isinstance(sources_list, list):
+                    valid_sources = []
+                    for source_id in sources_list:
+                        if source_id == "default":
+                            valid_sources.append("default")
+                        elif ObjectId.is_valid(source_id):
+                            valid_sources.append(DBRef("sources", ObjectId(source_id)))
+                        else:
+                            return make_response(
+                                jsonify(
+                                    {
+                                        "success": False,
+                                        "message": f"Invalid source ID in list: {source_id}",
+                                    }
+                                ),
+                                400,
+                            )
+                    update_fields[field] = valid_sources
+                else:
+                    update_fields[field] = []
+            elif field == "chunks":
+                chunks_value = data.get("chunks")
+                if chunks_value == "" or chunks_value is None:
+                    update_fields[field] = "2"
+                else:
+                    try:
+                        chunks_int = int(chunks_value)
+                        if chunks_int < 0:
+                            return make_response(
+                                jsonify(
+                                    {
+                                        "success": False,
+                                        "message": "Chunks value must be a non-negative integer",
+                                    }
+                                ),
+                                400,
+                            )
+                        update_fields[field] = str(chunks_int)
+                    except (ValueError, TypeError):
+                        return make_response(
+                            jsonify(
+                                {
+                                    "success": False,
+                                    "message": f"Invalid chunks value: {chunks_value}",
+                                }
+                            ),
+                            400,
+                        )
+            elif field == "tools":
+                tools_list = data.get("tools", [])
+                if isinstance(tools_list, list):
+                    update_fields[field] = tools_list
+                else:
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": False,
+                                "message": "Tools must be a list",
+                            }
+                        ),
+                        400,
+                    )
+            elif field == "json_schema":
+                json_schema = data.get("json_schema")
+                if json_schema is not None:
+                    if not isinstance(json_schema, dict):
+                        return make_response(
+                            jsonify(
+                                {
+                                    "success": False,
+                                    "message": "JSON schema must be a valid object",
+                                }
+                            ),
+                            400,
+                        )
+                    update_fields[field] = json_schema
+                else:
+                    update_fields[field] = None
+            else:
+                value = data[field]
+                if field in ["name", "description", "prompt_id", "agent_type"]:
+                    if not value or not str(value).strip():
+                        return make_response(
+                            jsonify(
+                                {
+                                    "success": False,
+                                    "message": f"Field '{field}' cannot be empty",
+                                }
+                            ),
+                            400,
+                        )
+                update_fields[field] = value
+        if image_url:
+            update_fields["image"] = image_url
+        if not update_fields:
+            return make_response(
+                jsonify(
+                    {
+                        "success": False,
+                        "message": "No valid update data provided",
+                    }
+                ),
+                400,
+            )
+        newly_generated_key = None
+        final_status = update_fields.get("status", existing_agent.get("status"))
+
+        if final_status == "published":
+            required_published_fields = {
+                "name": "Agent name",
+                "description": "Agent description",
+                "chunks": "Chunks count",
+                "prompt_id": "Prompt",
+                "agent_type": "Agent type",
+            }
+
+            missing_published_fields = []
+            for req_field, field_label in required_published_fields.items():
+                final_value = update_fields.get(
+                    req_field, existing_agent.get(req_field)
+                )
+                if not final_value:
+                    missing_published_fields.append(field_label)
+            source_val = update_fields.get("source", existing_agent.get("source"))
+            sources_val = update_fields.get(
+                "sources", existing_agent.get("sources", [])
+            )
+
+            has_valid_source = (
+                isinstance(source_val, DBRef)
+                or source_val == "default"
+                or (isinstance(sources_val, list) and len(sources_val) > 0)
+            )
+
+            if not has_valid_source:
+                missing_published_fields.append("Source")
+            if missing_published_fields:
+                return make_response(
+                    jsonify(
+                        {
+                            "success": False,
+                            "message": f"Cannot publish agent. Missing or invalid required fields: {', '.join(missing_published_fields)}",
+                        }
+                    ),
+                    400,
+                )
+            if not existing_agent.get("key"):
+                newly_generated_key = str(uuid.uuid4())
+                update_fields["key"] = newly_generated_key
+        update_fields["updatedAt"] = datetime.datetime.now(datetime.timezone.utc)
+
+        try:
+            result = agents_collection.update_one(
+                {"_id": oid, "user": user}, {"$set": update_fields}
+            )
+
+            if result.matched_count == 0:
+                return make_response(
+                    jsonify(
+                        {
+                            "success": False,
+                            "message": "Agent not found or update failed",
+                        }
+                    ),
+                    404,
+                )
+            if result.modified_count == 0 and result.matched_count == 1:
+                return make_response(
+                    jsonify(
+                        {
+                            "success": True,
+                            "message": "No changes detected",
+                            "id": agent_id,
+                        }
+                    ),
+                    200,
+                )
+        except Exception as err:
+            current_app.logger.error(
+                f"Error updating agent {agent_id}: {err}", exc_info=True
+            )
+            return make_response(
+                jsonify({"success": False, "message": "Database error during update"}),
+                500,
+            )
+        response_data = {
+            "success": True,
+            "id": agent_id,
+            "message": "Agent updated successfully",
+        }
+        if newly_generated_key:
+            response_data["key"] = newly_generated_key
+        return make_response(jsonify(response_data), 200)
+
+
+@agents_ns.route("/delete_agent")
+class DeleteAgent(Resource):
+    @api.doc(params={"id": "ID of the agent"}, description="Delete an agent by ID")
+    def delete(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        agent_id = request.args.get("id")
+        if not agent_id:
+            return make_response(
+                jsonify({"success": False, "message": "ID is required"}), 400
+            )
+        try:
+            deleted_agent = agents_collection.find_one_and_delete(
+                {"_id": ObjectId(agent_id), "user": user}
+            )
+            if not deleted_agent:
+                return make_response(
+                    jsonify({"success": False, "message": "Agent not found"}), 404
+                )
+            deleted_id = str(deleted_agent["_id"])
+        except Exception as err:
+            current_app.logger.error(f"Error deleting agent: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"id": deleted_id}), 200)
+
+
+@agents_ns.route("/pinned_agents")
+class PinnedAgents(Resource):
+    @api.doc(description="Get pinned agents for the user")
+    def get(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user_id = decoded_token.get("sub")
+
+        try:
+            user_doc = ensure_user_doc(user_id)
+            pinned_ids = user_doc.get("agent_preferences", {}).get("pinned", [])
+
+            if not pinned_ids:
+                return make_response(jsonify([]), 200)
+            pinned_object_ids = [ObjectId(agent_id) for agent_id in pinned_ids]
+
+            pinned_agents_cursor = agents_collection.find(
+                {"_id": {"$in": pinned_object_ids}}
+            )
+            pinned_agents = list(pinned_agents_cursor)
+            existing_ids = {str(agent["_id"]) for agent in pinned_agents}
+
+            # Clean up any stale pinned IDs
+
+            stale_ids = [
+                agent_id for agent_id in pinned_ids if agent_id not in existing_ids
+            ]
+            if stale_ids:
+                users_collection.update_one(
+                    {"user_id": user_id},
+                    {"$pullAll": {"agent_preferences.pinned": stale_ids}},
+                )
+            list_pinned_agents = [
+                {
+                    "id": str(agent["_id"]),
+                    "name": agent.get("name", ""),
+                    "description": agent.get("description", ""),
+                    "image": (
+                        generate_image_url(agent["image"]) if agent.get("image") else ""
+                    ),
+                    "source": (
+                        str(db.dereference(agent["source"])["_id"])
+                        if "source" in agent
+                        and agent["source"]
+                        and isinstance(agent["source"], DBRef)
+                        and db.dereference(agent["source"]) is not None
+                        else ""
+                    ),
+                    "chunks": agent.get("chunks", ""),
+                    "retriever": agent.get("retriever", ""),
+                    "prompt_id": agent.get("prompt_id", ""),
+                    "tools": agent.get("tools", []),
+                    "tool_details": resolve_tool_details(agent.get("tools", [])),
+                    "agent_type": agent.get("agent_type", ""),
+                    "status": agent.get("status", ""),
+                    "created_at": agent.get("createdAt", ""),
+                    "updated_at": agent.get("updatedAt", ""),
+                    "last_used_at": agent.get("lastUsedAt", ""),
+                    "key": (
+                        f"{agent['key'][:4]}...{agent['key'][-4:]}"
+                        if "key" in agent
+                        else ""
+                    ),
+                    "pinned": True,
+                }
+                for agent in pinned_agents
+                if "source" in agent or "retriever" in agent
+            ]
+        except Exception as err:
+            current_app.logger.error(f"Error retrieving pinned agents: {err}")
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify(list_pinned_agents), 200)
+
+
+@agents_ns.route("/pin_agent")
+class PinAgent(Resource):
+    @api.doc(params={"id": "ID of the agent"}, description="Pin or unpin an agent")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user_id = decoded_token.get("sub")
+        agent_id = request.args.get("id")
+
+        if not agent_id:
+            return make_response(
+                jsonify({"success": False, "message": "ID is required"}), 400
+            )
+        try:
+            agent = agents_collection.find_one({"_id": ObjectId(agent_id)})
+            if not agent:
+                return make_response(
+                    jsonify({"success": False, "message": "Agent not found"}), 404
+                )
+            user_doc = ensure_user_doc(user_id)
+            pinned_list = user_doc.get("agent_preferences", {}).get("pinned", [])
+
+            if agent_id in pinned_list:
+                users_collection.update_one(
+                    {"user_id": user_id},
+                    {"$pull": {"agent_preferences.pinned": agent_id}},
+                )
+                action = "unpinned"
+            else:
+                users_collection.update_one(
+                    {"user_id": user_id},
+                    {"$addToSet": {"agent_preferences.pinned": agent_id}},
+                )
+                action = "pinned"
+        except Exception as err:
+            current_app.logger.error(f"Error pinning/unpinning agent: {err}")
+            return make_response(
+                jsonify({"success": False, "message": "Server error"}), 500
+            )
+        return make_response(jsonify({"success": True, "action": action}), 200)
+
+
+@agents_ns.route("/remove_shared_agent")
+class RemoveSharedAgent(Resource):
+    @api.doc(
+        params={"id": "ID of the shared agent"},
+        description="Remove a shared agent from the current user's shared list",
+    )
+    def delete(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user_id = decoded_token.get("sub")
+        agent_id = request.args.get("id")
+
+        if not agent_id:
+            return make_response(
+                jsonify({"success": False, "message": "ID is required"}), 400
+            )
+        try:
+            agent = agents_collection.find_one(
+                {"_id": ObjectId(agent_id), "shared_publicly": True}
+            )
+            if not agent:
+                return make_response(
+                    jsonify({"success": False, "message": "Shared agent not found"}),
+                    404,
+                )
+            ensure_user_doc(user_id)
+            users_collection.update_one(
+                {"user_id": user_id},
+                {
+                    "$pull": {
+                        "agent_preferences.shared_with_me": agent_id,
+                        "agent_preferences.pinned": agent_id,
+                    }
+                },
+            )
+
+            return make_response(jsonify({"success": True, "action": "removed"}), 200)
+        except Exception as err:
+            current_app.logger.error(f"Error removing shared agent: {err}")
+            return make_response(
+                jsonify({"success": False, "message": "Server error"}), 500
+            )

application/api/user/agents/sharing.py

@@ -0,0 +1,254 @@
+"""Agent management sharing functionality."""
+
+import datetime
+import secrets
+
+from bson import DBRef
+from bson.objectid import ObjectId
+from flask import current_app, jsonify, make_response, request
+from flask_restx import fields, Namespace, Resource
+
+from application.api import api
+from application.api.user.base import (
+    agents_collection,
+    db,
+    ensure_user_doc,
+    resolve_tool_details,
+    user_tools_collection,
+    users_collection,
+)
+from application.utils import generate_image_url
+
+agents_sharing_ns = Namespace(
+    "agents", description="Agent management operations", path="/api"
+)
+
+
+@agents_sharing_ns.route("/shared_agent")
+class SharedAgent(Resource):
+    @api.doc(
+        params={
+            "token": "Shared token of the agent",
+        },
+        description="Get a shared agent by token or ID",
+    )
+    def get(self):
+        shared_token = request.args.get("token")
+
+        if not shared_token:
+            return make_response(
+                jsonify({"success": False, "message": "Token or ID is required"}), 400
+            )
+        try:
+            query = {
+                "shared_publicly": True,
+                "shared_token": shared_token,
+            }
+            shared_agent = agents_collection.find_one(query)
+            if not shared_agent:
+                return make_response(
+                    jsonify({"success": False, "message": "Shared agent not found"}),
+                    404,
+                )
+            agent_id = str(shared_agent["_id"])
+            data = {
+                "id": agent_id,
+                "user": shared_agent.get("user", ""),
+                "name": shared_agent.get("name", ""),
+                "image": (
+                    generate_image_url(shared_agent["image"])
+                    if shared_agent.get("image")
+                    else ""
+                ),
+                "description": shared_agent.get("description", ""),
+                "source": (
+                    str(source_doc["_id"])
+                    if isinstance(shared_agent.get("source"), DBRef)
+                    and (source_doc := db.dereference(shared_agent.get("source")))
+                    else ""
+                ),
+                "chunks": shared_agent.get("chunks", "0"),
+                "retriever": shared_agent.get("retriever", "classic"),
+                "prompt_id": shared_agent.get("prompt_id", "default"),
+                "tools": shared_agent.get("tools", []),
+                "tool_details": resolve_tool_details(shared_agent.get("tools", [])),
+                "agent_type": shared_agent.get("agent_type", ""),
+                "status": shared_agent.get("status", ""),
+                "json_schema": shared_agent.get("json_schema"),
+                "created_at": shared_agent.get("createdAt", ""),
+                "updated_at": shared_agent.get("updatedAt", ""),
+                "shared": shared_agent.get("shared_publicly", False),
+                "shared_token": shared_agent.get("shared_token", ""),
+                "shared_metadata": shared_agent.get("shared_metadata", {}),
+            }
+
+            if data["tools"]:
+                enriched_tools = []
+                for tool in data["tools"]:
+                    tool_data = user_tools_collection.find_one({"_id": ObjectId(tool)})
+                    if tool_data:
+                        enriched_tools.append(tool_data.get("name", ""))
+                data["tools"] = enriched_tools
+            decoded_token = getattr(request, "decoded_token", None)
+            if decoded_token:
+                user_id = decoded_token.get("sub")
+                owner_id = shared_agent.get("user")
+
+                if user_id != owner_id:
+                    ensure_user_doc(user_id)
+                    users_collection.update_one(
+                        {"user_id": user_id},
+                        {"$addToSet": {"agent_preferences.shared_with_me": agent_id}},
+                    )
+            return make_response(jsonify(data), 200)
+        except Exception as err:
+            current_app.logger.error(f"Error retrieving shared agent: {err}")
+            return make_response(jsonify({"success": False}), 400)
+
+
+@agents_sharing_ns.route("/shared_agents")
+class SharedAgents(Resource):
+    @api.doc(description="Get shared agents explicitly shared with the user")
+    def get(self):
+        try:
+            decoded_token = request.decoded_token
+            if not decoded_token:
+                return make_response(jsonify({"success": False}), 401)
+            user_id = decoded_token.get("sub")
+
+            user_doc = ensure_user_doc(user_id)
+            shared_with_ids = user_doc.get("agent_preferences", {}).get(
+                "shared_with_me", []
+            )
+            shared_object_ids = [ObjectId(id) for id in shared_with_ids]
+
+            shared_agents_cursor = agents_collection.find(
+                {"_id": {"$in": shared_object_ids}, "shared_publicly": True}
+            )
+            shared_agents = list(shared_agents_cursor)
+
+            found_ids_set = {str(agent["_id"]) for agent in shared_agents}
+            stale_ids = [id for id in shared_with_ids if id not in found_ids_set]
+            if stale_ids:
+                users_collection.update_one(
+                    {"user_id": user_id},
+                    {"$pullAll": {"agent_preferences.shared_with_me": stale_ids}},
+                )
+            pinned_ids = set(user_doc.get("agent_preferences", {}).get("pinned", []))
+
+            list_shared_agents = [
+                {
+                    "id": str(agent["_id"]),
+                    "name": agent.get("name", ""),
+                    "description": agent.get("description", ""),
+                    "image": (
+                        generate_image_url(agent["image"]) if agent.get("image") else ""
+                    ),
+                    "tools": agent.get("tools", []),
+                    "tool_details": resolve_tool_details(agent.get("tools", [])),
+                    "agent_type": agent.get("agent_type", ""),
+                    "status": agent.get("status", ""),
+                    "json_schema": agent.get("json_schema"),
+                    "created_at": agent.get("createdAt", ""),
+                    "updated_at": agent.get("updatedAt", ""),
+                    "pinned": str(agent["_id"]) in pinned_ids,
+                    "shared": agent.get("shared_publicly", False),
+                    "shared_token": agent.get("shared_token", ""),
+                    "shared_metadata": agent.get("shared_metadata", {}),
+                }
+                for agent in shared_agents
+            ]
+
+            return make_response(jsonify(list_shared_agents), 200)
+        except Exception as err:
+            current_app.logger.error(f"Error retrieving shared agents: {err}")
+            return make_response(jsonify({"success": False}), 400)
+
+
+@agents_sharing_ns.route("/share_agent")
+class ShareAgent(Resource):
+    @api.expect(
+        api.model(
+            "ShareAgentModel",
+            {
+                "id": fields.String(required=True, description="ID of the agent"),
+                "shared": fields.Boolean(
+                    required=True, description="Share or unshare the agent"
+                ),
+                "username": fields.String(
+                    required=False, description="Name of the user"
+                ),
+            },
+        )
+    )
+    @api.doc(description="Share or unshare an agent")
+    def put(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+
+        data = request.get_json()
+        if not data:
+            return make_response(
+                jsonify({"success": False, "message": "Missing JSON body"}), 400
+            )
+        agent_id = data.get("id")
+        shared = data.get("shared")
+        username = data.get("username", "")
+
+        if not agent_id:
+            return make_response(
+                jsonify({"success": False, "message": "ID is required"}), 400
+            )
+        if shared is None:
+            return make_response(
+                jsonify(
+                    {
+                        "success": False,
+                        "message": "Shared parameter is required and must be true or false",
+                    }
+                ),
+                400,
+            )
+        try:
+            try:
+                agent_oid = ObjectId(agent_id)
+            except Exception:
+                return make_response(
+                    jsonify({"success": False, "message": "Invalid agent ID"}), 400
+                )
+            agent = agents_collection.find_one({"_id": agent_oid, "user": user})
+            if not agent:
+                return make_response(
+                    jsonify({"success": False, "message": "Agent not found"}), 404
+                )
+            if shared:
+                shared_metadata = {
+                    "shared_by": username,
+                    "shared_at": datetime.datetime.now(datetime.timezone.utc),
+                }
+                shared_token = secrets.token_urlsafe(32)
+                agents_collection.update_one(
+                    {"_id": agent_oid, "user": user},
+                    {
+                        "$set": {
+                            "shared_publicly": shared,
+                            "shared_metadata": shared_metadata,
+                            "shared_token": shared_token,
+                        }
+                    },
+                )
+            else:
+                agents_collection.update_one(
+                    {"_id": agent_oid, "user": user},
+                    {"$set": {"shared_publicly": shared, "shared_token": None}},
+                    {"$unset": {"shared_metadata": ""}},
+                )
+        except Exception as err:
+            current_app.logger.error(f"Error sharing/unsharing agent: {err}")
+            return make_response(jsonify({"success": False, "error": str(err)}), 400)
+        shared_token = shared_token if shared else None
+        return make_response(
+            jsonify({"success": True, "shared_token": shared_token}), 200
+        )

application/api/user/agents/webhooks.py

@@ -0,0 +1,119 @@
+"""Agent management webhook handlers."""
+
+import secrets
+
+from bson.objectid import ObjectId
+from flask import current_app, jsonify, make_response, request
+from flask_restx import Namespace, Resource
+
+from application.api import api
+from application.api.user.base import agents_collection, require_agent
+from application.api.user.tasks import process_agent_webhook
+from application.core.settings import settings
+
+
+agents_webhooks_ns = Namespace(
+    "agents", description="Agent management operations", path="/api"
+)
+
+
+@agents_webhooks_ns.route("/agent_webhook")
+class AgentWebhook(Resource):
+    @api.doc(
+        params={"id": "ID of the agent"},
+        description="Generate webhook URL for the agent",
+    )
+    def get(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        agent_id = request.args.get("id")
+        if not agent_id:
+            return make_response(
+                jsonify({"success": False, "message": "ID is required"}), 400
+            )
+        try:
+            agent = agents_collection.find_one(
+                {"_id": ObjectId(agent_id), "user": user}
+            )
+            if not agent:
+                return make_response(
+                    jsonify({"success": False, "message": "Agent not found"}), 404
+                )
+            webhook_token = agent.get("incoming_webhook_token")
+            if not webhook_token:
+                webhook_token = secrets.token_urlsafe(32)
+                agents_collection.update_one(
+                    {"_id": ObjectId(agent_id), "user": user},
+                    {"$set": {"incoming_webhook_token": webhook_token}},
+                )
+            base_url = settings.API_URL.rstrip("/")
+            full_webhook_url = f"{base_url}/api/webhooks/agents/{webhook_token}"
+        except Exception as err:
+            current_app.logger.error(
+                f"Error generating webhook URL: {err}", exc_info=True
+            )
+            return make_response(
+                jsonify({"success": False, "message": "Error generating webhook URL"}),
+                400,
+            )
+        return make_response(
+            jsonify({"success": True, "webhook_url": full_webhook_url}), 200
+        )
+
+
+@agents_webhooks_ns.route("/webhooks/agents/<string:webhook_token>")
+class AgentWebhookListener(Resource):
+    method_decorators = [require_agent]
+
+    def _enqueue_webhook_task(self, agent_id_str, payload, source_method):
+        if not payload:
+            current_app.logger.warning(
+                f"Webhook ({source_method}) received for agent {agent_id_str} with empty payload."
+            )
+        current_app.logger.info(
+            f"Incoming {source_method} webhook for agent {agent_id_str}. Enqueuing task with payload: {payload}"
+        )
+
+        try:
+            task = process_agent_webhook.delay(
+                agent_id=agent_id_str,
+                payload=payload,
+            )
+            current_app.logger.info(
+                f"Task {task.id} enqueued for agent {agent_id_str} ({source_method})."
+            )
+            return make_response(jsonify({"success": True, "task_id": task.id}), 200)
+        except Exception as err:
+            current_app.logger.error(
+                f"Error enqueuing webhook task ({source_method}) for agent {agent_id_str}: {err}",
+                exc_info=True,
+            )
+            return make_response(
+                jsonify({"success": False, "message": "Error processing webhook"}), 500
+            )
+
+    @api.doc(
+        description="Webhook listener for agent events (POST). Expects JSON payload, which is used to trigger processing.",
+    )
+    def post(self, webhook_token, agent, agent_id_str):
+        payload = request.get_json()
+        if payload is None:
+            return make_response(
+                jsonify(
+                    {
+                        "success": False,
+                        "message": "Invalid or missing JSON data in request body",
+                    }
+                ),
+                400,
+            )
+        return self._enqueue_webhook_task(agent_id_str, payload, source_method="POST")
+
+    @api.doc(
+        description="Webhook listener for agent events (GET). Uses URL query parameters as payload to trigger processing.",
+    )
+    def get(self, webhook_token, agent, agent_id_str):
+        payload = request.args.to_dict(flat=True)
+        return self._enqueue_webhook_task(agent_id_str, payload, source_method="GET")

application/api/user/analytics/__init__.py

@@ -0,0 +1,5 @@
+"""Analytics module."""
+
+from .routes import analytics_ns
+
+__all__ = ["analytics_ns"]

application/api/user/analytics/routes.py

@@ -0,0 +1,540 @@
+"""Analytics and reporting routes."""
+
+import datetime
+
+from bson.objectid import ObjectId
+from flask import current_app, jsonify, make_response, request
+from flask_restx import fields, Namespace, Resource
+
+from application.api import api
+from application.api.user.base import (
+    agents_collection,
+    conversations_collection,
+    generate_date_range,
+    generate_hourly_range,
+    generate_minute_range,
+    token_usage_collection,
+    user_logs_collection,
+)
+
+analytics_ns = Namespace(
+    "analytics", description="Analytics and reporting operations", path="/api"
+)
+
+
+@analytics_ns.route("/get_message_analytics")
+class GetMessageAnalytics(Resource):
+    get_message_analytics_model = api.model(
+        "GetMessageAnalyticsModel",
+        {
+            "api_key_id": fields.String(required=False, description="API Key ID"),
+            "filter_option": fields.String(
+                required=False,
+                description="Filter option for analytics",
+                default="last_30_days",
+                enum=[
+                    "last_hour",
+                    "last_24_hour",
+                    "last_7_days",
+                    "last_15_days",
+                    "last_30_days",
+                ],
+            ),
+        },
+    )
+
+    @api.expect(get_message_analytics_model)
+    @api.doc(description="Get message analytics based on filter option")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        api_key_id = data.get("api_key_id")
+        filter_option = data.get("filter_option", "last_30_days")
+
+        try:
+            api_key = (
+                agents_collection.find_one({"_id": ObjectId(api_key_id), "user": user})[
+                    "key"
+                ]
+                if api_key_id
+                else None
+            )
+        except Exception as err:
+            current_app.logger.error(f"Error getting API key: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        end_date = datetime.datetime.now(datetime.timezone.utc)
+
+        if filter_option == "last_hour":
+            start_date = end_date - datetime.timedelta(hours=1)
+            group_format = "%Y-%m-%d %H:%M:00"
+        elif filter_option == "last_24_hour":
+            start_date = end_date - datetime.timedelta(hours=24)
+            group_format = "%Y-%m-%d %H:00"
+        else:
+            if filter_option in ["last_7_days", "last_15_days", "last_30_days"]:
+                filter_days = (
+                    6
+                    if filter_option == "last_7_days"
+                    else 14 if filter_option == "last_15_days" else 29
+                )
+            else:
+                return make_response(
+                    jsonify({"success": False, "message": "Invalid option"}), 400
+                )
+            start_date = end_date - datetime.timedelta(days=filter_days)
+            start_date = start_date.replace(hour=0, minute=0, second=0, microsecond=0)
+            end_date = end_date.replace(
+                hour=23, minute=59, second=59, microsecond=999999
+            )
+            group_format = "%Y-%m-%d"
+        try:
+            match_stage = {
+                "$match": {
+                    "user": user,
+                }
+            }
+            if api_key:
+                match_stage["$match"]["api_key"] = api_key
+            pipeline = [
+                match_stage,
+                {"$unwind": "$queries"},
+                {
+                    "$match": {
+                        "queries.timestamp": {"$gte": start_date, "$lte": end_date}
+                    }
+                },
+                {
+                    "$group": {
+                        "_id": {
+                            "$dateToString": {
+                                "format": group_format,
+                                "date": "$queries.timestamp",
+                            }
+                        },
+                        "count": {"$sum": 1},
+                    }
+                },
+                {"$sort": {"_id": 1}},
+            ]
+
+            message_data = conversations_collection.aggregate(pipeline)
+
+            if filter_option == "last_hour":
+                intervals = generate_minute_range(start_date, end_date)
+            elif filter_option == "last_24_hour":
+                intervals = generate_hourly_range(start_date, end_date)
+            else:
+                intervals = generate_date_range(start_date, end_date)
+            daily_messages = {interval: 0 for interval in intervals}
+
+            for entry in message_data:
+                daily_messages[entry["_id"]] = entry["count"]
+        except Exception as err:
+            current_app.logger.error(
+                f"Error getting message analytics: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(
+            jsonify({"success": True, "messages": daily_messages}), 200
+        )
+
+
+@analytics_ns.route("/get_token_analytics")
+class GetTokenAnalytics(Resource):
+    get_token_analytics_model = api.model(
+        "GetTokenAnalyticsModel",
+        {
+            "api_key_id": fields.String(required=False, description="API Key ID"),
+            "filter_option": fields.String(
+                required=False,
+                description="Filter option for analytics",
+                default="last_30_days",
+                enum=[
+                    "last_hour",
+                    "last_24_hour",
+                    "last_7_days",
+                    "last_15_days",
+                    "last_30_days",
+                ],
+            ),
+        },
+    )
+
+    @api.expect(get_token_analytics_model)
+    @api.doc(description="Get token analytics data")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        api_key_id = data.get("api_key_id")
+        filter_option = data.get("filter_option", "last_30_days")
+
+        try:
+            api_key = (
+                agents_collection.find_one({"_id": ObjectId(api_key_id), "user": user})[
+                    "key"
+                ]
+                if api_key_id
+                else None
+            )
+        except Exception as err:
+            current_app.logger.error(f"Error getting API key: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        end_date = datetime.datetime.now(datetime.timezone.utc)
+
+        if filter_option == "last_hour":
+            start_date = end_date - datetime.timedelta(hours=1)
+            group_format = "%Y-%m-%d %H:%M:00"
+            group_stage = {
+                "$group": {
+                    "_id": {
+                        "minute": {
+                            "$dateToString": {
+                                "format": group_format,
+                                "date": "$timestamp",
+                            }
+                        }
+                    },
+                    "total_tokens": {
+                        "$sum": {"$add": ["$prompt_tokens", "$generated_tokens"]}
+                    },
+                }
+            }
+        elif filter_option == "last_24_hour":
+            start_date = end_date - datetime.timedelta(hours=24)
+            group_format = "%Y-%m-%d %H:00"
+            group_stage = {
+                "$group": {
+                    "_id": {
+                        "hour": {
+                            "$dateToString": {
+                                "format": group_format,
+                                "date": "$timestamp",
+                            }
+                        }
+                    },
+                    "total_tokens": {
+                        "$sum": {"$add": ["$prompt_tokens", "$generated_tokens"]}
+                    },
+                }
+            }
+        else:
+            if filter_option in ["last_7_days", "last_15_days", "last_30_days"]:
+                filter_days = (
+                    6
+                    if filter_option == "last_7_days"
+                    else (14 if filter_option == "last_15_days" else 29)
+                )
+            else:
+                return make_response(
+                    jsonify({"success": False, "message": "Invalid option"}), 400
+                )
+            start_date = end_date - datetime.timedelta(days=filter_days)
+            start_date = start_date.replace(hour=0, minute=0, second=0, microsecond=0)
+            end_date = end_date.replace(
+                hour=23, minute=59, second=59, microsecond=999999
+            )
+            group_format = "%Y-%m-%d"
+            group_stage = {
+                "$group": {
+                    "_id": {
+                        "day": {
+                            "$dateToString": {
+                                "format": group_format,
+                                "date": "$timestamp",
+                            }
+                        }
+                    },
+                    "total_tokens": {
+                        "$sum": {"$add": ["$prompt_tokens", "$generated_tokens"]}
+                    },
+                }
+            }
+        try:
+            match_stage = {
+                "$match": {
+                    "user_id": user,
+                    "timestamp": {"$gte": start_date, "$lte": end_date},
+                }
+            }
+            if api_key:
+                match_stage["$match"]["api_key"] = api_key
+            token_usage_data = token_usage_collection.aggregate(
+                [
+                    match_stage,
+                    group_stage,
+                    {"$sort": {"_id": 1}},
+                ]
+            )
+
+            if filter_option == "last_hour":
+                intervals = generate_minute_range(start_date, end_date)
+            elif filter_option == "last_24_hour":
+                intervals = generate_hourly_range(start_date, end_date)
+            else:
+                intervals = generate_date_range(start_date, end_date)
+            daily_token_usage = {interval: 0 for interval in intervals}
+
+            for entry in token_usage_data:
+                if filter_option == "last_hour":
+                    daily_token_usage[entry["_id"]["minute"]] = entry["total_tokens"]
+                elif filter_option == "last_24_hour":
+                    daily_token_usage[entry["_id"]["hour"]] = entry["total_tokens"]
+                else:
+                    daily_token_usage[entry["_id"]["day"]] = entry["total_tokens"]
+        except Exception as err:
+            current_app.logger.error(
+                f"Error getting token analytics: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(
+            jsonify({"success": True, "token_usage": daily_token_usage}), 200
+        )
+
+
+@analytics_ns.route("/get_feedback_analytics")
+class GetFeedbackAnalytics(Resource):
+    get_feedback_analytics_model = api.model(
+        "GetFeedbackAnalyticsModel",
+        {
+            "api_key_id": fields.String(required=False, description="API Key ID"),
+            "filter_option": fields.String(
+                required=False,
+                description="Filter option for analytics",
+                default="last_30_days",
+                enum=[
+                    "last_hour",
+                    "last_24_hour",
+                    "last_7_days",
+                    "last_15_days",
+                    "last_30_days",
+                ],
+            ),
+        },
+    )
+
+    @api.expect(get_feedback_analytics_model)
+    @api.doc(description="Get feedback analytics data")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        api_key_id = data.get("api_key_id")
+        filter_option = data.get("filter_option", "last_30_days")
+
+        try:
+            api_key = (
+                agents_collection.find_one({"_id": ObjectId(api_key_id), "user": user})[
+                    "key"
+                ]
+                if api_key_id
+                else None
+            )
+        except Exception as err:
+            current_app.logger.error(f"Error getting API key: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        end_date = datetime.datetime.now(datetime.timezone.utc)
+
+        if filter_option == "last_hour":
+            start_date = end_date - datetime.timedelta(hours=1)
+            group_format = "%Y-%m-%d %H:%M:00"
+            date_field = {
+                "$dateToString": {
+                    "format": group_format,
+                    "date": "$queries.feedback_timestamp",
+                }
+            }
+        elif filter_option == "last_24_hour":
+            start_date = end_date - datetime.timedelta(hours=24)
+            group_format = "%Y-%m-%d %H:00"
+            date_field = {
+                "$dateToString": {
+                    "format": group_format,
+                    "date": "$queries.feedback_timestamp",
+                }
+            }
+        else:
+            if filter_option in ["last_7_days", "last_15_days", "last_30_days"]:
+                filter_days = (
+                    6
+                    if filter_option == "last_7_days"
+                    else (14 if filter_option == "last_15_days" else 29)
+                )
+            else:
+                return make_response(
+                    jsonify({"success": False, "message": "Invalid option"}), 400
+                )
+            start_date = end_date - datetime.timedelta(days=filter_days)
+            start_date = start_date.replace(hour=0, minute=0, second=0, microsecond=0)
+            end_date = end_date.replace(
+                hour=23, minute=59, second=59, microsecond=999999
+            )
+            group_format = "%Y-%m-%d"
+            date_field = {
+                "$dateToString": {
+                    "format": group_format,
+                    "date": "$queries.feedback_timestamp",
+                }
+            }
+        try:
+            match_stage = {
+                "$match": {
+                    "queries.feedback_timestamp": {
+                        "$gte": start_date,
+                        "$lte": end_date,
+                    },
+                    "queries.feedback": {"$exists": True},
+                }
+            }
+            if api_key:
+                match_stage["$match"]["api_key"] = api_key
+            pipeline = [
+                match_stage,
+                {"$unwind": "$queries"},
+                {"$match": {"queries.feedback": {"$exists": True}}},
+                {
+                    "$group": {
+                        "_id": {"time": date_field, "feedback": "$queries.feedback"},
+                        "count": {"$sum": 1},
+                    }
+                },
+                {
+                    "$group": {
+                        "_id": "$_id.time",
+                        "positive": {
+                            "$sum": {
+                                "$cond": [
+                                    {"$eq": ["$_id.feedback", "LIKE"]},
+                                    "$count",
+                                    0,
+                                ]
+                            }
+                        },
+                        "negative": {
+                            "$sum": {
+                                "$cond": [
+                                    {"$eq": ["$_id.feedback", "DISLIKE"]},
+                                    "$count",
+                                    0,
+                                ]
+                            }
+                        },
+                    }
+                },
+                {"$sort": {"_id": 1}},
+            ]
+
+            feedback_data = conversations_collection.aggregate(pipeline)
+
+            if filter_option == "last_hour":
+                intervals = generate_minute_range(start_date, end_date)
+            elif filter_option == "last_24_hour":
+                intervals = generate_hourly_range(start_date, end_date)
+            else:
+                intervals = generate_date_range(start_date, end_date)
+            daily_feedback = {
+                interval: {"positive": 0, "negative": 0} for interval in intervals
+            }
+
+            for entry in feedback_data:
+                daily_feedback[entry["_id"]] = {
+                    "positive": entry["positive"],
+                    "negative": entry["negative"],
+                }
+        except Exception as err:
+            current_app.logger.error(
+                f"Error getting feedback analytics: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(
+            jsonify({"success": True, "feedback": daily_feedback}), 200
+        )
+
+
+@analytics_ns.route("/get_user_logs")
+class GetUserLogs(Resource):
+    get_user_logs_model = api.model(
+        "GetUserLogsModel",
+        {
+            "page": fields.Integer(
+                required=False,
+                description="Page number for pagination",
+                default=1,
+            ),
+            "api_key_id": fields.String(required=False, description="API Key ID"),
+            "page_size": fields.Integer(
+                required=False,
+                description="Number of logs per page",
+                default=10,
+            ),
+        },
+    )
+
+    @api.expect(get_user_logs_model)
+    @api.doc(description="Get user logs with pagination")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        page = int(data.get("page", 1))
+        api_key_id = data.get("api_key_id")
+        page_size = int(data.get("page_size", 10))
+        skip = (page - 1) * page_size
+
+        try:
+            api_key = (
+                agents_collection.find_one({"_id": ObjectId(api_key_id)})["key"]
+                if api_key_id
+                else None
+            )
+        except Exception as err:
+            current_app.logger.error(f"Error getting API key: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        query = {"user": user}
+        if api_key:
+            query = {"api_key": api_key}
+        items_cursor = (
+            user_logs_collection.find(query)
+            .sort("timestamp", -1)
+            .skip(skip)
+            .limit(page_size + 1)
+        )
+        items = list(items_cursor)
+
+        results = [
+            {
+                "id": str(item.get("_id")),
+                "action": item.get("action"),
+                "level": item.get("level"),
+                "user": item.get("user"),
+                "question": item.get("question"),
+                "sources": item.get("sources"),
+                "retriever_params": item.get("retriever_params"),
+                "timestamp": item.get("timestamp"),
+            }
+            for item in items[:page_size]
+        ]
+
+        has_more = len(items) > page_size
+
+        return make_response(
+            jsonify(
+                {
+                    "success": True,
+                    "logs": results,
+                    "page": page,
+                    "page_size": page_size,
+                    "has_more": has_more,
+                }
+            ),
+            200,
+        )

application/api/user/attachments/__init__.py

@@ -0,0 +1,5 @@
+"""Attachments module."""
+
+from .routes import attachments_ns
+
+__all__ = ["attachments_ns"]

application/api/user/attachments/routes.py

@@ -0,0 +1,150 @@
+"""File attachments and media routes."""
+
+import os
+
+from bson.objectid import ObjectId
+from flask import current_app, jsonify, make_response, request
+from flask_restx import fields, Namespace, Resource
+
+from application.api import api
+from application.api.user.base import agents_collection, storage
+from application.api.user.tasks import store_attachment
+from application.core.settings import settings
+from application.tts.google_tts import GoogleTTS
+from application.utils import safe_filename
+
+
+attachments_ns = Namespace(
+    "attachments", description="File attachments and media operations", path="/api"
+)
+
+
+@attachments_ns.route("/store_attachment")
+class StoreAttachment(Resource):
+    @api.expect(
+        api.model(
+            "AttachmentModel",
+            {
+                "file": fields.Raw(required=True, description="File to upload"),
+                "api_key": fields.String(
+                    required=False, description="API key (optional)"
+                ),
+            },
+        )
+    )
+    @api.doc(
+        description="Stores a single attachment without vectorization or training. Supports user or API key authentication."
+    )
+    def post(self):
+        decoded_token = getattr(request, "decoded_token", None)
+        api_key = request.form.get("api_key") or request.args.get("api_key")
+        file = request.files.get("file")
+
+        if not file or file.filename == "":
+            return make_response(
+                jsonify({"status": "error", "message": "Missing file"}),
+                400,
+            )
+        user = None
+        if decoded_token:
+            user = safe_filename(decoded_token.get("sub"))
+        elif api_key:
+            agent = agents_collection.find_one({"key": api_key})
+            if not agent:
+                return make_response(
+                    jsonify({"success": False, "message": "Invalid API key"}), 401
+                )
+            user = safe_filename(agent.get("user"))
+        else:
+            return make_response(
+                jsonify({"success": False, "message": "Authentication required"}), 401
+            )
+        try:
+            attachment_id = ObjectId()
+            original_filename = safe_filename(os.path.basename(file.filename))
+            relative_path = f"{settings.UPLOAD_FOLDER}/{user}/attachments/{str(attachment_id)}/{original_filename}"
+
+            metadata = storage.save_file(file, relative_path)
+
+            file_info = {
+                "filename": original_filename,
+                "attachment_id": str(attachment_id),
+                "path": relative_path,
+                "metadata": metadata,
+            }
+
+            task = store_attachment.delay(file_info, user)
+
+            return make_response(
+                jsonify(
+                    {
+                        "success": True,
+                        "task_id": task.id,
+                        "message": "File uploaded successfully. Processing started.",
+                    }
+                ),
+                200,
+            )
+        except Exception as err:
+            current_app.logger.error(f"Error storing attachment: {err}", exc_info=True)
+            return make_response(jsonify({"success": False, "error": str(err)}), 400)
+
+
+@attachments_ns.route("/images/<path:image_path>")
+class ServeImage(Resource):
+    @api.doc(description="Serve an image from storage")
+    def get(self, image_path):
+        try:
+            file_obj = storage.get_file(image_path)
+            extension = image_path.split(".")[-1].lower()
+            content_type = f"image/{extension}"
+            if extension == "jpg":
+                content_type = "image/jpeg"
+            response = make_response(file_obj.read())
+            response.headers.set("Content-Type", content_type)
+            response.headers.set("Cache-Control", "max-age=86400")
+
+            return response
+        except FileNotFoundError:
+            return make_response(
+                jsonify({"success": False, "message": "Image not found"}), 404
+            )
+        except Exception as e:
+            current_app.logger.error(f"Error serving image: {e}")
+            return make_response(
+                jsonify({"success": False, "message": "Error retrieving image"}), 500
+            )
+
+
+@attachments_ns.route("/tts")
+class TextToSpeech(Resource):
+    tts_model = api.model(
+        "TextToSpeechModel",
+        {
+            "text": fields.String(
+                required=True, description="Text to be synthesized as audio"
+            ),
+        },
+    )
+
+    @api.expect(tts_model)
+    @api.doc(description="Synthesize audio speech from text")
+    def post(self):
+        data = request.get_json()
+        text = data["text"]
+        try:
+            tts_instance = GoogleTTS()
+            audio_base64, detected_language = tts_instance.text_to_speech(text)
+            return make_response(
+                jsonify(
+                    {
+                        "success": True,
+                        "audio_base64": audio_base64,
+                        "lang": detected_language,
+                    }
+                ),
+                200,
+            )
+        except Exception as err:
+            current_app.logger.error(f"Error synthesizing audio: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)

application/api/user/base.py

@@ -0,0 +1,222 @@
+"""
+Shared utilities, database connections, and helper functions for user API routes.
+"""
+
+import datetime
+import os
+import uuid
+from functools import wraps
+from typing import Optional, Tuple
+
+from bson.objectid import ObjectId
+from flask import current_app, jsonify, make_response, Response
+from pymongo import ReturnDocument
+from werkzeug.utils import secure_filename
+
+from application.core.mongo_db import MongoDB
+from application.core.settings import settings
+from application.storage.storage_creator import StorageCreator
+from application.vectorstore.vector_creator import VectorCreator
+
+
+storage = StorageCreator.get_storage()
+
+
+mongo = MongoDB.get_client()
+db = mongo[settings.MONGO_DB_NAME]
+
+
+conversations_collection = db["conversations"]
+sources_collection = db["sources"]
+prompts_collection = db["prompts"]
+feedback_collection = db["feedback"]
+agents_collection = db["agents"]
+token_usage_collection = db["token_usage"]
+shared_conversations_collections = db["shared_conversations"]
+users_collection = db["users"]
+user_logs_collection = db["user_logs"]
+user_tools_collection = db["user_tools"]
+attachments_collection = db["attachments"]
+
+
+try:
+    agents_collection.create_index(
+        [("shared", 1)],
+        name="shared_index",
+        background=True,
+    )
+    users_collection.create_index("user_id", unique=True)
+except Exception as e:
+    print("Error creating indexes:", e)
+current_dir = os.path.dirname(
+    os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+)
+
+
+def generate_minute_range(start_date, end_date):
+    """Generate a dictionary with minute-level time ranges."""
+    return {
+        (start_date + datetime.timedelta(minutes=i)).strftime("%Y-%m-%d %H:%M:00"): 0
+        for i in range(int((end_date - start_date).total_seconds() // 60) + 1)
+    }
+
+
+def generate_hourly_range(start_date, end_date):
+    """Generate a dictionary with hourly time ranges."""
+    return {
+        (start_date + datetime.timedelta(hours=i)).strftime("%Y-%m-%d %H:00"): 0
+        for i in range(int((end_date - start_date).total_seconds() // 3600) + 1)
+    }
+
+
+def generate_date_range(start_date, end_date):
+    """Generate a dictionary with daily date ranges."""
+    return {
+        (start_date + datetime.timedelta(days=i)).strftime("%Y-%m-%d"): 0
+        for i in range((end_date - start_date).days + 1)
+    }
+
+
+def ensure_user_doc(user_id):
+    """
+    Ensure user document exists with proper agent preferences structure.
+
+    Args:
+        user_id: The user ID to ensure
+
+    Returns:
+        The user document
+    """
+    default_prefs = {
+        "pinned": [],
+        "shared_with_me": [],
+    }
+
+    user_doc = users_collection.find_one_and_update(
+        {"user_id": user_id},
+        {"$setOnInsert": {"agent_preferences": default_prefs}},
+        upsert=True,
+        return_document=ReturnDocument.AFTER,
+    )
+
+    prefs = user_doc.get("agent_preferences", {})
+    updates = {}
+    if "pinned" not in prefs:
+        updates["agent_preferences.pinned"] = []
+    if "shared_with_me" not in prefs:
+        updates["agent_preferences.shared_with_me"] = []
+    if updates:
+        users_collection.update_one({"user_id": user_id}, {"$set": updates})
+        user_doc = users_collection.find_one({"user_id": user_id})
+    return user_doc
+
+
+def resolve_tool_details(tool_ids):
+    """
+    Resolve tool IDs to their details.
+
+    Args:
+        tool_ids: List of tool IDs
+
+    Returns:
+        List of tool details with id, name, and display_name
+    """
+    tools = user_tools_collection.find(
+        {"_id": {"$in": [ObjectId(tid) for tid in tool_ids]}}
+    )
+    return [
+        {
+            "id": str(tool["_id"]),
+            "name": tool.get("name", ""),
+            "display_name": tool.get("displayName", tool.get("name", "")),
+        }
+        for tool in tools
+    ]
+
+
+def get_vector_store(source_id):
+    """
+    Get the Vector Store for a given source ID.
+
+    Args:
+        source_id (str): source id of the document
+
+    Returns:
+        Vector store instance
+    """
+    store = VectorCreator.create_vectorstore(
+        settings.VECTOR_STORE,
+        source_id=source_id,
+        embeddings_key=os.getenv("EMBEDDINGS_KEY"),
+    )
+    return store
+
+
+def handle_image_upload(
+    request, existing_url: str, user: str, storage, base_path: str = "attachments/"
+) -> Tuple[str, Optional[Response]]:
+    """
+    Handle image file upload from request.
+
+    Args:
+        request: Flask request object
+        existing_url: Existing image URL (fallback)
+        user: User ID
+        storage: Storage instance
+        base_path: Base path for upload
+
+    Returns:
+        Tuple of (image_url, error_response)
+    """
+    image_url = existing_url
+
+    if "image" in request.files:
+        file = request.files["image"]
+        if file.filename != "":
+            filename = secure_filename(file.filename)
+            upload_path = f"{settings.UPLOAD_FOLDER.rstrip('/')}/{user}/{base_path.rstrip('/')}/{uuid.uuid4()}_{filename}"
+            try:
+                storage.save_file(file, upload_path, storage_class="STANDARD")
+                image_url = upload_path
+            except Exception as e:
+                current_app.logger.error(f"Error uploading image: {e}")
+                return None, make_response(
+                    jsonify({"success": False, "message": "Image upload failed"}),
+                    400,
+                )
+    return image_url, None
+
+
+def require_agent(func):
+    """
+    Decorator to require valid agent webhook token.
+
+    Args:
+        func: Function to decorate
+
+    Returns:
+        Wrapped function
+    """
+
+    @wraps(func)
+    def wrapper(*args, **kwargs):
+        webhook_token = kwargs.get("webhook_token")
+        if not webhook_token:
+            return make_response(
+                jsonify({"success": False, "message": "Webhook token missing"}), 400
+            )
+        agent = agents_collection.find_one(
+            {"incoming_webhook_token": webhook_token}, {"_id": 1}
+        )
+        if not agent:
+            current_app.logger.warning(
+                f"Webhook attempt with invalid token: {webhook_token}"
+            )
+            return make_response(
+                jsonify({"success": False, "message": "Agent not found"}), 404
+            )
+        kwargs["agent"] = agent
+        kwargs["agent_id_str"] = str(agent["_id"])
+        return func(*args, **kwargs)
+
+    return wrapper

application/api/user/conversations/__init__.py

@@ -0,0 +1,5 @@
+"""Conversation management module."""
+
+from .routes import conversations_ns
+
+__all__ = ["conversations_ns"]

application/api/user/conversations/routes.py

@@ -0,0 +1,280 @@
+"""Conversation management routes."""
+
+import datetime
+
+from bson.objectid import ObjectId
+from flask import current_app, jsonify, make_response, request
+from flask_restx import fields, Namespace, Resource
+
+from application.api import api
+from application.api.user.base import attachments_collection, conversations_collection
+from application.utils import check_required_fields
+
+conversations_ns = Namespace(
+    "conversations", description="Conversation management operations", path="/api"
+)
+
+
+@conversations_ns.route("/delete_conversation")
+class DeleteConversation(Resource):
+    @api.doc(
+        description="Deletes a conversation by ID",
+        params={"id": "The ID of the conversation to delete"},
+    )
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        conversation_id = request.args.get("id")
+        if not conversation_id:
+            return make_response(
+                jsonify({"success": False, "message": "ID is required"}), 400
+            )
+        try:
+            conversations_collection.delete_one(
+                {"_id": ObjectId(conversation_id), "user": decoded_token["sub"]}
+            )
+        except Exception as err:
+            current_app.logger.error(
+                f"Error deleting conversation: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True}), 200)
+
+
+@conversations_ns.route("/delete_all_conversations")
+class DeleteAllConversations(Resource):
+    @api.doc(
+        description="Deletes all conversations for a specific user",
+    )
+    def get(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user_id = decoded_token.get("sub")
+        try:
+            conversations_collection.delete_many({"user": user_id})
+        except Exception as err:
+            current_app.logger.error(
+                f"Error deleting all conversations: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True}), 200)
+
+
+@conversations_ns.route("/get_conversations")
+class GetConversations(Resource):
+    @api.doc(
+        description="Retrieve a list of the latest 30 conversations (excluding API key conversations)",
+    )
+    def get(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        try:
+            conversations = (
+                conversations_collection.find(
+                    {
+                        "$or": [
+                            {"api_key": {"$exists": False}},
+                            {"agent_id": {"$exists": True}},
+                        ],
+                        "user": decoded_token.get("sub"),
+                    }
+                )
+                .sort("date", -1)
+                .limit(30)
+            )
+
+            list_conversations = [
+                {
+                    "id": str(conversation["_id"]),
+                    "name": conversation["name"],
+                    "agent_id": conversation.get("agent_id", None),
+                    "is_shared_usage": conversation.get("is_shared_usage", False),
+                    "shared_token": conversation.get("shared_token", None),
+                }
+                for conversation in conversations
+            ]
+        except Exception as err:
+            current_app.logger.error(
+                f"Error retrieving conversations: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify(list_conversations), 200)
+
+
+@conversations_ns.route("/get_single_conversation")
+class GetSingleConversation(Resource):
+    @api.doc(
+        description="Retrieve a single conversation by ID",
+        params={"id": "The conversation ID"},
+    )
+    def get(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        conversation_id = request.args.get("id")
+        if not conversation_id:
+            return make_response(
+                jsonify({"success": False, "message": "ID is required"}), 400
+            )
+        try:
+            conversation = conversations_collection.find_one(
+                {"_id": ObjectId(conversation_id), "user": decoded_token.get("sub")}
+            )
+            if not conversation:
+                return make_response(jsonify({"status": "not found"}), 404)
+            # Process queries to include attachment names
+
+            queries = conversation["queries"]
+            for query in queries:
+                if "attachments" in query and query["attachments"]:
+                    attachment_details = []
+                    for attachment_id in query["attachments"]:
+                        try:
+                            attachment = attachments_collection.find_one(
+                                {"_id": ObjectId(attachment_id)}
+                            )
+                            if attachment:
+                                attachment_details.append(
+                                    {
+                                        "id": str(attachment["_id"]),
+                                        "fileName": attachment.get(
+                                            "filename", "Unknown file"
+                                        ),
+                                    }
+                                )
+                        except Exception as e:
+                            current_app.logger.error(
+                                f"Error retrieving attachment {attachment_id}: {e}",
+                                exc_info=True,
+                            )
+                    query["attachments"] = attachment_details
+        except Exception as err:
+            current_app.logger.error(
+                f"Error retrieving conversation: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        data = {
+            "queries": queries,
+            "agent_id": conversation.get("agent_id"),
+            "is_shared_usage": conversation.get("is_shared_usage", False),
+            "shared_token": conversation.get("shared_token", None),
+        }
+        return make_response(jsonify(data), 200)
+
+
+@conversations_ns.route("/update_conversation_name")
+class UpdateConversationName(Resource):
+    @api.expect(
+        api.model(
+            "UpdateConversationModel",
+            {
+                "id": fields.String(required=True, description="Conversation ID"),
+                "name": fields.String(
+                    required=True, description="New name of the conversation"
+                ),
+            },
+        )
+    )
+    @api.doc(
+        description="Updates the name of a conversation",
+    )
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        data = request.get_json()
+        required_fields = ["id", "name"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        try:
+            conversations_collection.update_one(
+                {"_id": ObjectId(data["id"]), "user": decoded_token.get("sub")},
+                {"$set": {"name": data["name"]}},
+            )
+        except Exception as err:
+            current_app.logger.error(
+                f"Error updating conversation name: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True}), 200)
+
+
+@conversations_ns.route("/feedback")
+class SubmitFeedback(Resource):
+    @api.expect(
+        api.model(
+            "FeedbackModel",
+            {
+                "question": fields.String(
+                    required=False, description="The user question"
+                ),
+                "answer": fields.String(required=False, description="The AI answer"),
+                "feedback": fields.String(required=True, description="User feedback"),
+                "question_index": fields.Integer(
+                    required=True,
+                    description="The question number in that particular conversation",
+                ),
+                "conversation_id": fields.String(
+                    required=True, description="id of the particular conversation"
+                ),
+                "api_key": fields.String(description="Optional API key"),
+            },
+        )
+    )
+    @api.doc(
+        description="Submit feedback for a conversation",
+    )
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        data = request.get_json()
+        required_fields = ["feedback", "conversation_id", "question_index"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        try:
+            if data["feedback"] is None:
+                # Remove feedback and feedback_timestamp if feedback is null
+
+                conversations_collection.update_one(
+                    {
+                        "_id": ObjectId(data["conversation_id"]),
+                        "user": decoded_token.get("sub"),
+                        f"queries.{data['question_index']}": {"$exists": True},
+                    },
+                    {
+                        "$unset": {
+                            f"queries.{data['question_index']}.feedback": "",
+                            f"queries.{data['question_index']}.feedback_timestamp": "",
+                        }
+                    },
+                )
+            else:
+                # Set feedback and feedback_timestamp if feedback has a value
+
+                conversations_collection.update_one(
+                    {
+                        "_id": ObjectId(data["conversation_id"]),
+                        "user": decoded_token.get("sub"),
+                        f"queries.{data['question_index']}": {"$exists": True},
+                    },
+                    {
+                        "$set": {
+                            f"queries.{data['question_index']}.feedback": data[
+                                "feedback"
+                            ],
+                            f"queries.{data['question_index']}.feedback_timestamp": datetime.datetime.now(
+                                datetime.timezone.utc
+                            ),
+                        }
+                    },
+                )
+        except Exception as err:
+            current_app.logger.error(f"Error submitting feedback: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True}), 200)

application/api/user/prompts/__init__.py

@@ -0,0 +1,5 @@
+"""Prompts module."""
+
+from .routes import prompts_ns
+
+__all__ = ["prompts_ns"]

application/api/user/prompts/routes.py

@@ -0,0 +1,191 @@
+"""Prompt management routes."""
+
+import os
+
+from bson.objectid import ObjectId
+from flask import current_app, jsonify, make_response, request
+from flask_restx import fields, Namespace, Resource
+
+from application.api import api
+from application.api.user.base import current_dir, prompts_collection
+from application.utils import check_required_fields
+
+prompts_ns = Namespace(
+    "prompts", description="Prompt management operations", path="/api"
+)
+
+
+@prompts_ns.route("/create_prompt")
+class CreatePrompt(Resource):
+    create_prompt_model = api.model(
+        "CreatePromptModel",
+        {
+            "content": fields.String(
+                required=True, description="Content of the prompt"
+            ),
+            "name": fields.String(required=True, description="Name of the prompt"),
+        },
+    )
+
+    @api.expect(create_prompt_model)
+    @api.doc(description="Create a new prompt")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        data = request.get_json()
+        required_fields = ["content", "name"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        user = decoded_token.get("sub")
+        try:
+
+            resp = prompts_collection.insert_one(
+                {
+                    "name": data["name"],
+                    "content": data["content"],
+                    "user": user,
+                }
+            )
+            new_id = str(resp.inserted_id)
+        except Exception as err:
+            current_app.logger.error(f"Error creating prompt: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"id": new_id}), 200)
+
+
+@prompts_ns.route("/get_prompts")
+class GetPrompts(Resource):
+    @api.doc(description="Get all prompts for the user")
+    def get(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        try:
+            prompts = prompts_collection.find({"user": user})
+            list_prompts = [
+                {"id": "default", "name": "default", "type": "public"},
+                {"id": "creative", "name": "creative", "type": "public"},
+                {"id": "strict", "name": "strict", "type": "public"},
+            ]
+
+            for prompt in prompts:
+                list_prompts.append(
+                    {
+                        "id": str(prompt["_id"]),
+                        "name": prompt["name"],
+                        "type": "private",
+                    }
+                )
+        except Exception as err:
+            current_app.logger.error(f"Error retrieving prompts: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify(list_prompts), 200)
+
+
+@prompts_ns.route("/get_single_prompt")
+class GetSinglePrompt(Resource):
+    @api.doc(params={"id": "ID of the prompt"}, description="Get a single prompt by ID")
+    def get(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        prompt_id = request.args.get("id")
+        if not prompt_id:
+            return make_response(
+                jsonify({"success": False, "message": "ID is required"}), 400
+            )
+        try:
+            if prompt_id == "default":
+                with open(
+                    os.path.join(current_dir, "prompts", "chat_combine_default.txt"),
+                    "r",
+                ) as f:
+                    chat_combine_template = f.read()
+                return make_response(jsonify({"content": chat_combine_template}), 200)
+            elif prompt_id == "creative":
+                with open(
+                    os.path.join(current_dir, "prompts", "chat_combine_creative.txt"),
+                    "r",
+                ) as f:
+                    chat_reduce_creative = f.read()
+                return make_response(jsonify({"content": chat_reduce_creative}), 200)
+            elif prompt_id == "strict":
+                with open(
+                    os.path.join(current_dir, "prompts", "chat_combine_strict.txt"), "r"
+                ) as f:
+                    chat_reduce_strict = f.read()
+                return make_response(jsonify({"content": chat_reduce_strict}), 200)
+            prompt = prompts_collection.find_one(
+                {"_id": ObjectId(prompt_id), "user": user}
+            )
+        except Exception as err:
+            current_app.logger.error(f"Error retrieving prompt: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"content": prompt["content"]}), 200)
+
+
+@prompts_ns.route("/delete_prompt")
+class DeletePrompt(Resource):
+    delete_prompt_model = api.model(
+        "DeletePromptModel",
+        {"id": fields.String(required=True, description="Prompt ID to delete")},
+    )
+
+    @api.expect(delete_prompt_model)
+    @api.doc(description="Delete a prompt by ID")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        required_fields = ["id"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        try:
+            prompts_collection.delete_one({"_id": ObjectId(data["id"]), "user": user})
+        except Exception as err:
+            current_app.logger.error(f"Error deleting prompt: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True}), 200)
+
+
+@prompts_ns.route("/update_prompt")
+class UpdatePrompt(Resource):
+    update_prompt_model = api.model(
+        "UpdatePromptModel",
+        {
+            "id": fields.String(required=True, description="Prompt ID to update"),
+            "name": fields.String(required=True, description="New name of the prompt"),
+            "content": fields.String(
+                required=True, description="New content of the prompt"
+            ),
+        },
+    )
+
+    @api.expect(update_prompt_model)
+    @api.doc(description="Update an existing prompt")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        required_fields = ["id", "name", "content"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        try:
+            prompts_collection.update_one(
+                {"_id": ObjectId(data["id"]), "user": user},
+                {"$set": {"name": data["name"], "content": data["content"]}},
+            )
+        except Exception as err:
+            current_app.logger.error(f"Error updating prompt: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True}), 200)

application/api/user/sharing/__init__.py

@@ -0,0 +1,5 @@
+"""Sharing module."""
+
+from .routes import sharing_ns
+
+__all__ = ["sharing_ns"]

application/api/user/sharing/routes.py

@@ -0,0 +1,301 @@
+"""Conversation sharing routes."""
+
+import uuid
+
+from bson.binary import Binary, UuidRepresentation
+from bson.dbref import DBRef
+from bson.objectid import ObjectId
+from flask import current_app, jsonify, make_response, request
+from flask_restx import fields, inputs, Namespace, Resource
+
+from application.api import api
+from application.api.user.base import (
+    agents_collection,
+    attachments_collection,
+    conversations_collection,
+    db,
+    shared_conversations_collections,
+)
+from application.utils import check_required_fields
+
+sharing_ns = Namespace(
+    "sharing", description="Conversation sharing operations", path="/api"
+)
+
+
+@sharing_ns.route("/share")
+class ShareConversation(Resource):
+    share_conversation_model = api.model(
+        "ShareConversationModel",
+        {
+            "conversation_id": fields.String(
+                required=True, description="Conversation ID"
+            ),
+            "user": fields.String(description="User ID (optional)"),
+            "prompt_id": fields.String(description="Prompt ID (optional)"),
+            "chunks": fields.Integer(description="Chunks count (optional)"),
+        },
+    )
+
+    @api.expect(share_conversation_model)
+    @api.doc(description="Share a conversation")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        required_fields = ["conversation_id"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        is_promptable = request.args.get("isPromptable", type=inputs.boolean)
+        if is_promptable is None:
+            return make_response(
+                jsonify({"success": False, "message": "isPromptable is required"}), 400
+            )
+        conversation_id = data["conversation_id"]
+
+        try:
+            conversation = conversations_collection.find_one(
+                {"_id": ObjectId(conversation_id)}
+            )
+            if conversation is None:
+                return make_response(
+                    jsonify(
+                        {
+                            "status": "error",
+                            "message": "Conversation does not exist",
+                        }
+                    ),
+                    404,
+                )
+            current_n_queries = len(conversation["queries"])
+            explicit_binary = Binary.from_uuid(
+                uuid.uuid4(), UuidRepresentation.STANDARD
+            )
+
+            if is_promptable:
+                prompt_id = data.get("prompt_id", "default")
+                chunks = data.get("chunks", "2")
+
+                name = conversation["name"] + "(shared)"
+                new_api_key_data = {
+                    "prompt_id": prompt_id,
+                    "chunks": chunks,
+                    "user": user,
+                }
+
+                if "source" in data and ObjectId.is_valid(data["source"]):
+                    new_api_key_data["source"] = DBRef(
+                        "sources", ObjectId(data["source"])
+                    )
+                if "retriever" in data:
+                    new_api_key_data["retriever"] = data["retriever"]
+                pre_existing_api_document = agents_collection.find_one(new_api_key_data)
+                if pre_existing_api_document:
+                    api_uuid = pre_existing_api_document["key"]
+                    pre_existing = shared_conversations_collections.find_one(
+                        {
+                            "conversation_id": DBRef(
+                                "conversations", ObjectId(conversation_id)
+                            ),
+                            "isPromptable": is_promptable,
+                            "first_n_queries": current_n_queries,
+                            "user": user,
+                            "api_key": api_uuid,
+                        }
+                    )
+                    if pre_existing is not None:
+                        return make_response(
+                            jsonify(
+                                {
+                                    "success": True,
+                                    "identifier": str(pre_existing["uuid"].as_uuid()),
+                                }
+                            ),
+                            200,
+                        )
+                    else:
+                        shared_conversations_collections.insert_one(
+                            {
+                                "uuid": explicit_binary,
+                                "conversation_id": {
+                                    "$ref": "conversations",
+                                    "$id": ObjectId(conversation_id),
+                                },
+                                "isPromptable": is_promptable,
+                                "first_n_queries": current_n_queries,
+                                "user": user,
+                                "api_key": api_uuid,
+                            }
+                        )
+                        return make_response(
+                            jsonify(
+                                {
+                                    "success": True,
+                                    "identifier": str(explicit_binary.as_uuid()),
+                                }
+                            ),
+                            201,
+                        )
+                else:
+                    api_uuid = str(uuid.uuid4())
+                    new_api_key_data["key"] = api_uuid
+                    new_api_key_data["name"] = name
+
+                    if "source" in data and ObjectId.is_valid(data["source"]):
+                        new_api_key_data["source"] = DBRef(
+                            "sources", ObjectId(data["source"])
+                        )
+                    if "retriever" in data:
+                        new_api_key_data["retriever"] = data["retriever"]
+                    agents_collection.insert_one(new_api_key_data)
+                    shared_conversations_collections.insert_one(
+                        {
+                            "uuid": explicit_binary,
+                            "conversation_id": {
+                                "$ref": "conversations",
+                                "$id": ObjectId(conversation_id),
+                            },
+                            "isPromptable": is_promptable,
+                            "first_n_queries": current_n_queries,
+                            "user": user,
+                            "api_key": api_uuid,
+                        }
+                    )
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": True,
+                                "identifier": str(explicit_binary.as_uuid()),
+                            }
+                        ),
+                        201,
+                    )
+            pre_existing = shared_conversations_collections.find_one(
+                {
+                    "conversation_id": DBRef(
+                        "conversations", ObjectId(conversation_id)
+                    ),
+                    "isPromptable": is_promptable,
+                    "first_n_queries": current_n_queries,
+                    "user": user,
+                }
+            )
+            if pre_existing is not None:
+                return make_response(
+                    jsonify(
+                        {
+                            "success": True,
+                            "identifier": str(pre_existing["uuid"].as_uuid()),
+                        }
+                    ),
+                    200,
+                )
+            else:
+                shared_conversations_collections.insert_one(
+                    {
+                        "uuid": explicit_binary,
+                        "conversation_id": {
+                            "$ref": "conversations",
+                            "$id": ObjectId(conversation_id),
+                        },
+                        "isPromptable": is_promptable,
+                        "first_n_queries": current_n_queries,
+                        "user": user,
+                    }
+                )
+                return make_response(
+                    jsonify(
+                        {"success": True, "identifier": str(explicit_binary.as_uuid())}
+                    ),
+                    201,
+                )
+        except Exception as err:
+            current_app.logger.error(
+                f"Error sharing conversation: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+
+
+@sharing_ns.route("/shared_conversation/<string:identifier>")
+class GetPubliclySharedConversations(Resource):
+    @api.doc(description="Get publicly shared conversations by identifier")
+    def get(self, identifier: str):
+        try:
+            query_uuid = Binary.from_uuid(
+                uuid.UUID(identifier), UuidRepresentation.STANDARD
+            )
+            shared = shared_conversations_collections.find_one({"uuid": query_uuid})
+            conversation_queries = []
+
+            if (
+                shared
+                and "conversation_id" in shared
+                and isinstance(shared["conversation_id"], DBRef)
+            ):
+                conversation_ref = shared["conversation_id"]
+                conversation = db.dereference(conversation_ref)
+                if conversation is None:
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": False,
+                                "error": "might have broken url or the conversation does not exist",
+                            }
+                        ),
+                        404,
+                    )
+                conversation_queries = conversation["queries"][
+                    : (shared["first_n_queries"])
+                ]
+
+                for query in conversation_queries:
+                    if "attachments" in query and query["attachments"]:
+                        attachment_details = []
+                        for attachment_id in query["attachments"]:
+                            try:
+                                attachment = attachments_collection.find_one(
+                                    {"_id": ObjectId(attachment_id)}
+                                )
+                                if attachment:
+                                    attachment_details.append(
+                                        {
+                                            "id": str(attachment["_id"]),
+                                            "fileName": attachment.get(
+                                                "filename", "Unknown file"
+                                            ),
+                                        }
+                                    )
+                            except Exception as e:
+                                current_app.logger.error(
+                                    f"Error retrieving attachment {attachment_id}: {e}",
+                                    exc_info=True,
+                                )
+                        query["attachments"] = attachment_details
+            else:
+                return make_response(
+                    jsonify(
+                        {
+                            "success": False,
+                            "error": "might have broken url or the conversation does not exist",
+                        }
+                    ),
+                    404,
+                )
+            date = conversation["_id"].generation_time.isoformat()
+            res = {
+                "success": True,
+                "queries": conversation_queries,
+                "title": conversation["name"],
+                "timestamp": date,
+            }
+            if shared["isPromptable"] and "api_key" in shared:
+                res["api_key"] = shared["api_key"]
+            return make_response(jsonify(res), 200)
+        except Exception as err:
+            current_app.logger.error(
+                f"Error getting shared conversation: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)

application/api/user/sources/__init__.py

@@ -0,0 +1,7 @@
+"""Sources module."""
+
+from .chunks import sources_chunks_ns
+from .routes import sources_ns
+from .upload import sources_upload_ns
+
+__all__ = ["sources_ns", "sources_chunks_ns", "sources_upload_ns"]

application/api/user/sources/chunks.py

@@ -0,0 +1,278 @@
+"""Source document management chunk management."""
+
+from bson.objectid import ObjectId
+from flask import current_app, jsonify, make_response, request
+from flask_restx import fields, Namespace, Resource
+
+from application.api import api
+from application.api.user.base import get_vector_store, sources_collection
+from application.utils import check_required_fields, num_tokens_from_string
+
+sources_chunks_ns = Namespace(
+    "sources", description="Source document management operations", path="/api"
+)
+
+
+@sources_chunks_ns.route("/get_chunks")
+class GetChunks(Resource):
+    @api.doc(
+        description="Retrieves chunks from a document, optionally filtered by file path and search term",
+        params={
+            "id": "The document ID",
+            "page": "Page number for pagination",
+            "per_page": "Number of chunks per page",
+            "path": "Optional: Filter chunks by relative file path",
+            "search": "Optional: Search term to filter chunks by title or content",
+        },
+    )
+    def get(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        doc_id = request.args.get("id")
+        page = int(request.args.get("page", 1))
+        per_page = int(request.args.get("per_page", 10))
+        path = request.args.get("path")
+        search_term = request.args.get("search", "").strip().lower()
+
+        if not ObjectId.is_valid(doc_id):
+            return make_response(jsonify({"error": "Invalid doc_id"}), 400)
+        doc = sources_collection.find_one({"_id": ObjectId(doc_id), "user": user})
+        if not doc:
+            return make_response(
+                jsonify({"error": "Document not found or access denied"}), 404
+            )
+        try:
+            store = get_vector_store(doc_id)
+            chunks = store.get_chunks()
+
+            filtered_chunks = []
+            for chunk in chunks:
+                metadata = chunk.get("metadata", {})
+
+                # Filter by path if provided
+
+                if path:
+                    chunk_source = metadata.get("source", "")
+                    # Check if the chunk's source matches the requested path
+
+                    if not chunk_source or not chunk_source.endswith(path):
+                        continue
+                # Filter by search term if provided
+
+                if search_term:
+                    text_match = search_term in chunk.get("text", "").lower()
+                    title_match = search_term in metadata.get("title", "").lower()
+
+                    if not (text_match or title_match):
+                        continue
+                filtered_chunks.append(chunk)
+            chunks = filtered_chunks
+
+            total_chunks = len(chunks)
+            start = (page - 1) * per_page
+            end = start + per_page
+            paginated_chunks = chunks[start:end]
+
+            return make_response(
+                jsonify(
+                    {
+                        "page": page,
+                        "per_page": per_page,
+                        "total": total_chunks,
+                        "chunks": paginated_chunks,
+                        "path": path if path else None,
+                        "search": search_term if search_term else None,
+                    }
+                ),
+                200,
+            )
+        except Exception as e:
+            current_app.logger.error(f"Error getting chunks: {e}", exc_info=True)
+            return make_response(jsonify({"success": False}), 500)
+
+
+@sources_chunks_ns.route("/add_chunk")
+class AddChunk(Resource):
+    @api.expect(
+        api.model(
+            "AddChunkModel",
+            {
+                "id": fields.String(required=True, description="Document ID"),
+                "text": fields.String(required=True, description="Text of the chunk"),
+                "metadata": fields.Raw(
+                    required=False,
+                    description="Metadata associated with the chunk",
+                ),
+            },
+        )
+    )
+    @api.doc(
+        description="Adds a new chunk to the document",
+    )
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        required_fields = ["id", "text"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        doc_id = data.get("id")
+        text = data.get("text")
+        metadata = data.get("metadata", {})
+        token_count = num_tokens_from_string(text)
+        metadata["token_count"] = token_count
+
+        if not ObjectId.is_valid(doc_id):
+            return make_response(jsonify({"error": "Invalid doc_id"}), 400)
+        doc = sources_collection.find_one({"_id": ObjectId(doc_id), "user": user})
+        if not doc:
+            return make_response(
+                jsonify({"error": "Document not found or access denied"}), 404
+            )
+        try:
+            store = get_vector_store(doc_id)
+            chunk_id = store.add_chunk(text, metadata)
+            return make_response(
+                jsonify({"message": "Chunk added successfully", "chunk_id": chunk_id}),
+                201,
+            )
+        except Exception as e:
+            current_app.logger.error(f"Error adding chunk: {e}", exc_info=True)
+            return make_response(jsonify({"success": False}), 500)
+
+
+@sources_chunks_ns.route("/delete_chunk")
+class DeleteChunk(Resource):
+    @api.doc(
+        description="Deletes a specific chunk from the document.",
+        params={"id": "The document ID", "chunk_id": "The ID of the chunk to delete"},
+    )
+    def delete(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        doc_id = request.args.get("id")
+        chunk_id = request.args.get("chunk_id")
+
+        if not ObjectId.is_valid(doc_id):
+            return make_response(jsonify({"error": "Invalid doc_id"}), 400)
+        doc = sources_collection.find_one({"_id": ObjectId(doc_id), "user": user})
+        if not doc:
+            return make_response(
+                jsonify({"error": "Document not found or access denied"}), 404
+            )
+        try:
+            store = get_vector_store(doc_id)
+            deleted = store.delete_chunk(chunk_id)
+            if deleted:
+                return make_response(
+                    jsonify({"message": "Chunk deleted successfully"}), 200
+                )
+            else:
+                return make_response(
+                    jsonify({"message": "Chunk not found or could not be deleted"}),
+                    404,
+                )
+        except Exception as e:
+            current_app.logger.error(f"Error deleting chunk: {e}", exc_info=True)
+            return make_response(jsonify({"success": False}), 500)
+
+
+@sources_chunks_ns.route("/update_chunk")
+class UpdateChunk(Resource):
+    @api.expect(
+        api.model(
+            "UpdateChunkModel",
+            {
+                "id": fields.String(required=True, description="Document ID"),
+                "chunk_id": fields.String(
+                    required=True, description="Chunk ID to update"
+                ),
+                "text": fields.String(
+                    required=False, description="New text of the chunk"
+                ),
+                "metadata": fields.Raw(
+                    required=False,
+                    description="Updated metadata associated with the chunk",
+                ),
+            },
+        )
+    )
+    @api.doc(
+        description="Updates an existing chunk in the document.",
+    )
+    def put(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        required_fields = ["id", "chunk_id"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        doc_id = data.get("id")
+        chunk_id = data.get("chunk_id")
+        text = data.get("text")
+        metadata = data.get("metadata")
+
+        if text is not None:
+            token_count = num_tokens_from_string(text)
+            if metadata is None:
+                metadata = {}
+            metadata["token_count"] = token_count
+        if not ObjectId.is_valid(doc_id):
+            return make_response(jsonify({"error": "Invalid doc_id"}), 400)
+        doc = sources_collection.find_one({"_id": ObjectId(doc_id), "user": user})
+        if not doc:
+            return make_response(
+                jsonify({"error": "Document not found or access denied"}), 404
+            )
+        try:
+            store = get_vector_store(doc_id)
+
+            chunks = store.get_chunks()
+            existing_chunk = next((c for c in chunks if c["doc_id"] == chunk_id), None)
+            if not existing_chunk:
+                return make_response(jsonify({"error": "Chunk not found"}), 404)
+            new_text = text if text is not None else existing_chunk["text"]
+
+            if metadata is not None:
+                new_metadata = existing_chunk["metadata"].copy()
+                new_metadata.update(metadata)
+            else:
+                new_metadata = existing_chunk["metadata"].copy()
+            if text is not None:
+                new_metadata["token_count"] = num_tokens_from_string(new_text)
+            try:
+                new_chunk_id = store.add_chunk(new_text, new_metadata)
+
+                deleted = store.delete_chunk(chunk_id)
+                if not deleted:
+                    current_app.logger.warning(
+                        f"Failed to delete old chunk {chunk_id}, but new chunk {new_chunk_id} was created"
+                    )
+                return make_response(
+                    jsonify(
+                        {
+                            "message": "Chunk updated successfully",
+                            "chunk_id": new_chunk_id,
+                            "original_chunk_id": chunk_id,
+                        }
+                    ),
+                    200,
+                )
+            except Exception as add_error:
+                current_app.logger.error(f"Failed to add updated chunk: {add_error}")
+                return make_response(
+                    jsonify({"error": "Failed to update chunk - addition failed"}), 500
+                )
+        except Exception as e:
+            current_app.logger.error(f"Error updating chunk: {e}", exc_info=True)
+            return make_response(jsonify({"success": False}), 500)

application/api/user/sources/routes.py

@@ -0,0 +1,350 @@
+"""Source document management routes."""
+
+import json
+import math
+import os
+
+from bson.objectid import ObjectId
+from flask import current_app, jsonify, make_response, redirect, request
+from flask_restx import fields, Namespace, Resource
+from werkzeug.utils import secure_filename
+
+from application.api import api
+from application.api.user.base import sources_collection
+from application.core.settings import settings
+from application.storage.storage_creator import StorageCreator
+from application.utils import check_required_fields
+from application.vectorstore.vector_creator import VectorCreator
+
+
+sources_ns = Namespace(
+    "sources", description="Source document management operations", path="/api"
+)
+
+
+@sources_ns.route("/sources")
+class CombinedJson(Resource):
+    @api.doc(description="Provide JSON file with combined available indexes")
+    def get(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = [
+            {
+                "name": "Default",
+                "date": "default",
+                "model": settings.EMBEDDINGS_NAME,
+                "location": "remote",
+                "tokens": "",
+                "retriever": "classic",
+            }
+        ]
+
+        try:
+            for index in sources_collection.find({"user": user}).sort("date", -1):
+                data.append(
+                    {
+                        "id": str(index["_id"]),
+                        "name": index.get("name"),
+                        "date": index.get("date"),
+                        "model": settings.EMBEDDINGS_NAME,
+                        "location": "local",
+                        "tokens": index.get("tokens", ""),
+                        "retriever": index.get("retriever", "classic"),
+                        "syncFrequency": index.get("sync_frequency", ""),
+                        "is_nested": bool(index.get("directory_structure")),
+                        "type": index.get(
+                            "type", "file"
+                        ),  # Add type field with default "file"
+                    }
+                )
+        except Exception as err:
+            current_app.logger.error(f"Error retrieving sources: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify(data), 200)
+
+
+@sources_ns.route("/sources/paginated")
+class PaginatedSources(Resource):
+    @api.doc(description="Get document with pagination, sorting and filtering")
+    def get(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        sort_field = request.args.get("sort", "date")  # Default to 'date'
+        sort_order = request.args.get("order", "desc")  # Default to 'desc'
+        page = int(request.args.get("page", 1))  # Default to 1
+        rows_per_page = int(request.args.get("rows", 10))  # Default to 10
+        # add .strip() to remove leading and trailing whitespaces
+
+        search_term = request.args.get(
+            "search", ""
+        ).strip()  # add search for filter documents
+
+        # Prepare query for filtering
+
+        query = {"user": user}
+        if search_term:
+            query["name"] = {
+                "$regex": search_term,
+                "$options": "i",  # using case-insensitive search
+            }
+        total_documents = sources_collection.count_documents(query)
+        total_pages = max(1, math.ceil(total_documents / rows_per_page))
+        page = min(
+            max(1, page), total_pages
+        )  # add this to make sure page inbound is within the range
+        sort_order = 1 if sort_order == "asc" else -1
+        skip = (page - 1) * rows_per_page
+
+        try:
+            documents = (
+                sources_collection.find(query)
+                .sort(sort_field, sort_order)
+                .skip(skip)
+                .limit(rows_per_page)
+            )
+
+            paginated_docs = []
+            for doc in documents:
+                doc_data = {
+                    "id": str(doc["_id"]),
+                    "name": doc.get("name", ""),
+                    "date": doc.get("date", ""),
+                    "model": settings.EMBEDDINGS_NAME,
+                    "location": "local",
+                    "tokens": doc.get("tokens", ""),
+                    "retriever": doc.get("retriever", "classic"),
+                    "syncFrequency": doc.get("sync_frequency", ""),
+                    "isNested": bool(doc.get("directory_structure")),
+                    "type": doc.get("type", "file"),
+                }
+                paginated_docs.append(doc_data)
+            response = {
+                "total": total_documents,
+                "totalPages": total_pages,
+                "currentPage": page,
+                "paginated": paginated_docs,
+            }
+            return make_response(jsonify(response), 200)
+        except Exception as err:
+            current_app.logger.error(
+                f"Error retrieving paginated sources: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+
+
+@sources_ns.route("/docs_check")
+class CheckDocs(Resource):
+    check_docs_model = api.model(
+        "CheckDocsModel",
+        {"docs": fields.String(required=True, description="Document name")},
+    )
+
+    @api.expect(check_docs_model)
+    @api.doc(description="Check if document exists")
+    def post(self):
+        data = request.get_json()
+        required_fields = ["docs"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        try:
+            vectorstore = "vectors/" + secure_filename(data["docs"])
+            if os.path.exists(vectorstore) or data["docs"] == "default":
+                return {"status": "exists"}, 200
+        except Exception as err:
+            current_app.logger.error(f"Error checking document: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"status": "not found"}), 404)
+
+
+@sources_ns.route("/delete_by_ids")
+class DeleteByIds(Resource):
+    @api.doc(
+        description="Deletes documents from the vector store by IDs",
+        params={"path": "Comma-separated list of IDs"},
+    )
+    def get(self):
+        ids = request.args.get("path")
+        if not ids:
+            return make_response(
+                jsonify({"success": False, "message": "Missing required fields"}), 400
+            )
+        try:
+            result = sources_collection.delete_index(ids=ids)
+            if result:
+                return make_response(jsonify({"success": True}), 200)
+        except Exception as err:
+            current_app.logger.error(f"Error deleting indexes: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": False}), 400)
+
+
+@sources_ns.route("/delete_old")
+class DeleteOldIndexes(Resource):
+    @api.doc(
+        description="Deletes old indexes and associated files",
+        params={"source_id": "The source ID to delete"},
+    )
+    def get(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        source_id = request.args.get("source_id")
+        if not source_id:
+            return make_response(
+                jsonify({"success": False, "message": "Missing required fields"}), 400
+            )
+        doc = sources_collection.find_one(
+            {"_id": ObjectId(source_id), "user": decoded_token.get("sub")}
+        )
+        if not doc:
+            return make_response(jsonify({"status": "not found"}), 404)
+        storage = StorageCreator.get_storage()
+
+        try:
+            # Delete vector index
+
+            if settings.VECTOR_STORE == "faiss":
+                index_path = f"indexes/{str(doc['_id'])}"
+                if storage.file_exists(f"{index_path}/index.faiss"):
+                    storage.delete_file(f"{index_path}/index.faiss")
+                if storage.file_exists(f"{index_path}/index.pkl"):
+                    storage.delete_file(f"{index_path}/index.pkl")
+            else:
+                vectorstore = VectorCreator.create_vectorstore(
+                    settings.VECTOR_STORE, source_id=str(doc["_id"])
+                )
+                vectorstore.delete_index()
+            if "file_path" in doc and doc["file_path"]:
+                file_path = doc["file_path"]
+                if storage.is_directory(file_path):
+                    files = storage.list_files(file_path)
+                    for f in files:
+                        storage.delete_file(f)
+                else:
+                    storage.delete_file(file_path)
+        except FileNotFoundError:
+            pass
+        except Exception as err:
+            current_app.logger.error(
+                f"Error deleting files and indexes: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        sources_collection.delete_one({"_id": ObjectId(source_id)})
+        return make_response(jsonify({"success": True}), 200)
+
+
+@sources_ns.route("/combine")
+class RedirectToSources(Resource):
+    @api.doc(
+        description="Redirects /api/combine to /api/sources for backward compatibility"
+    )
+    def get(self):
+        return redirect("/api/sources", code=301)
+
+
+@sources_ns.route("/manage_sync")
+class ManageSync(Resource):
+    manage_sync_model = api.model(
+        "ManageSyncModel",
+        {
+            "source_id": fields.String(required=True, description="Source ID"),
+            "sync_frequency": fields.String(
+                required=True,
+                description="Sync frequency (never, daily, weekly, monthly)",
+            ),
+        },
+    )
+
+    @api.expect(manage_sync_model)
+    @api.doc(description="Manage sync frequency for sources")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        required_fields = ["source_id", "sync_frequency"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        source_id = data["source_id"]
+        sync_frequency = data["sync_frequency"]
+
+        if sync_frequency not in ["never", "daily", "weekly", "monthly"]:
+            return make_response(
+                jsonify({"success": False, "message": "Invalid frequency"}), 400
+            )
+        update_data = {"$set": {"sync_frequency": sync_frequency}}
+        try:
+            sources_collection.update_one(
+                {
+                    "_id": ObjectId(source_id),
+                    "user": user,
+                },
+                update_data,
+            )
+        except Exception as err:
+            current_app.logger.error(
+                f"Error updating sync frequency: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True}), 200)
+
+
+@sources_ns.route("/directory_structure")
+class DirectoryStructure(Resource):
+    @api.doc(
+        description="Get the directory structure for a document",
+        params={"id": "The document ID"},
+    )
+    def get(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        doc_id = request.args.get("id")
+
+        if not doc_id:
+            return make_response(jsonify({"error": "Document ID is required"}), 400)
+        if not ObjectId.is_valid(doc_id):
+            return make_response(jsonify({"error": "Invalid document ID"}), 400)
+        try:
+            doc = sources_collection.find_one({"_id": ObjectId(doc_id), "user": user})
+            if not doc:
+                return make_response(
+                    jsonify({"error": "Document not found or access denied"}), 404
+                )
+            directory_structure = doc.get("directory_structure", {})
+            base_path = doc.get("file_path", "")
+
+            provider = None
+            remote_data = doc.get("remote_data")
+            try:
+                if isinstance(remote_data, str) and remote_data:
+                    remote_data_obj = json.loads(remote_data)
+                    provider = remote_data_obj.get("provider")
+            except Exception as e:
+                current_app.logger.warning(
+                    f"Failed to parse remote_data for doc {doc_id}: {e}"
+                )
+            return make_response(
+                jsonify(
+                    {
+                        "success": True,
+                        "directory_structure": directory_structure,
+                        "base_path": base_path,
+                        "provider": provider,
+                    }
+                ),
+                200,
+            )
+        except Exception as e:
+            current_app.logger.error(
+                f"Error retrieving directory structure: {e}", exc_info=True
+            )
+            return make_response(jsonify({"success": False, "error": str(e)}), 500)

application/api/user/sources/upload.py

@@ -0,0 +1,572 @@
+"""Source document management upload functionality."""
+
+import json
+import os
+import tempfile
+import zipfile
+
+from bson.objectid import ObjectId
+from flask import current_app, jsonify, make_response, request
+from flask_restx import fields, Namespace, Resource
+
+from application.api import api
+from application.api.user.base import sources_collection
+from application.api.user.tasks import ingest, ingest_connector_task, ingest_remote
+from application.core.settings import settings
+from application.parser.connectors.connector_creator import ConnectorCreator
+from application.storage.storage_creator import StorageCreator
+from application.utils import check_required_fields, safe_filename
+
+
+sources_upload_ns = Namespace(
+    "sources", description="Source document management operations", path="/api"
+)
+
+
+@sources_upload_ns.route("/upload")
+class UploadFile(Resource):
+    @api.expect(
+        api.model(
+            "UploadModel",
+            {
+                "user": fields.String(required=True, description="User ID"),
+                "name": fields.String(required=True, description="Job name"),
+                "file": fields.Raw(required=True, description="File(s) to upload"),
+            },
+        )
+    )
+    @api.doc(
+        description="Uploads a file to be vectorized and indexed",
+    )
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        data = request.form
+        files = request.files.getlist("file")
+        required_fields = ["user", "name"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields or not files or all(file.filename == "" for file in files):
+            return make_response(
+                jsonify(
+                    {
+                        "status": "error",
+                        "message": "Missing required fields or files",
+                    }
+                ),
+                400,
+            )
+        user = decoded_token.get("sub")
+        job_name = request.form["name"]
+
+        # Create safe versions for filesystem operations
+
+        safe_user = safe_filename(user)
+        dir_name = safe_filename(job_name)
+        base_path = f"{settings.UPLOAD_FOLDER}/{safe_user}/{dir_name}"
+
+        try:
+            storage = StorageCreator.get_storage()
+
+            for file in files:
+                original_filename = file.filename
+                safe_file = safe_filename(original_filename)
+
+                with tempfile.TemporaryDirectory() as temp_dir:
+                    temp_file_path = os.path.join(temp_dir, safe_file)
+                    file.save(temp_file_path)
+
+                    if zipfile.is_zipfile(temp_file_path):
+                        try:
+                            with zipfile.ZipFile(temp_file_path, "r") as zip_ref:
+                                zip_ref.extractall(path=temp_dir)
+
+                                # Walk through extracted files and upload them
+
+                                for root, _, files in os.walk(temp_dir):
+                                    for extracted_file in files:
+                                        if (
+                                            os.path.join(root, extracted_file)
+                                            == temp_file_path
+                                        ):
+                                            continue
+                                        rel_path = os.path.relpath(
+                                            os.path.join(root, extracted_file), temp_dir
+                                        )
+                                        storage_path = f"{base_path}/{rel_path}"
+
+                                        with open(
+                                            os.path.join(root, extracted_file), "rb"
+                                        ) as f:
+                                            storage.save_file(f, storage_path)
+                        except Exception as e:
+                            current_app.logger.error(
+                                f"Error extracting zip: {e}", exc_info=True
+                            )
+                            # If zip extraction fails, save the original zip file
+
+                            file_path = f"{base_path}/{safe_file}"
+                            with open(temp_file_path, "rb") as f:
+                                storage.save_file(f, file_path)
+                    else:
+                        # For non-zip files, save directly
+
+                        file_path = f"{base_path}/{safe_file}"
+                        with open(temp_file_path, "rb") as f:
+                            storage.save_file(f, file_path)
+            task = ingest.delay(
+                settings.UPLOAD_FOLDER,
+                [
+                    ".rst",
+                    ".md",
+                    ".pdf",
+                    ".txt",
+                    ".docx",
+                    ".csv",
+                    ".epub",
+                    ".html",
+                    ".mdx",
+                    ".json",
+                    ".xlsx",
+                    ".pptx",
+                    ".png",
+                    ".jpg",
+                    ".jpeg",
+                ],
+                job_name,
+                user,
+                file_path=base_path,
+                filename=dir_name,
+            )
+        except Exception as err:
+            current_app.logger.error(f"Error uploading file: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True, "task_id": task.id}), 200)
+
+
+@sources_upload_ns.route("/remote")
+class UploadRemote(Resource):
+    @api.expect(
+        api.model(
+            "RemoteUploadModel",
+            {
+                "user": fields.String(required=True, description="User ID"),
+                "source": fields.String(
+                    required=True, description="Source of the data"
+                ),
+                "name": fields.String(required=True, description="Job name"),
+                "data": fields.String(required=True, description="Data to process"),
+                "repo_url": fields.String(description="GitHub repository URL"),
+            },
+        )
+    )
+    @api.doc(
+        description="Uploads remote source for vectorization",
+    )
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        data = request.form
+        required_fields = ["user", "source", "name", "data"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        try:
+            config = json.loads(data["data"])
+            source_data = None
+
+            if data["source"] == "github":
+                source_data = config.get("repo_url")
+            elif data["source"] in ["crawler", "url"]:
+                source_data = config.get("url")
+            elif data["source"] == "reddit":
+                source_data = config
+            elif data["source"] in ConnectorCreator.get_supported_connectors():
+                session_token = config.get("session_token")
+                if not session_token:
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": False,
+                                "error": f"Missing session_token in {data['source']} configuration",
+                            }
+                        ),
+                        400,
+                    )
+                # Process file_ids
+
+                file_ids = config.get("file_ids", [])
+                if isinstance(file_ids, str):
+                    file_ids = [id.strip() for id in file_ids.split(",") if id.strip()]
+                elif not isinstance(file_ids, list):
+                    file_ids = []
+                # Process folder_ids
+
+                folder_ids = config.get("folder_ids", [])
+                if isinstance(folder_ids, str):
+                    folder_ids = [
+                        id.strip() for id in folder_ids.split(",") if id.strip()
+                    ]
+                elif not isinstance(folder_ids, list):
+                    folder_ids = []
+                config["file_ids"] = file_ids
+                config["folder_ids"] = folder_ids
+
+                task = ingest_connector_task.delay(
+                    job_name=data["name"],
+                    user=decoded_token.get("sub"),
+                    source_type=data["source"],
+                    session_token=session_token,
+                    file_ids=file_ids,
+                    folder_ids=folder_ids,
+                    recursive=config.get("recursive", False),
+                    retriever=config.get("retriever", "classic"),
+                )
+                return make_response(
+                    jsonify({"success": True, "task_id": task.id}), 200
+                )
+            task = ingest_remote.delay(
+                source_data=source_data,
+                job_name=data["name"],
+                user=decoded_token.get("sub"),
+                loader=data["source"],
+            )
+        except Exception as err:
+            current_app.logger.error(
+                f"Error uploading remote source: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True, "task_id": task.id}), 200)
+
+
+@sources_upload_ns.route("/manage_source_files")
+class ManageSourceFiles(Resource):
+    @api.expect(
+        api.model(
+            "ManageSourceFilesModel",
+            {
+                "source_id": fields.String(
+                    required=True, description="Source ID to modify"
+                ),
+                "operation": fields.String(
+                    required=True,
+                    description="Operation: 'add', 'remove', or 'remove_directory'",
+                ),
+                "file_paths": fields.List(
+                    fields.String,
+                    required=False,
+                    description="File paths to remove (for remove operation)",
+                ),
+                "directory_path": fields.String(
+                    required=False,
+                    description="Directory path to remove (for remove_directory operation)",
+                ),
+                "file": fields.Raw(
+                    required=False, description="Files to add (for add operation)"
+                ),
+                "parent_dir": fields.String(
+                    required=False,
+                    description="Parent directory path relative to source root",
+                ),
+            },
+        )
+    )
+    @api.doc(
+        description="Add files, remove files, or remove directories from an existing source",
+    )
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(
+                jsonify({"success": False, "message": "Unauthorized"}), 401
+            )
+        user = decoded_token.get("sub")
+        source_id = request.form.get("source_id")
+        operation = request.form.get("operation")
+
+        if not source_id or not operation:
+            return make_response(
+                jsonify(
+                    {
+                        "success": False,
+                        "message": "source_id and operation are required",
+                    }
+                ),
+                400,
+            )
+        if operation not in ["add", "remove", "remove_directory"]:
+            return make_response(
+                jsonify(
+                    {
+                        "success": False,
+                        "message": "operation must be 'add', 'remove', or 'remove_directory'",
+                    }
+                ),
+                400,
+            )
+        try:
+            ObjectId(source_id)
+        except Exception:
+            return make_response(
+                jsonify({"success": False, "message": "Invalid source ID format"}), 400
+            )
+        try:
+            source = sources_collection.find_one(
+                {"_id": ObjectId(source_id), "user": user}
+            )
+            if not source:
+                return make_response(
+                    jsonify(
+                        {
+                            "success": False,
+                            "message": "Source not found or access denied",
+                        }
+                    ),
+                    404,
+                )
+        except Exception as err:
+            current_app.logger.error(f"Error finding source: {err}", exc_info=True)
+            return make_response(
+                jsonify({"success": False, "message": "Database error"}), 500
+            )
+        try:
+            storage = StorageCreator.get_storage()
+            source_file_path = source.get("file_path", "")
+            parent_dir = request.form.get("parent_dir", "")
+
+            if parent_dir and (parent_dir.startswith("/") or ".." in parent_dir):
+                return make_response(
+                    jsonify(
+                        {"success": False, "message": "Invalid parent directory path"}
+                    ),
+                    400,
+                )
+            if operation == "add":
+                files = request.files.getlist("file")
+                if not files or all(file.filename == "" for file in files):
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": False,
+                                "message": "No files provided for add operation",
+                            }
+                        ),
+                        400,
+                    )
+                added_files = []
+
+                target_dir = source_file_path
+                if parent_dir:
+                    target_dir = f"{source_file_path}/{parent_dir}"
+                for file in files:
+                    if file.filename:
+                        safe_filename_str = safe_filename(file.filename)
+                        file_path = f"{target_dir}/{safe_filename_str}"
+
+                        # Save file to storage
+
+                        storage.save_file(file, file_path)
+                        added_files.append(safe_filename_str)
+                # Trigger re-ingestion pipeline
+
+                from application.api.user.tasks import reingest_source_task
+
+                task = reingest_source_task.delay(source_id=source_id, user=user)
+
+                return make_response(
+                    jsonify(
+                        {
+                            "success": True,
+                            "message": f"Added {len(added_files)} files",
+                            "added_files": added_files,
+                            "parent_dir": parent_dir,
+                            "reingest_task_id": task.id,
+                        }
+                    ),
+                    200,
+                )
+            elif operation == "remove":
+                file_paths_str = request.form.get("file_paths")
+                if not file_paths_str:
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": False,
+                                "message": "file_paths required for remove operation",
+                            }
+                        ),
+                        400,
+                    )
+                try:
+                    file_paths = (
+                        json.loads(file_paths_str)
+                        if isinstance(file_paths_str, str)
+                        else file_paths_str
+                    )
+                except Exception:
+                    return make_response(
+                        jsonify(
+                            {"success": False, "message": "Invalid file_paths format"}
+                        ),
+                        400,
+                    )
+                # Remove files from storage and directory structure
+
+                removed_files = []
+                for file_path in file_paths:
+                    full_path = f"{source_file_path}/{file_path}"
+
+                    # Remove from storage
+
+                    if storage.file_exists(full_path):
+                        storage.delete_file(full_path)
+                        removed_files.append(file_path)
+                # Trigger re-ingestion pipeline
+
+                from application.api.user.tasks import reingest_source_task
+
+                task = reingest_source_task.delay(source_id=source_id, user=user)
+
+                return make_response(
+                    jsonify(
+                        {
+                            "success": True,
+                            "message": f"Removed {len(removed_files)} files",
+                            "removed_files": removed_files,
+                            "reingest_task_id": task.id,
+                        }
+                    ),
+                    200,
+                )
+            elif operation == "remove_directory":
+                directory_path = request.form.get("directory_path")
+                if not directory_path:
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": False,
+                                "message": "directory_path required for remove_directory operation",
+                            }
+                        ),
+                        400,
+                    )
+                # Validate directory path (prevent path traversal)
+
+                if directory_path.startswith("/") or ".." in directory_path:
+                    current_app.logger.warning(
+                        f"Invalid directory path attempted for removal. "
+                        f"User: {user}, Source ID: {source_id}, Directory path: {directory_path}"
+                    )
+                    return make_response(
+                        jsonify(
+                            {"success": False, "message": "Invalid directory path"}
+                        ),
+                        400,
+                    )
+                full_directory_path = (
+                    f"{source_file_path}/{directory_path}"
+                    if directory_path
+                    else source_file_path
+                )
+
+                if not storage.is_directory(full_directory_path):
+                    current_app.logger.warning(
+                        f"Directory not found or is not a directory for removal. "
+                        f"User: {user}, Source ID: {source_id}, Directory path: {directory_path}, "
+                        f"Full path: {full_directory_path}"
+                    )
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": False,
+                                "message": "Directory not found or is not a directory",
+                            }
+                        ),
+                        404,
+                    )
+                success = storage.remove_directory(full_directory_path)
+
+                if not success:
+                    current_app.logger.error(
+                        f"Failed to remove directory from storage. "
+                        f"User: {user}, Source ID: {source_id}, Directory path: {directory_path}, "
+                        f"Full path: {full_directory_path}"
+                    )
+                    return make_response(
+                        jsonify(
+                            {"success": False, "message": "Failed to remove directory"}
+                        ),
+                        500,
+                    )
+                current_app.logger.info(
+                    f"Successfully removed directory. "
+                    f"User: {user}, Source ID: {source_id}, Directory path: {directory_path}, "
+                    f"Full path: {full_directory_path}"
+                )
+
+                # Trigger re-ingestion pipeline
+
+                from application.api.user.tasks import reingest_source_task
+
+                task = reingest_source_task.delay(source_id=source_id, user=user)
+
+                return make_response(
+                    jsonify(
+                        {
+                            "success": True,
+                            "message": f"Successfully removed directory: {directory_path}",
+                            "removed_directory": directory_path,
+                            "reingest_task_id": task.id,
+                        }
+                    ),
+                    200,
+                )
+        except Exception as err:
+            error_context = f"operation={operation}, user={user}, source_id={source_id}"
+            if operation == "remove_directory":
+                directory_path = request.form.get("directory_path", "")
+                error_context += f", directory_path={directory_path}"
+            elif operation == "remove":
+                file_paths_str = request.form.get("file_paths", "")
+                error_context += f", file_paths={file_paths_str}"
+            elif operation == "add":
+                parent_dir = request.form.get("parent_dir", "")
+                error_context += f", parent_dir={parent_dir}"
+            current_app.logger.error(
+                f"Error managing source files: {err} ({error_context})", exc_info=True
+            )
+            return make_response(
+                jsonify({"success": False, "message": "Operation failed"}), 500
+            )
+
+
+@sources_upload_ns.route("/task_status")
+class TaskStatus(Resource):
+    task_status_model = api.model(
+        "TaskStatusModel",
+        {"task_id": fields.String(required=True, description="Task ID")},
+    )
+
+    @api.expect(task_status_model)
+    @api.doc(description="Get celery job status")
+    def get(self):
+        task_id = request.args.get("task_id")
+        if not task_id:
+            return make_response(
+                jsonify({"success": False, "message": "Task ID is required"}), 400
+            )
+        try:
+            from application.celery_init import celery
+
+            task = celery.AsyncResult(task_id)
+            task_meta = task.info
+            print(f"Task status: {task.status}")
+            if not isinstance(
+                task_meta, (dict, list, str, int, float, bool, type(None))
+            ):
+                task_meta = str(task_meta)  # Convert to a string representation
+        except Exception as err:
+            current_app.logger.error(f"Error getting task status: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"status": task.status, "result": task_meta}), 200)

application/api/user/tools/__init__.py

@@ -0,0 +1,6 @@
+"""Tools module."""
+
+from .mcp import tools_mcp_ns
+from .routes import tools_ns
+
+__all__ = ["tools_ns", "tools_mcp_ns"]

application/api/user/tools/mcp.py

@@ -0,0 +1,333 @@
+"""Tool management MCP server integration."""
+
+import json
+from email.quoprimime import unquote
+
+from bson.objectid import ObjectId
+from flask import current_app, jsonify, make_response, redirect, request
+from flask_restx import fields, Namespace, Resource
+
+from application.agents.tools.mcp_tool import MCPOAuthManager, MCPTool
+from application.api import api
+from application.api.user.base import user_tools_collection
+from application.cache import get_redis_instance
+from application.security.encryption import encrypt_credentials
+from application.utils import check_required_fields
+
+tools_mcp_ns = Namespace("tools", description="Tool management operations", path="/api")
+
+
+@tools_mcp_ns.route("/mcp_server/test")
+class TestMCPServerConfig(Resource):
+    @api.expect(
+        api.model(
+            "MCPServerTestModel",
+            {
+                "config": fields.Raw(
+                    required=True, description="MCP server configuration to test"
+                ),
+            },
+        )
+    )
+    @api.doc(description="Test MCP server connection with provided configuration")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+
+        required_fields = ["config"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        try:
+            config = data["config"]
+
+            auth_credentials = {}
+            auth_type = config.get("auth_type", "none")
+
+            if auth_type == "api_key" and "api_key" in config:
+                auth_credentials["api_key"] = config["api_key"]
+                if "api_key_header" in config:
+                    auth_credentials["api_key_header"] = config["api_key_header"]
+            elif auth_type == "bearer" and "bearer_token" in config:
+                auth_credentials["bearer_token"] = config["bearer_token"]
+            elif auth_type == "basic":
+                if "username" in config:
+                    auth_credentials["username"] = config["username"]
+                if "password" in config:
+                    auth_credentials["password"] = config["password"]
+            test_config = config.copy()
+            test_config["auth_credentials"] = auth_credentials
+
+            mcp_tool = MCPTool(config=test_config, user_id=user)
+            result = mcp_tool.test_connection()
+
+            return make_response(jsonify(result), 200)
+        except Exception as e:
+            current_app.logger.error(f"Error testing MCP server: {e}", exc_info=True)
+            return make_response(
+                jsonify(
+                    {"success": False, "error": f"Connection test failed: {str(e)}"}
+                ),
+                500,
+            )
+
+
+@tools_mcp_ns.route("/mcp_server/save")
+class MCPServerSave(Resource):
+    @api.expect(
+        api.model(
+            "MCPServerSaveModel",
+            {
+                "id": fields.String(
+                    required=False, description="Tool ID for updates (optional)"
+                ),
+                "displayName": fields.String(
+                    required=True, description="Display name for the MCP server"
+                ),
+                "config": fields.Raw(
+                    required=True, description="MCP server configuration"
+                ),
+                "status": fields.Boolean(
+                    required=False, default=True, description="Tool status"
+                ),
+            },
+        )
+    )
+    @api.doc(description="Create or update MCP server with automatic tool discovery")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+
+        required_fields = ["displayName", "config"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        try:
+            config = data["config"]
+
+            auth_credentials = {}
+            auth_type = config.get("auth_type", "none")
+            if auth_type == "api_key":
+                if "api_key" in config and config["api_key"]:
+                    auth_credentials["api_key"] = config["api_key"]
+                if "api_key_header" in config:
+                    auth_credentials["api_key_header"] = config["api_key_header"]
+            elif auth_type == "bearer":
+                if "bearer_token" in config and config["bearer_token"]:
+                    auth_credentials["bearer_token"] = config["bearer_token"]
+            elif auth_type == "basic":
+                if "username" in config and config["username"]:
+                    auth_credentials["username"] = config["username"]
+                if "password" in config and config["password"]:
+                    auth_credentials["password"] = config["password"]
+            mcp_config = config.copy()
+            mcp_config["auth_credentials"] = auth_credentials
+
+            if auth_type == "oauth":
+                if not config.get("oauth_task_id"):
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": False,
+                                "error": "Connection not authorized. Please complete the OAuth authorization first.",
+                            }
+                        ),
+                        400,
+                    )
+                redis_client = get_redis_instance()
+                manager = MCPOAuthManager(redis_client)
+                result = manager.get_oauth_status(config["oauth_task_id"])
+                if not result.get("status") == "completed":
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": False,
+                                "error": "OAuth failed or not completed. Please try authorizing again.",
+                            }
+                        ),
+                        400,
+                    )
+                actions_metadata = result.get("tools", [])
+            elif auth_type == "none" or auth_credentials:
+                mcp_tool = MCPTool(config=mcp_config, user_id=user)
+                mcp_tool.discover_tools()
+                actions_metadata = mcp_tool.get_actions_metadata()
+            else:
+                raise Exception(
+                    "No valid credentials provided for the selected authentication type"
+                )
+            storage_config = config.copy()
+            if auth_credentials:
+                encrypted_credentials_string = encrypt_credentials(
+                    auth_credentials, user
+                )
+                storage_config["encrypted_credentials"] = encrypted_credentials_string
+            for field in [
+                "api_key",
+                "bearer_token",
+                "username",
+                "password",
+                "api_key_header",
+            ]:
+                storage_config.pop(field, None)
+            transformed_actions = []
+            for action in actions_metadata:
+                action["active"] = True
+                if "parameters" in action:
+                    if "properties" in action["parameters"]:
+                        for param_name, param_details in action["parameters"][
+                            "properties"
+                        ].items():
+                            param_details["filled_by_llm"] = True
+                            param_details["value"] = ""
+                transformed_actions.append(action)
+            tool_data = {
+                "name": "mcp_tool",
+                "displayName": data["displayName"],
+                "customName": data["displayName"],
+                "description": f"MCP Server: {storage_config.get('server_url', 'Unknown')}",
+                "config": storage_config,
+                "actions": transformed_actions,
+                "status": data.get("status", True),
+                "user": user,
+            }
+
+            tool_id = data.get("id")
+            if tool_id:
+                result = user_tools_collection.update_one(
+                    {"_id": ObjectId(tool_id), "user": user, "name": "mcp_tool"},
+                    {"$set": {k: v for k, v in tool_data.items() if k != "user"}},
+                )
+                if result.matched_count == 0:
+                    return make_response(
+                        jsonify(
+                            {
+                                "success": False,
+                                "error": "Tool not found or access denied",
+                            }
+                        ),
+                        404,
+                    )
+                response_data = {
+                    "success": True,
+                    "id": tool_id,
+                    "message": f"MCP server updated successfully! Discovered {len(transformed_actions)} tools.",
+                    "tools_count": len(transformed_actions),
+                }
+            else:
+                result = user_tools_collection.insert_one(tool_data)
+                tool_id = str(result.inserted_id)
+                response_data = {
+                    "success": True,
+                    "id": tool_id,
+                    "message": f"MCP server created successfully! Discovered {len(transformed_actions)} tools.",
+                    "tools_count": len(transformed_actions),
+                }
+            return make_response(jsonify(response_data), 200)
+        except Exception as e:
+            current_app.logger.error(f"Error saving MCP server: {e}", exc_info=True)
+            return make_response(
+                jsonify(
+                    {"success": False, "error": f"Failed to save MCP server: {str(e)}"}
+                ),
+                500,
+            )
+
+
+@tools_mcp_ns.route("/mcp_server/callback")
+class MCPOAuthCallback(Resource):
+    @api.expect(
+        api.model(
+            "MCPServerCallbackModel",
+            {
+                "code": fields.String(required=True, description="Authorization code"),
+                "state": fields.String(required=True, description="State parameter"),
+                "error": fields.String(
+                    required=False, description="Error message (if any)"
+                ),
+            },
+        )
+    )
+    @api.doc(
+        description="Handle OAuth callback by providing the authorization code and state"
+    )
+    def get(self):
+        code = request.args.get("code")
+        state = request.args.get("state")
+        error = request.args.get("error")
+
+        if error:
+            return redirect(
+                f"/api/connectors/callback-status?status=error&message=OAuth+error:+{error}.+Please+try+again+and+make+sure+to+grant+all+requested+permissions,+including+offline+access.&provider=mcp_tool"
+            )
+        if not code or not state:
+            return redirect(
+                "/api/connectors/callback-status?status=error&message=Authorization+code+or+state+not+provided.+Please+complete+the+authorization+process+and+make+sure+to+grant+offline+access.&provider=mcp_tool"
+            )
+        try:
+            redis_client = get_redis_instance()
+            if not redis_client:
+                return redirect(
+                    "/api/connectors/callback-status?status=error&message=Internal+server+error:+Redis+not+available.&provider=mcp_tool"
+                )
+            code = unquote(code)
+            manager = MCPOAuthManager(redis_client)
+            success = manager.handle_oauth_callback(state, code, error)
+            if success:
+                return redirect(
+                    "/api/connectors/callback-status?status=success&message=Authorization+code+received+successfully.+You+can+close+this+window.&provider=mcp_tool"
+                )
+            else:
+                return redirect(
+                    "/api/connectors/callback-status?status=error&message=OAuth+callback+failed.&provider=mcp_tool"
+                )
+        except Exception as e:
+            current_app.logger.error(
+                f"Error handling MCP OAuth callback: {str(e)}", exc_info=True
+            )
+            return redirect(
+                f"/api/connectors/callback-status?status=error&message=Internal+server+error:+{str(e)}.&provider=mcp_tool"
+            )
+
+
+@tools_mcp_ns.route("/mcp_server/oauth_status/<string:task_id>")
+class MCPOAuthStatus(Resource):
+    def get(self, task_id):
+        """
+        Get current status of OAuth flow.
+        Frontend should poll this endpoint periodically.
+        """
+        try:
+            redis_client = get_redis_instance()
+            status_key = f"mcp_oauth_status:{task_id}"
+            status_data = redis_client.get(status_key)
+
+            if status_data:
+                status = json.loads(status_data)
+                return make_response(
+                    jsonify({"success": True, "task_id": task_id, **status})
+                )
+            else:
+                return make_response(
+                    jsonify(
+                        {
+                            "success": False,
+                            "error": "Task not found or expired",
+                            "task_id": task_id,
+                        }
+                    ),
+                    404,
+                )
+        except Exception as e:
+            current_app.logger.error(
+                f"Error getting OAuth status for task {task_id}: {str(e)}"
+            )
+            return make_response(
+                jsonify({"success": False, "error": str(e), "task_id": task_id}), 500
+            )

application/api/user/tools/routes.py

@@ -0,0 +1,415 @@
+"""Tool management routes."""
+
+from bson.objectid import ObjectId
+from flask import current_app, jsonify, make_response, request
+from flask_restx import fields, Namespace, Resource
+
+from application.agents.tools.tool_manager import ToolManager
+from application.api import api
+from application.api.user.base import user_tools_collection
+from application.security.encryption import decrypt_credentials, encrypt_credentials
+from application.utils import check_required_fields, validate_function_name
+
+tool_config = {}
+tool_manager = ToolManager(config=tool_config)
+
+
+tools_ns = Namespace("tools", description="Tool management operations", path="/api")
+
+
+@tools_ns.route("/available_tools")
+class AvailableTools(Resource):
+    @api.doc(description="Get available tools for a user")
+    def get(self):
+        try:
+            tools_metadata = []
+            for tool_name, tool_instance in tool_manager.tools.items():
+                doc = tool_instance.__doc__.strip()
+                lines = doc.split("\n", 1)
+                name = lines[0].strip()
+                description = lines[1].strip() if len(lines) > 1 else ""
+                tools_metadata.append(
+                    {
+                        "name": tool_name,
+                        "displayName": name,
+                        "description": description,
+                        "configRequirements": tool_instance.get_config_requirements(),
+                    }
+                )
+        except Exception as err:
+            current_app.logger.error(
+                f"Error getting available tools: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True, "data": tools_metadata}), 200)
+
+
+@tools_ns.route("/get_tools")
+class GetTools(Resource):
+    @api.doc(description="Get tools created by a user")
+    def get(self):
+        try:
+            decoded_token = request.decoded_token
+            if not decoded_token:
+                return make_response(jsonify({"success": False}), 401)
+            user = decoded_token.get("sub")
+            tools = user_tools_collection.find({"user": user})
+            user_tools = []
+            for tool in tools:
+                tool["id"] = str(tool["_id"])
+                tool.pop("_id")
+                user_tools.append(tool)
+        except Exception as err:
+            current_app.logger.error(f"Error getting user tools: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True, "tools": user_tools}), 200)
+
+
+@tools_ns.route("/create_tool")
+class CreateTool(Resource):
+    @api.expect(
+        api.model(
+            "CreateToolModel",
+            {
+                "name": fields.String(required=True, description="Name of the tool"),
+                "displayName": fields.String(
+                    required=True, description="Display name for the tool"
+                ),
+                "description": fields.String(
+                    required=True, description="Tool description"
+                ),
+                "config": fields.Raw(
+                    required=True, description="Configuration of the tool"
+                ),
+                "customName": fields.String(
+                    required=False, description="Custom name for the tool"
+                ),
+                "status": fields.Boolean(
+                    required=True, description="Status of the tool"
+                ),
+            },
+        )
+    )
+    @api.doc(description="Create a new tool")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        required_fields = [
+            "name",
+            "displayName",
+            "description",
+            "config",
+            "status",
+        ]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        try:
+            tool_instance = tool_manager.tools.get(data["name"])
+            if not tool_instance:
+                return make_response(
+                    jsonify({"success": False, "message": "Tool not found"}), 404
+                )
+            actions_metadata = tool_instance.get_actions_metadata()
+            transformed_actions = []
+            for action in actions_metadata:
+                action["active"] = True
+                if "parameters" in action:
+                    if "properties" in action["parameters"]:
+                        for param_name, param_details in action["parameters"][
+                            "properties"
+                        ].items():
+                            param_details["filled_by_llm"] = True
+                            param_details["value"] = ""
+                transformed_actions.append(action)
+        except Exception as err:
+            current_app.logger.error(
+                f"Error getting tool actions: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        try:
+            new_tool = {
+                "user": user,
+                "name": data["name"],
+                "displayName": data["displayName"],
+                "description": data["description"],
+                "customName": data.get("customName", ""),
+                "actions": transformed_actions,
+                "config": data["config"],
+                "status": data["status"],
+            }
+            resp = user_tools_collection.insert_one(new_tool)
+            new_id = str(resp.inserted_id)
+        except Exception as err:
+            current_app.logger.error(f"Error creating tool: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"id": new_id}), 200)
+
+
+@tools_ns.route("/update_tool")
+class UpdateTool(Resource):
+    @api.expect(
+        api.model(
+            "UpdateToolModel",
+            {
+                "id": fields.String(required=True, description="Tool ID"),
+                "name": fields.String(description="Name of the tool"),
+                "displayName": fields.String(description="Display name for the tool"),
+                "customName": fields.String(description="Custom name for the tool"),
+                "description": fields.String(description="Tool description"),
+                "config": fields.Raw(description="Configuration of the tool"),
+                "actions": fields.List(
+                    fields.Raw, description="Actions the tool can perform"
+                ),
+                "status": fields.Boolean(description="Status of the tool"),
+            },
+        )
+    )
+    @api.doc(description="Update a tool by ID")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        required_fields = ["id"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        try:
+            update_data = {}
+            if "name" in data:
+                update_data["name"] = data["name"]
+            if "displayName" in data:
+                update_data["displayName"] = data["displayName"]
+            if "customName" in data:
+                update_data["customName"] = data["customName"]
+            if "description" in data:
+                update_data["description"] = data["description"]
+            if "actions" in data:
+                update_data["actions"] = data["actions"]
+            if "config" in data:
+                if "actions" in data["config"]:
+                    for action_name in list(data["config"]["actions"].keys()):
+                        if not validate_function_name(action_name):
+                            return make_response(
+                                jsonify(
+                                    {
+                                        "success": False,
+                                        "message": f"Invalid function name '{action_name}'. Function names must match pattern '^[a-zA-Z0-9_-]+$'.",
+                                        "param": "tools[].function.name",
+                                    }
+                                ),
+                                400,
+                            )
+                tool_doc = user_tools_collection.find_one(
+                    {"_id": ObjectId(data["id"]), "user": user}
+                )
+                if tool_doc and tool_doc.get("name") == "mcp_tool":
+                    config = data["config"]
+                    existing_config = tool_doc.get("config", {})
+                    storage_config = existing_config.copy()
+
+                    storage_config.update(config)
+                    existing_credentials = {}
+                    if "encrypted_credentials" in existing_config:
+                        existing_credentials = decrypt_credentials(
+                            existing_config["encrypted_credentials"], user
+                        )
+                    auth_credentials = existing_credentials.copy()
+                    auth_type = storage_config.get("auth_type", "none")
+                    if auth_type == "api_key":
+                        if "api_key" in config and config["api_key"]:
+                            auth_credentials["api_key"] = config["api_key"]
+                        if "api_key_header" in config:
+                            auth_credentials["api_key_header"] = config[
+                                "api_key_header"
+                            ]
+                    elif auth_type == "bearer":
+                        if "bearer_token" in config and config["bearer_token"]:
+                            auth_credentials["bearer_token"] = config["bearer_token"]
+                        elif "encrypted_token" in config and config["encrypted_token"]:
+                            auth_credentials["bearer_token"] = config["encrypted_token"]
+                    elif auth_type == "basic":
+                        if "username" in config and config["username"]:
+                            auth_credentials["username"] = config["username"]
+                        if "password" in config and config["password"]:
+                            auth_credentials["password"] = config["password"]
+                    if auth_type != "none" and auth_credentials:
+                        encrypted_credentials_string = encrypt_credentials(
+                            auth_credentials, user
+                        )
+                        storage_config["encrypted_credentials"] = (
+                            encrypted_credentials_string
+                        )
+                    elif auth_type == "none":
+                        storage_config.pop("encrypted_credentials", None)
+                    for field in [
+                        "api_key",
+                        "bearer_token",
+                        "encrypted_token",
+                        "username",
+                        "password",
+                        "api_key_header",
+                    ]:
+                        storage_config.pop(field, None)
+                    update_data["config"] = storage_config
+                else:
+                    update_data["config"] = data["config"]
+            if "status" in data:
+                update_data["status"] = data["status"]
+            user_tools_collection.update_one(
+                {"_id": ObjectId(data["id"]), "user": user},
+                {"$set": update_data},
+            )
+        except Exception as err:
+            current_app.logger.error(f"Error updating tool: {err}", exc_info=True)
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True}), 200)
+
+
+@tools_ns.route("/update_tool_config")
+class UpdateToolConfig(Resource):
+    @api.expect(
+        api.model(
+            "UpdateToolConfigModel",
+            {
+                "id": fields.String(required=True, description="Tool ID"),
+                "config": fields.Raw(
+                    required=True, description="Configuration of the tool"
+                ),
+            },
+        )
+    )
+    @api.doc(description="Update the configuration of a tool")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        required_fields = ["id", "config"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        try:
+            user_tools_collection.update_one(
+                {"_id": ObjectId(data["id"]), "user": user},
+                {"$set": {"config": data["config"]}},
+            )
+        except Exception as err:
+            current_app.logger.error(
+                f"Error updating tool config: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True}), 200)
+
+
+@tools_ns.route("/update_tool_actions")
+class UpdateToolActions(Resource):
+    @api.expect(
+        api.model(
+            "UpdateToolActionsModel",
+            {
+                "id": fields.String(required=True, description="Tool ID"),
+                "actions": fields.List(
+                    fields.Raw,
+                    required=True,
+                    description="Actions the tool can perform",
+                ),
+            },
+        )
+    )
+    @api.doc(description="Update the actions of a tool")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        required_fields = ["id", "actions"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        try:
+            user_tools_collection.update_one(
+                {"_id": ObjectId(data["id"]), "user": user},
+                {"$set": {"actions": data["actions"]}},
+            )
+        except Exception as err:
+            current_app.logger.error(
+                f"Error updating tool actions: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True}), 200)
+
+
+@tools_ns.route("/update_tool_status")
+class UpdateToolStatus(Resource):
+    @api.expect(
+        api.model(
+            "UpdateToolStatusModel",
+            {
+                "id": fields.String(required=True, description="Tool ID"),
+                "status": fields.Boolean(
+                    required=True, description="Status of the tool"
+                ),
+            },
+        )
+    )
+    @api.doc(description="Update the status of a tool")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        required_fields = ["id", "status"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        try:
+            user_tools_collection.update_one(
+                {"_id": ObjectId(data["id"]), "user": user},
+                {"$set": {"status": data["status"]}},
+            )
+        except Exception as err:
+            current_app.logger.error(
+                f"Error updating tool status: {err}", exc_info=True
+            )
+            return make_response(jsonify({"success": False}), 400)
+        return make_response(jsonify({"success": True}), 200)
+
+
+@tools_ns.route("/delete_tool")
+class DeleteTool(Resource):
+    @api.expect(
+        api.model(
+            "DeleteToolModel",
+            {"id": fields.String(required=True, description="Tool ID")},
+        )
+    )
+    @api.doc(description="Delete a tool by ID")
+    def post(self):
+        decoded_token = request.decoded_token
+        if not decoded_token:
+            return make_response(jsonify({"success": False}), 401)
+        user = decoded_token.get("sub")
+        data = request.get_json()
+        required_fields = ["id"]
+        missing_fields = check_required_fields(data, required_fields)
+        if missing_fields:
+            return missing_fields
+        try:
+            result = user_tools_collection.delete_one(
+                {"_id": ObjectId(data["id"]), "user": user}
+            )
+            if result.deleted_count == 0:
+                return {"success": False, "message": "Tool not found"}, 404
+        except Exception as err:
+            current_app.logger.error(f"Error deleting tool: {err}", exc_info=True)
+            return {"success": False}, 400
+        return {"success": True}, 200

application/core/settings.py

@@ -41,10 +41,15 @@ class Settings(BaseSettings):
     FALLBACK_LLM_API_KEY: Optional[str] = None  # api key for fallback llm
 
     # Google Drive integration
-    GOOGLE_CLIENT_ID: Optional[str] = None # Replace with your actual Google OAuth client ID
-    GOOGLE_CLIENT_SECRET: Optional[str] = None# Replace with your actual Google OAuth client secret
-    CONNECTOR_REDIRECT_BASE_URI: Optional[str] = "http://127.0.0.1:7091/api/connectors/callback" ##add redirect url as it is to your provider's console(gcp)
-
+    GOOGLE_CLIENT_ID: Optional[str] = (
+        None  # Replace with your actual Google OAuth client ID
+    )
+    GOOGLE_CLIENT_SECRET: Optional[str] = (
+        None  # Replace with your actual Google OAuth client secret
+    )
+    CONNECTOR_REDIRECT_BASE_URI: Optional[str] = (
+        "http://127.0.0.1:7091/api/connectors/callback"  ##add redirect url as it is to your provider's console(gcp)
+    )
 
     # LLM Cache
     CACHE_REDIS_URL: str = "redis://localhost:6379/2"
@@ -118,6 +123,7 @@ class Settings(BaseSettings):
     # Encryption settings
     ENCRYPTION_SECRET_KEY: str = "default-docsgpt-encryption-key"
 
+    ELEVENLABS_API_KEY: Optional[str] = None
 
 path = Path(__file__).parent.parent.absolute()
 settings = Settings(_env_file=path.joinpath(".env"), _env_file_encoding="utf-8")

application/retriever/classic_rag.py

@@ -36,6 +36,11 @@ class ClassicRAG(BaseRetriever):
                 self.chunks = 2
         else:
             self.chunks = chunks
+        user_identifier = user_api_key if user_api_key else "default"
+        logging.info(
+            f"ClassicRAG initialized with chunks={self.chunks}, user_api_key={user_identifier}, "
+            f"sources={'active_docs' in source and source['active_docs'] is not None}"
+        )
         self.gpt_model = gpt_model
         self.token_limit = (
             token_limit
@@ -92,17 +97,12 @@ class ClassicRAG(BaseRetriever):
             or not self.vectorstores
         ):
             return self.original_question
-        prompt = f"""Given the following conversation history:
-
-        {self.chat_history}
-
-
-
-        Rephrase the following user question to be a standalone search query 
-
-        that captures all relevant context from the conversation:
-
-        """
+        prompt = (
+            "Given the following conversation history:\n"
+            f"{self.chat_history}\n\n"
+            "Rephrase the following user question to be a standalone search query "
+            "that captures all relevant context from the conversation:\n"
+        )
 
         messages = [
             {"role": "system", "content": prompt},
@@ -120,10 +120,20 @@ class ClassicRAG(BaseRetriever):
     def _get_data(self):
         """Retrieve relevant documents from configured vectorstores"""
         if self.chunks == 0 or not self.vectorstores:
+            logging.info(
+                f"ClassicRAG._get_data: Skipping retrieval - chunks={self.chunks}, "
+                f"vectorstores_count={len(self.vectorstores) if self.vectorstores else 0}"
+            )
             return []
         all_docs = []
         chunks_per_source = max(1, self.chunks // len(self.vectorstores))
 
+        logging.info(
+            f"ClassicRAG._get_data: Starting retrieval with chunks={self.chunks}, "
+            f"vectorstores={self.vectorstores}, chunks_per_source={chunks_per_source}, "
+            f"query='{self.question[:50]}...'"
+        )
+
         for vectorstore_id in self.vectorstores:
             if vectorstore_id:
                 try:
@@ -172,6 +182,10 @@ class ClassicRAG(BaseRetriever):
                         exc_info=True,
                     )
                     continue
+        logging.info(
+            f"ClassicRAG._get_data: Retrieval complete - retrieved {len(all_docs)} documents "
+            f"(requested chunks={self.chunks}, chunks_per_source={chunks_per_source})"
+        )
         return all_docs
 
     def search(self, query: str = ""):

application/storage/local.py

@@ -26,7 +26,7 @@ class LocalStorage(BaseStorage):
             return path
         return os.path.join(self.base_dir, path)
 
-    def save_file(self, file_data: BinaryIO, path: str) -> dict:
+    def save_file(self, file_data: BinaryIO, path: str, **kwargs) -> dict:
         """Save a file to local storage."""
         full_path = self._get_full_path(path)
 

application/tts/elevenlabs.py

@@ -1,84 +1,30 @@
-import asyncio
-import websockets
-import json
-import base64
 from io import BytesIO
+import base64
 from application.tts.base import BaseTTS
+from application.core.settings import settings
 
 
 class ElevenlabsTTS(BaseTTS):
-    def __init__(self):        
-        self.api_key = 'ELEVENLABS_API_KEY'# here you should put your api key
-        self.model = "eleven_flash_v2_5"
-        self.voice = "VOICE_ID" # this is the hash code for the voice not the name!
-        self.write_audio = 1
+    def __init__(self):
+        from elevenlabs.client import ElevenLabs
+
+        self.client = ElevenLabs(
+            api_key=settings.ELEVENLABS_API_KEY,
+            )
+    
 
     def text_to_speech(self, text):
-        asyncio.run(self._text_to_speech_websocket(text))
+        lang = "en"
+        audio = self.client.generate(
+            text=text,
+            model="eleven_multilingual_v2",
+            voice="Brian",
+        )
+        audio_data = BytesIO()
+        for chunk in audio:
+            audio_data.write(chunk)
+        audio_bytes = audio_data.getvalue()
 
-    async def _text_to_speech_websocket(self, text):
-        uri = f"wss://api.elevenlabs.io/v1/text-to-speech/{self.voice}/stream-input?model_id={self.model}"
-        websocket = await websockets.connect(uri)
-        payload = {
-            "text": " ",
-            "voice_settings": {
-                "stability": 0.5,
-                "similarity_boost": 0.8,
-            },
-            "xi_api_key": self.api_key,
-        }
-
-        await websocket.send(json.dumps(payload))
-        
-        async def listen():
-            while 1:
-                try:
-                    msg = await websocket.recv()
-                    data = json.loads(msg)
-
-                    if data.get("audio"):
-                        print("audio received")
-                        yield base64.b64decode(data["audio"])
-                    elif data.get("isFinal"):
-                        break
-                except websockets.exceptions.ConnectionClosed:
-                    print("websocket closed")
-                    break
-        listen_task =  asyncio.create_task(self.stream(listen()))
-        
-        await websocket.send(json.dumps({"text": text}))
-        # this is to signal the end of the text, either use this or flush
-        await websocket.send(json.dumps({"text": ""})) 
-
-        await listen_task
-    
-    async def stream(self, audio_stream):
-        if self.write_audio:
-            audio_bytes = BytesIO()
-            async for chunk in audio_stream:
-                if chunk:
-                    audio_bytes.write(chunk)
-            with open("output_audio.mp3", "wb") as f:
-                f.write(audio_bytes.getvalue())
-        
-        else:
-            async for chunk in audio_stream:
-                pass # depends on the streamer!
-
-
-def test_elevenlabs_websocket():
-    """
-    Tests the ElevenlabsTTS text_to_speech method with a sample prompt.
-    Prints out the base64-encoded result and writes it to 'output_audio.mp3'.
-    """
-    # Instantiate your TTS class
-    tts = ElevenlabsTTS()
-
-    # Call the method with some sample text
-    tts.text_to_speech("Hello from ElevenLabs WebSocket!")
-
-    print("Saved audio to output_audio.mp3.")
-
-
-if __name__ == "__main__":
-    test_elevenlabs_websocket()
+        # Encode to base64
+        audio_base64 = base64.b64encode(audio_bytes).decode("utf-8")
+        return audio_base64, lang

application/vectorstore/base.py

@@ -1,3 +1,4 @@
+import logging
 import os
 from abc import ABC, abstractmethod
 
@@ -9,13 +10,27 @@ from application.core.settings import settings
 
 class EmbeddingsWrapper:
     def __init__(self, model_name, *args, **kwargs):
-        self.model = SentenceTransformer(
-            model_name,
-            config_kwargs={"allow_dangerous_deserialization": True},
-            *args,
-            **kwargs
-        )
-        self.dimension = self.model.get_sentence_embedding_dimension()
+        logging.info(f"Initializing EmbeddingsWrapper with model: {model_name}")
+        try:
+            kwargs.setdefault("trust_remote_code", True)
+            self.model = SentenceTransformer(
+                model_name,
+                config_kwargs={"allow_dangerous_deserialization": True},
+                *args,
+                **kwargs,
+            )
+            if self.model is None or self.model._first_module() is None:
+                raise ValueError(
+                    f"SentenceTransformer model failed to load properly for: {model_name}"
+                )
+            self.dimension = self.model.get_sentence_embedding_dimension()
+            logging.info(f"Successfully loaded model with dimension: {self.dimension}")
+        except Exception as e:
+            logging.error(
+                f"Failed to initialize SentenceTransformer with model {model_name}: {str(e)}",
+                exc_info=True,
+            )
+            raise
 
     def embed_query(self, query: str):
         return self.model.encode(query).tolist()
@@ -117,15 +132,29 @@ class BaseVectorStore(ABC):
                     embeddings_name, openai_api_key=embeddings_key
                 )
         elif embeddings_name == "huggingface_sentence-transformers/all-mpnet-base-v2":
-            if os.path.exists("./models/all-mpnet-base-v2"):
+            possible_paths = [
+                "/app/models/all-mpnet-base-v2",  # Docker absolute path
+                "./models/all-mpnet-base-v2",  # Relative path
+            ]
+            local_model_path = None
+            for path in possible_paths:
+                if os.path.exists(path):
+                    local_model_path = path
+                    logging.info(f"Found local model at path: {path}")
+                    break
+                else:
+                    logging.info(f"Path does not exist: {path}")
+            if local_model_path:
                 embedding_instance = EmbeddingsSingleton.get_instance(
-                    embeddings_name="./models/all-mpnet-base-v2",
+                    local_model_path,
                 )
             else:
+                logging.warning(
+                    f"Local model not found in any of the paths: {possible_paths}. Falling back to HuggingFace download."
+                )
                 embedding_instance = EmbeddingsSingleton.get_instance(
                     embeddings_name,
                 )
         else:
             embedding_instance = EmbeddingsSingleton.get_instance(embeddings_name)
-
         return embedding_instance

frontend/package-lock.json

@@ -12,7 +12,7 @@
         "chart.js": "^4.4.4",
         "clsx": "^2.1.1",
         "copy-to-clipboard": "^3.3.3",
-        "i18next": "^24.2.0",
+        "i18next": "^25.5.3",
         "i18next-browser-languagedetector": "^8.0.2",
         "lodash": "^4.17.21",
         "mermaid": "^11.6.0",
@@ -321,9 +321,9 @@
       }
     },
     "node_modules/@babel/runtime": {
-      "version": "7.27.3",
-      "resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.27.3.tgz",
-      "integrity": "sha512-7EYtGezsdiDMyY80+65EzwiGmcJqpmcZCojSXaRgdrBaGtWTgDZKq69cPIVped6MkIM78cTQ2GOiEYjwOlG4xw==",
+      "version": "7.28.4",
+      "resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.28.4.tgz",
+      "integrity": "sha512-Q/N6JNWvIvPnLDvjlE1OUBLPQHH6l3CltCEsHIujp45zQUSSh8K+gHnaEX45yAT1nyngnINhvWtzN+Nb9D8RAQ==",
       "license": "MIT",
       "engines": {
         "node": ">=6.9.0"
@@ -6217,9 +6217,9 @@
       }
     },
     "node_modules/i18next": {
-      "version": "24.2.0",
-      "resolved": "https://registry.npmjs.org/i18next/-/i18next-24.2.0.tgz",
-      "integrity": "sha512-ArJJTS1lV6lgKH7yEf4EpgNZ7+THl7bsGxxougPYiXRTJ/Fe1j08/TBpV9QsXCIYVfdE/HWG/xLezJ5DOlfBOA==",
+      "version": "25.5.3",
+      "resolved": "https://registry.npmjs.org/i18next/-/i18next-25.5.3.tgz",
+      "integrity": "sha512-joFqorDeQ6YpIXni944upwnuHBf5IoPMuqAchGVeQLdWC2JOjxgM9V8UGLhNIIH/Q8QleRxIi0BSRQehSrDLcg==",
       "funding": [
         {
           "type": "individual",
@@ -6234,8 +6234,9 @@
           "url": "https://www.i18next.com/how-to/faq#i18next-is-awesome.-how-can-i-support-the-project"
         }
       ],
+      "license": "MIT",
       "dependencies": {
-        "@babel/runtime": "^7.23.2"
+        "@babel/runtime": "^7.27.6"
       },
       "peerDependencies": {
         "typescript": "^5"

frontend/package.json

@@ -23,7 +23,7 @@
     "chart.js": "^4.4.4",
     "clsx": "^2.1.1",
     "copy-to-clipboard": "^3.3.3",
-    "i18next": "^24.2.0",
+    "i18next": "^25.5.3",
     "i18next-browser-languagedetector": "^8.0.2",
     "lodash": "^4.17.21",
     "mermaid": "^11.6.0",

frontend/public/toolIcons/tool_memory.svg

@@ -0,0 +1,3 @@
+<svg xmlns="http://www.w3.org/2000/svg" height="24px" viewBox="0 -960 960 960" width="24px" fill="#e3e3e3">
+  <path d="M240-80q-33 0-56.5-23.5T160-160v-480q0-33 23.5-56.5T240-720h80v-80q0-17 11.5-28.5T360-840q17 0 28.5 11.5T400-800v80h40v-80q0-17 11.5-28.5T480-840q17 0 28.5 11.5T520-800v80h40v-80q0-17 11.5-28.5T600-840q17 0 28.5 11.5T640-800v80h80q33 0 56.5 23.5T800-640v480q0 33-23.5 56.5T720-80H240Zm0-80h480v-480H240v480Zm120-320v-80h240v80H360Zm0 120v-80h240v80H360Zm0 120v-80h160v80H360ZM240-160v-480 480Z"/>
+</svg>

frontend/public/toolIcons/tool_notes.svg

@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg" height="24px" viewBox="0 -960 960 960" width="24px" fill="#e3e3e3"><path d="M320-240h320v-80H320v80Zm0-160h320v-80H320v80ZM240-80q-33 0-56.5-23.5T160-160v-640q0-33 23.5-56.5T240-880h320l240 240v480q0 33-23.5 56.5T720-80H240Zm280-520v-200H240v640h480v-440H520ZM240-800v200-200 640-640Z"/></svg>

frontend/src/App.tsx

@@ -7,6 +7,7 @@ import Agents from './agents';
 import SharedAgentGate from './agents/SharedAgentGate';
 import ActionButtons from './components/ActionButtons';
 import Spinner from './components/Spinner';
+import UploadToast from './components/UploadToast';
 import Conversation from './conversation/Conversation';
 import { SharedConversation } from './conversation/SharedConversation';
 import { useDarkTheme, useMediaQuery } from './hooks';
@@ -33,18 +34,19 @@ function MainLayout() {
   const [navOpen, setNavOpen] = useState(!(isMobile || isTablet));
 
   return (
-    <div className="relative h-screen overflow-hidden dark:bg-raisin-black">
+    <div className="dark:bg-raisin-black relative h-screen overflow-hidden">
       <Navigation navOpen={navOpen} setNavOpen={setNavOpen} />
       <ActionButtons showNewChat={true} showShare={true} />
       <div
-        className={`h-[calc(100dvh-64px)] overflow-auto lg:h-screen ${
+        className={`h-[calc(100dvh-64px)] overflow-auto transition-all duration-300 ease-in-out lg:h-screen ${
           !(isMobile || isTablet)
-            ? `ml-0 ${!navOpen ? 'lg:mx-auto' : 'lg:ml-72'}`
+            ? `${navOpen ? 'lg:ml-72' : 'lg:ml-0'}`
             : 'ml-0 lg:ml-16'
         }`}
       >
         <Outlet />
       </div>
+      <UploadToast />
     </div>
   );
 }

frontend/src/Navigation.tsx

@@ -14,7 +14,7 @@ import Github from './assets/git_nav.svg';
 import Hamburger from './assets/hamburger.svg';
 import openNewChat from './assets/openNewChat.svg';
 import Pin from './assets/pin.svg';
-import Robot from './assets/robot.svg';
+import AgentImage from './components/AgentImage';
 import SettingGear from './assets/settingGear.svg';
 import Spark from './assets/spark.svg';
 import SpinnerDark from './assets/spinner-dark.svg';
@@ -292,20 +292,26 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
   useDefaultDocument();
   return (
     <>
-      {!navOpen && (
-        <div className="absolute top-3 left-3 z-20 hidden transition-all duration-25 lg:block">
+      {(isMobile || isTablet) && navOpen && (
+        <div
+          className="fixed inset-0 z-10 bg-black opacity-50 transition-opacity duration-300"
+          onClick={() => setNavOpen(false)}
+        />
+      )}
+
+      {
+        <div className="absolute top-3 left-3 z-20 hidden transition-all duration-300 ease-in-out lg:block">
           <div className="flex items-center gap-3">
             <button
               onClick={() => {
                 setNavOpen(!navOpen);
               }}
+              className="transition-transform duration-200 hover:scale-110"
             >
               <img
                 src={Expand}
                 alt="Toggle navigation menu"
-                className={`${
-                  !navOpen ? 'rotate-180' : 'rotate-0'
-                } m-auto transition-all duration-200`}
+                className="m-auto transition-all duration-300 ease-in-out"
               />
             </button>
             {queries?.length > 0 && (
@@ -313,6 +319,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
                 onClick={() => {
                   newChat();
                 }}
+                className="transition-transform duration-200 hover:scale-110"
               >
                 <img
                   src={openNewChat}
@@ -326,12 +333,12 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
             </div>
           </div>
         </div>
-      )}
+      }
       <div
         ref={navRef}
         className={`${
           !navOpen && '-ml-96 md:-ml-72'
-        } bg-lotion dark:border-r-purple-taupe dark:bg-chinese-black fixed top-0 z-20 flex h-full w-72 flex-col border-r border-b-0 transition-all duration-20 dark:text-white`}
+        } bg-lotion dark:border-r-purple-taupe dark:bg-chinese-black fixed top-0 z-20 flex h-full w-72 flex-col border-r border-b-0 transition-all duration-300 ease-in-out dark:text-white`}
       >
         <div
           className={'visible mt-2 flex h-[6vh] w-full justify-between md:h-12'}
@@ -345,7 +352,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
             }}
           >
             <a href="/" className="flex gap-1.5">
-              <img className="mb-2 h-10" src={DocsGPT3} alt="DocsGPT Logo" />
+              <img className="h-10" src={DocsGPT3} alt="DocsGPT Logo" />
               <p className="my-auto text-2xl font-semibold">DocsGPT</p>
             </a>
           </div>
@@ -358,9 +365,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
             <img
               src={Expand}
               alt="Toggle navigation menu"
-              className={`${
-                !navOpen ? 'rotate-180' : 'rotate-0'
-              } m-auto transition-all duration-200`}
+              className="m-auto transition-all duration-300 ease-in-out hover:scale-110"
             />
           </button>
         </div>
@@ -419,12 +424,8 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
                     >
                       <div className="flex items-center gap-2">
                         <div className="flex w-6 justify-center">
-                          <img
-                            src={
-                              agent.image && agent.image.trim() !== ''
-                                ? agent.image
-                                : Robot
-                            }
+                          <AgentImage
+                            src={agent.image}
                             alt="agent-logo"
                             className="h-6 w-6 rounded-full object-contain"
                           />
@@ -576,7 +577,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
                 </NavLink>
                 <NavLink
                   target="_blank"
-                  to={'https://twitter.com/docsgptai'}
+                  to={'https://x.com/docsgptai'}
                   className={
                     'rounded-full hover:bg-gray-100 dark:hover:bg-[#28292E]'
                   }
@@ -585,7 +586,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
                     src={Twitter}
                     width={20}
                     height={20}
-                    alt="Follow us on Twitter"
+                    alt="Follow us on X"
                     className="m-2 self-center filter dark:invert"
                   />
                 </NavLink>

frontend/src/PageNotFound.tsx

@@ -2,11 +2,11 @@ import { Link } from 'react-router-dom';
 
 export default function PageNotFound() {
   return (
-    <div className="grid min-h-screen dark:bg-raisin-black">
-      <p className="mx-auto my-auto mt-20 flex w-full max-w-6xl flex-col place-items-center gap-6 rounded-3xl bg-gray-100 p-6 text-jet dark:bg-outer-space dark:text-gray-100 lg:p-10 xl:p-16">
+    <div className="dark:bg-raisin-black grid min-h-screen">
+      <p className="text-jet dark:bg-outer-space mx-auto my-auto mt-20 flex w-full max-w-6xl flex-col place-items-center gap-6 rounded-3xl bg-gray-100 p-6 lg:p-10 xl:p-16 dark:text-gray-100">
         <h1>404</h1>
         <p>The page you are looking for does not exist.</p>
-        <button className="pointer-cursor mr-4 flex cursor-pointer items-center justify-center rounded-full bg-blue-1000 px-4 py-2 text-white transition-colors duration-100 hover:bg-blue-3000">
+        <button className="pointer-cursor bg-blue-1000 hover:bg-blue-3000 mr-4 flex cursor-pointer items-center justify-center rounded-full px-4 py-2 text-white transition-colors duration-100">
           <Link to="/">Go Back Home</Link>
         </button>
       </p>

frontend/src/agents/AgentCard.tsx

@@ -3,7 +3,7 @@ import { useDispatch, useSelector } from 'react-redux';
 import { useNavigate } from 'react-router-dom';
 
 import userService from '../api/services/userService';
-import Robot from '../assets/robot.svg';
+import AgentImage from '../components/AgentImage';
 import ThreeDots from '../assets/three-dots.svg';
 import ContextMenu, { MenuOption } from '../components/ContextMenu';
 import ConfirmationModal from '../modals/ConfirmationModal';
@@ -82,8 +82,8 @@ export default function AgentCard({
 
       <div className="w-full">
         <div className="flex w-full items-center gap-1 px-1">
-          <img
-            src={agent.image && agent.image.trim() !== '' ? agent.image : Robot}
+          <AgentImage
+            src={agent.image}
             alt={`${agent.name}`}
             className="h-7 w-7 rounded-full object-contain"
           />

frontend/src/agents/AgentLogs.tsx

@@ -49,7 +49,7 @@ export default function AgentLogs() {
         </p>
       </div>
       <div className="mt-5 flex w-full flex-wrap items-center justify-between gap-2 px-4">
-        <h1 className="text-eerie-black m-0 text-[40px] font-bold dark:text-white">
+        <h1 className="text-eerie-black m-0 text-[32px] font-bold md:text-[40px] dark:text-white">
           Agent Logs
         </h1>
       </div>

frontend/src/agents/NewAgent.tsx

@@ -46,11 +46,11 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) {
     image: '',
     source: '',
     sources: [],
-    chunks: '',
-    retriever: '',
+    chunks: '2',
+    retriever: 'classic',
     prompt_id: 'default',
     tools: [],
-    agent_type: '',
+    agent_type: 'classic',
     status: '',
     json_schema: undefined,
   });
@@ -122,7 +122,8 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) {
       agent.name && agent.description && agent.prompt_id && agent.agent_type;
     const isJsonSchemaValidOrEmpty =
       jsonSchemaText.trim() === '' || jsonSchemaValid;
-    return hasRequiredFields && isJsonSchemaValidOrEmpty;
+    const hasSource = selectedSourceIds.size > 0;
+    return hasRequiredFields && isJsonSchemaValidOrEmpty && hasSource;
   };
 
   const isJsonSchemaInvalid = () => {
@@ -353,6 +354,26 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) {
     getPrompts();
   }, [token]);
 
+  // Auto-select default source if none selected
+  useEffect(() => {
+    if (sourceDocs && sourceDocs.length > 0 && selectedSourceIds.size === 0) {
+      const defaultSource = sourceDocs.find((s) => s.name === 'Default');
+      if (defaultSource) {
+        setSelectedSourceIds(
+          new Set([
+            defaultSource.id || defaultSource.retriever || defaultSource.name,
+          ]),
+        );
+      } else {
+        setSelectedSourceIds(
+          new Set([
+            sourceDocs[0].id || sourceDocs[0].retriever || sourceDocs[0].name,
+          ]),
+        );
+      }
+    }
+  }, [sourceDocs, selectedSourceIds.size]);
+
   useEffect(() => {
     if ((mode === 'edit' || mode === 'draft') && agentId) {
       const getAgent = async () => {
@@ -506,7 +527,7 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) {
         </p>
       </div>
       <div className="mt-5 flex w-full flex-wrap items-center justify-between gap-2 px-4">
-        <h1 className="text-eerie-black m-0 text-[40px] font-bold dark:text-white">
+        <h1 className="text-eerie-black m-0 text-[32px] font-bold lg:text-[40px] dark:text-white">
           {modeConfig[effectiveMode].heading}
         </h1>
         <div className="flex flex-wrap items-center gap-1">
@@ -650,7 +671,34 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) {
                   }
                   selectedIds={selectedSourceIds}
                   onSelectionChange={(newSelectedIds: Set<string | number>) => {
-                    setSelectedSourceIds(newSelectedIds);
+                    if (
+                      newSelectedIds.size === 0 &&
+                      sourceDocs &&
+                      sourceDocs.length > 0
+                    ) {
+                      const defaultSource = sourceDocs.find(
+                        (s) => s.name === 'Default',
+                      );
+                      if (defaultSource) {
+                        setSelectedSourceIds(
+                          new Set([
+                            defaultSource.id ||
+                              defaultSource.retriever ||
+                              defaultSource.name,
+                          ]),
+                        );
+                      } else {
+                        setSelectedSourceIds(
+                          new Set([
+                            sourceDocs[0].id ||
+                              sourceDocs[0].retriever ||
+                              sourceDocs[0].name,
+                          ]),
+                        );
+                      }
+                    } else {
+                      setSelectedSourceIds(newSelectedIds);
+                    }
                   }}
                   title="Select Sources"
                   searchPlaceholder="Search sources..."

frontend/src/agents/SharedAgent.tsx

@@ -6,7 +6,7 @@ import { useParams } from 'react-router-dom';
 import userService from '../api/services/userService';
 import NoFilesDarkIcon from '../assets/no-files-dark.svg';
 import NoFilesIcon from '../assets/no-files.svg';
-import Robot from '../assets/robot.svg';
+import AgentImage from '../components/AgentImage';
 import MessageInput from '../components/MessageInput';
 import Spinner from '../components/Spinner';
 import ConversationMessages from '../conversation/ConversationMessages';
@@ -152,12 +152,8 @@ export default function SharedAgent() {
   return (
     <div className="relative h-full w-full">
       <div className="absolute top-5 left-4 hidden items-center gap-3 sm:flex">
-        <img
-          src={
-            sharedAgent.image && sharedAgent.image.trim() !== ''
-              ? sharedAgent.image
-              : Robot
-          }
+        <AgentImage
+          src={sharedAgent.image}
           alt="agent-logo"
           className="h-6 w-6 rounded-full object-contain"
         />

frontend/src/agents/SharedAgentCard.tsx

@@ -1,4 +1,4 @@
-import Robot from '../assets/robot.svg';
+import AgentImage from '../components/AgentImage';
 import { Agent } from './types';
 
 export default function SharedAgentCard({ agent }: { agent: Agent }) {
@@ -6,8 +6,8 @@ export default function SharedAgentCard({ agent }: { agent: Agent }) {
     <div className="border-dark-gray dark:border-grey flex w-full max-w-[720px] flex-col rounded-3xl border p-6 shadow-xs sm:w-fit sm:min-w-[480px]">
       <div className="flex items-center gap-3">
         <div className="flex h-12 w-12 items-center justify-center overflow-hidden rounded-full p-1">
-          <img
-            src={agent.image && agent.image.trim() !== '' ? agent.image : Robot}
+          <AgentImage
+            src={agent.image}
             className="h-full w-full rounded-full object-contain"
           />
         </div>

frontend/src/agents/index.tsx

@@ -8,7 +8,7 @@ import Link from '../assets/link-gray.svg';
 import Monitoring from '../assets/monitoring.svg';
 import Pin from '../assets/pin.svg';
 import Trash from '../assets/red-trash.svg';
-import Robot from '../assets/robot.svg';
+import AgentImage from '../components/AgentImage';
 import ThreeDots from '../assets/three-dots.svg';
 import UnPin from '../assets/unpin.svg';
 import ContextMenu, { MenuOption } from '../components/ContextMenu';
@@ -111,7 +111,7 @@ function AgentsList() {
   }, [token]);
   return (
     <div className="p-4 md:p-12">
-      <h1 className="text-eerie-black mb-0 text-[40px] font-bold dark:text-[#E0E0E0]">
+      <h1 className="text-eerie-black mb-0 text-[32px] font-bold lg:text-[40px] dark:text-[#E0E0E0]">
         Agents
       </h1>
       <p className="dark:text-gray-4000 mt-5 text-[15px] text-[#71717A]">
@@ -138,11 +138,7 @@ function AgentsList() {
               </button>
               <div className="w-full">
                 <div className="flex w-full items-center px-1">
-                  <img
-                    src={Robot}
-                    alt="agent-logo"
-                    className="h-7 w-7 rounded-full"
-                  />
+                  <AgentImage className="h-7 w-7 rounded-full" />
                 </div>
                 <div className="mt-2">
                   <p
@@ -436,8 +432,8 @@ function AgentCard({
       </div>
       <div className="w-full">
         <div className="flex w-full items-center gap-1 px-1">
-          <img
-            src={agent.image && agent.image.trim() !== '' ? agent.image : Robot}
+          <AgentImage
+            src={agent.image}
             alt={`${agent.name}`}
             className="h-7 w-7 rounded-full object-contain"
           />

frontend/src/assets/check-circle-filled.svg

@@ -0,0 +1,3 @@
+<svg width="24" height="25" viewBox="0 0 24 25" fill="none" xmlns="http://www.w3.org/2000/svg">
+<path d="M12 2.5C17.523 2.5 22 6.977 22 12.5C22 18.023 17.523 22.5 12 22.5C6.477 22.5 2 18.023 2 12.5C2 6.977 6.477 2.5 12 2.5ZM15.22 9.47L10.75 13.94L8.78 11.97C8.63783 11.8375 8.44978 11.7654 8.25548 11.7688C8.06118 11.7723 7.87579 11.851 7.73838 11.9884C7.60097 12.1258 7.52225 12.3112 7.51883 12.5055C7.5154 12.6998 7.58752 12.8878 7.72 13.03L10.22 15.53C10.3606 15.6705 10.5512 15.7493 10.75 15.7493C10.9488 15.7493 11.1394 15.6705 11.28 15.53L16.28 10.53C16.3537 10.4613 16.4128 10.3785 16.4538 10.2865C16.4948 10.1945 16.5168 10.0952 16.5186 9.99452C16.5204 9.89382 16.5018 9.79379 16.4641 9.7004C16.4264 9.60701 16.3703 9.52218 16.299 9.45096C16.2278 9.37974 16.143 9.3236 16.0496 9.28588C15.9562 9.24816 15.8562 9.22963 15.7555 9.23141C15.6548 9.23318 15.5555 9.25523 15.4635 9.29622C15.3715 9.33721 15.2887 9.39631 15.22 9.47Z" fill="#0C9D35"/>
+</svg>

frontend/src/assets/warn.svg

@@ -0,0 +1,3 @@
+<svg width="20" height="21" viewBox="0 0 20 21" fill="none" xmlns="http://www.w3.org/2000/svg">
+<path d="M15 1.83989C16.5202 2.71758 17.7826 3.97997 18.6603 5.50017C19.538 7.02038 20 8.74483 20 10.5002C20 12.2556 19.5379 13.98 18.6602 15.5002C17.7825 17.0204 16.5201 18.2828 14.9999 19.1605C13.4797 20.0381 11.7552 20.5002 9.99984 20.5001C8.24446 20.5001 6.52002 20.038 4.99984 19.1603C3.47965 18.2826 2.21729 17.0202 1.33963 15.5C0.46198 13.9797 -4.45897e-05 12.2553 3.22765e-09 10.4999L0.00500012 10.1759C0.0610032 8.44888 0.563548 6.76585 1.46364 5.29089C2.36373 3.81592 3.63065 2.59934 5.14089 1.75977C6.65113 0.920205 8.35315 0.486289 10.081 0.50033C11.8089 0.514371 13.5036 0.97589 15 1.83989ZM10 13.4999C9.73478 13.4999 9.48043 13.6052 9.29289 13.7928C9.10536 13.9803 9 14.2347 9 14.4999V14.5099C9 14.7751 9.10536 15.0295 9.29289 15.217C9.48043 15.4045 9.73478 15.5099 10 15.5099C10.2652 15.5099 10.5196 15.4045 10.7071 15.217C10.8946 15.0295 11 14.7751 11 14.5099V14.4999C11 14.2347 10.8946 13.9803 10.7071 13.7928C10.5196 13.6052 10.2652 13.4999 10 13.4999ZM10 6.49989C9.73478 6.49989 9.48043 6.60525 9.29289 6.79279C9.10536 6.98032 9 7.23468 9 7.49989V11.4999C9 11.7651 9.10536 12.0195 9.29289 12.207C9.48043 12.3945 9.73478 12.4999 10 12.4999C10.2652 12.4999 10.5196 12.3945 10.7071 12.207C10.8946 12.0195 11 11.7651 11 11.4999V7.49989C11 7.23468 10.8946 6.98032 10.7071 6.79279C10.5196 6.60525 10.2652 6.49989 10 6.49989Z" fill="#EA4335"/>
+</svg>

frontend/src/components/ActionButtons.tsx

@@ -41,13 +41,13 @@ export default function ActionButtons({
     navigate('/');
   };
   return (
-    <div className="fixed right-4 top-0 z-10 flex h-16 flex-col justify-center">
+    <div className="fixed top-0 right-4 z-10 flex h-16 flex-col justify-center">
       <div className={`flex items-center gap-2 sm:gap-4 ${className}`}>
         {showNewChat && (
           <button
             title="Open New Chat"
             onClick={newChat}
-            className="flex items-center gap-1 rounded-full p-2 hover:bg-bright-gray dark:hover:bg-[#28292E] lg:hidden"
+            className="hover:bg-bright-gray flex items-center gap-1 rounded-full p-2 lg:hidden dark:hover:bg-[#28292E]"
           >
             <img
               className="filter dark:invert"
@@ -64,7 +64,7 @@ export default function ActionButtons({
             <button
               title="Share"
               onClick={() => setShareModalState(true)}
-              className="rounded-full p-2 hover:bg-bright-gray dark:hover:bg-[#28292E]"
+              className="hover:bg-bright-gray rounded-full p-2 dark:hover:bg-[#28292E]"
             >
               <img
                 className="filter dark:invert"

frontend/src/components/AgentImage.tsx

@@ -0,0 +1,40 @@
+import { useState, useEffect } from 'react';
+import Robot from '../assets/robot.svg';
+
+type AgentImageProps = {
+  src?: string | null;
+  alt?: string;
+  className?: string;
+  fallbackSrc?: string;
+};
+
+export default function AgentImage({
+  src,
+  alt = 'agent',
+  className = '',
+  fallbackSrc = Robot,
+}: AgentImageProps) {
+  const [currentSrc, setCurrentSrc] = useState(
+    src && src.trim() !== '' ? src : fallbackSrc,
+  );
+
+  useEffect(() => {
+    const newSrc = src && src.trim() !== '' ? src : fallbackSrc;
+    if (newSrc !== currentSrc) {
+      setCurrentSrc(newSrc);
+    }
+  }, [src, fallbackSrc]);
+
+  return (
+    <img
+      src={currentSrc}
+      alt={alt}
+      className={className}
+      referrerPolicy="no-referrer"
+      crossOrigin="anonymous"
+      onError={() => {
+        if (currentSrc !== fallbackSrc) setCurrentSrc(fallbackSrc);
+      }}
+    />
+  );
+}

frontend/src/components/Chunks.tsx

@@ -2,12 +2,16 @@ import React, { useState, useEffect, useRef } from 'react';
 import { useSelector } from 'react-redux';
 import { useTranslation } from 'react-i18next';
 import { selectToken } from '../preferences/preferenceSlice';
-import { useDarkTheme, useLoaderState, useMediaQuery, useOutsideAlerter } from '../hooks';
+import {
+  useDarkTheme,
+  useLoaderState,
+  useMediaQuery,
+  useOutsideAlerter,
+} from '../hooks';
 import userService from '../api/services/userService';
 import ArrowLeft from '../assets/arrow-left.svg';
 import NoFilesIcon from '../assets/no-files.svg';
 import NoFilesDarkIcon from '../assets/no-files-dark.svg';
-import OutlineSource from '../assets/outline-source.svg';
 import SkeletonLoader from './SkeletonLoader';
 import ConfirmationModal from '../modals/ConfirmationModal';
 import { ActiveState } from '../models/misc';
@@ -33,7 +37,7 @@ const LineNumberedTextarea: React.FC<LineNumberedTextareaProps> = ({
   ariaLabel,
   className = '',
   editable = true,
-  onDoubleClick
+  onDoubleClick,
 }) => {
   const { isMobile } = useMediaQuery();
 
@@ -45,28 +49,31 @@ const LineNumberedTextarea: React.FC<LineNumberedTextareaProps> = ({
   const contentLines = value.split('\n').length;
 
   const heightOffset = isMobile ? 200 : 300;
-  const minLinesForDisplay = Math.ceil((typeof window !== 'undefined' ? window.innerHeight - heightOffset : 600) / lineHeight);
+  const minLinesForDisplay = Math.ceil(
+    (typeof window !== 'undefined' ? window.innerHeight - heightOffset : 600) /
+      lineHeight,
+  );
   const totalLines = Math.max(contentLines, minLinesForDisplay);
 
   return (
     <div className={`relative w-full ${className}`}>
       <div
-        className="absolute left-0 top-0 w-8 lg:w-12 text-right text-gray-500 dark:text-gray-400 text-xs lg:text-sm font-mono leading-[19.93px] select-none pr-2 lg:pr-3 pointer-events-none"
+        className="pointer-events-none absolute top-0 left-0 w-8 pr-2 text-right font-mono text-xs leading-[19.93px] text-gray-500 select-none lg:w-12 lg:pr-3 lg:text-sm dark:text-gray-400"
         style={{
-          height: `${totalLines * lineHeight}px`
+          height: `${totalLines * lineHeight}px`,
         }}
       >
         {Array.from({ length: totalLines }, (_, i) => (
           <div
             key={i + 1}
-            className="flex items-center justify-end h-[19.93px] leading-[19.93px]"
+            className="flex h-[19.93px] items-center justify-end leading-[19.93px]"
           >
             {i + 1}
           </div>
         ))}
       </div>
       <textarea
-        className={`w-full resize-none bg-transparent dark:text-white font-['Inter'] text-[13.68px] leading-[19.93px] text-[#18181B] outline-none border-none pl-8 lg:pl-12 overflow-hidden ${isMobile ? 'min-h-[calc(100vh-200px)]' : 'min-h-[calc(100vh-300px)]'} ${!editable ? 'select-none' : ''}`}
+        className={`w-full resize-none overflow-hidden border-none bg-transparent pl-8 font-['Inter'] text-[13.68px] leading-[19.93px] text-[#18181B] outline-none lg:pl-12 dark:text-white ${isMobile ? 'min-h-[calc(100vh-200px)]' : 'min-h-[calc(100vh-300px)]'} ${!editable ? 'select-none' : ''}`}
         value={value}
         onChange={editable ? handleChange : undefined}
         onDoubleClick={onDoubleClick}
@@ -75,7 +82,7 @@ const LineNumberedTextarea: React.FC<LineNumberedTextareaProps> = ({
         rows={totalLines}
         readOnly={!editable}
         style={{
-          height: `${totalLines * lineHeight}px`
+          height: `${totalLines * lineHeight}px`,
         }}
       />
     </div>
@@ -105,7 +112,9 @@ const Chunks: React.FC<ChunksProps> = ({
   onFileSelect,
 }) => {
   const [fileSearchQuery, setFileSearchQuery] = useState('');
-  const [fileSearchResults, setFileSearchResults] = useState<SearchResult[]>([]);
+  const [fileSearchResults, setFileSearchResults] = useState<SearchResult[]>(
+    [],
+  );
   const searchDropdownRef = useRef<HTMLDivElement>(null);
   const { t } = useTranslation();
   const token = useSelector(selectToken);
@@ -120,7 +129,8 @@ const Chunks: React.FC<ChunksProps> = ({
   const [editingTitle, setEditingTitle] = useState('');
   const [editingText, setEditingText] = useState('');
   const [isAddingChunk, setIsAddingChunk] = useState(false);
-  const [deleteModalState, setDeleteModalState] = useState<ActiveState>('INACTIVE');
+  const [deleteModalState, setDeleteModalState] =
+    useState<ActiveState>('INACTIVE');
   const [chunkToDelete, setChunkToDelete] = useState<ChunkType | null>(null);
   const [isEditing, setIsEditing] = useState(false);
 
@@ -189,7 +199,6 @@ const Chunks: React.FC<ChunksProps> = ({
   };
 
   const handleUpdateChunk = (title: string, text: string, chunk: ChunkType) => {
-
     if (!text.trim()) {
       return;
     }
@@ -274,7 +283,7 @@ const Chunks: React.FC<ChunksProps> = ({
   useEffect(() => {
     !loading && fetchChunks();
   }, [page, perPage, path]);
-  
+
   useEffect(() => {
     setSearchTerm('');
     setPage(1);
@@ -284,35 +293,45 @@ const Chunks: React.FC<ChunksProps> = ({
 
   const renderPathNavigation = () => {
     return (
-      <div className="mb-0 min-h-[38px] flex flex-col sm:flex-row sm:items-center sm:justify-between text-base gap-2">
+      <div className="mb-0 flex min-h-[38px] flex-col gap-2 text-base sm:flex-row sm:items-center sm:justify-between">
         <div className="flex w-full items-center sm:w-auto">
           <button
-            className="mr-3 flex h-[29px] w-[29px] items-center justify-center rounded-full border p-2 text-sm text-gray-400 dark:border-0 dark:bg-[#28292D] dark:text-gray-500 dark:hover:bg-[#2E2F34] transition-all duration-200 font-medium"
-            onClick={editingChunk ? () => setEditingChunk(null) : isAddingChunk ? () => setIsAddingChunk(false) : handleGoBack}
+            className="mr-3 flex h-[29px] w-[29px] items-center justify-center rounded-full border p-2 text-sm font-medium text-gray-400 transition-all duration-200 dark:border-0 dark:bg-[#28292D] dark:text-gray-500 dark:hover:bg-[#2E2F34]"
+            onClick={
+              editingChunk
+                ? () => setEditingChunk(null)
+                : isAddingChunk
+                  ? () => setIsAddingChunk(false)
+                  : handleGoBack
+            }
           >
             <img src={ArrowLeft} alt="left-arrow" className="h-3 w-3" />
           </button>
 
-          <div className="flex items-center flex-wrap">
+          <div className="flex flex-wrap items-center">
             {/* Removed the directory icon */}
-            <span className="text-[#7D54D1] font-semibold break-words">
+            <span className="font-semibold break-words text-[#7D54D1]">
               {documentName}
             </span>
 
             {pathParts.length > 0 && (
               <>
-                <span className="mx-1 text-gray-500 flex-shrink-0">/</span>
+                <span className="mx-1 flex-shrink-0 text-gray-500">/</span>
                 {pathParts.map((part, index) => (
                   <React.Fragment key={index}>
-                    <span className={`break-words ${
-                      index < pathParts.length - 1 
-                        ? 'text-[#7D54D1] font-medium' 
-                        : 'text-gray-700 dark:text-gray-300'
-                    }`}>
+                    <span
+                      className={`break-words ${
+                        index < pathParts.length - 1
+                          ? 'font-medium text-[#7D54D1]'
+                          : 'text-gray-700 dark:text-gray-300'
+                      }`}
+                    >
                       {part}
                     </span>
                     {index < pathParts.length - 1 && (
-                      <span className="mx-1 text-gray-500 flex-shrink-0">/</span>
+                      <span className="mx-1 flex-shrink-0 text-gray-500">
+                        /
+                      </span>
                     )}
                   </React.Fragment>
                 ))}
@@ -321,18 +340,18 @@ const Chunks: React.FC<ChunksProps> = ({
           </div>
         </div>
 
-        <div className="flex flex-row flex-nowrap items-center gap-2 w-full sm:w-auto justify-end mt-2 sm:mt-0 overflow-x-auto">
+        <div className="mt-2 flex w-full flex-row flex-nowrap items-center justify-end gap-2 overflow-x-auto sm:mt-0 sm:w-auto">
           {editingChunk ? (
             !isEditing ? (
               <>
                 <button
-                  className="bg-purple-30 hover:bg-violets-are-blue flex h-[38px] min-w-[108px] items-center justify-center rounded-full px-4 text-[14px] whitespace-nowrap text-white font-medium"
+                  className="bg-purple-30 hover:bg-violets-are-blue flex h-[38px] min-w-[108px] items-center justify-center rounded-full px-4 text-[14px] font-medium whitespace-nowrap text-white"
                   onClick={() => setIsEditing(true)}
                 >
                   {t('modals.chunk.edit')}
                 </button>
                 <button
-                  className="rounded-full border border-solid border-red-500 px-4 py-1 text-[14px] text-nowrap text-red-500 hover:bg-red-500 hover:text-white h-[38px] min-w-[108px] flex items-center justify-center font-medium"
+                  className="flex h-[38px] min-w-[108px] items-center justify-center rounded-full border border-solid border-red-500 px-4 py-1 text-[14px] font-medium text-nowrap text-red-500 hover:bg-red-500 hover:text-white"
                   onClick={() => {
                     confirmDeleteChunk(editingChunk);
                   }}
@@ -346,28 +365,40 @@ const Chunks: React.FC<ChunksProps> = ({
                   onClick={() => {
                     setIsEditing(false);
                   }}
-                  className="dark:text-light-gray cursor-pointer rounded-full px-4 py-1 text-sm font-medium hover:bg-gray-100 dark:bg-transparent dark:hover:bg-[#767183]/50 text-nowrap h-[38px] min-w-[108px] flex items-center justify-center"
+                  className="dark:text-light-gray flex h-[38px] min-w-[108px] cursor-pointer items-center justify-center rounded-full px-4 py-1 text-sm font-medium text-nowrap hover:bg-gray-100 dark:bg-transparent dark:hover:bg-[#767183]/50"
                 >
                   {t('modals.chunk.cancel')}
                 </button>
                 <button
                   onClick={() => {
                     if (editingText.trim()) {
-                      const hasChanges = editingTitle !== (editingChunk?.metadata?.title || '') ||
-                                       editingText !== (editingChunk?.text || '');
+                      const hasChanges =
+                        editingTitle !==
+                          (editingChunk?.metadata?.title || '') ||
+                        editingText !== (editingChunk?.text || '');
 
                       if (hasChanges) {
-                        handleUpdateChunk(editingTitle, editingText, editingChunk);
+                        handleUpdateChunk(
+                          editingTitle,
+                          editingText,
+                          editingChunk,
+                        );
                       }
                       setIsEditing(false);
                       setEditingChunk(null);
                     }
                   }}
-                  disabled={!editingText.trim() || (editingTitle === (editingChunk?.metadata?.title || '') && editingText === (editingChunk?.text || ''))}
-                  className={`text-nowrap rounded-full px-4 py-1 text-[14px] text-white transition-all flex items-center justify-center h-[38px] min-w-[108px] font-medium ${
-                    editingText.trim() && (editingTitle !== (editingChunk?.metadata?.title || '') || editingText !== (editingChunk?.text || ''))
+                  disabled={
+                    !editingText.trim() ||
+                    (editingTitle === (editingChunk?.metadata?.title || '') &&
+                      editingText === (editingChunk?.text || ''))
+                  }
+                  className={`flex h-[38px] min-w-[108px] items-center justify-center rounded-full px-4 py-1 text-[14px] font-medium text-nowrap text-white transition-all ${
+                    editingText.trim() &&
+                    (editingTitle !== (editingChunk?.metadata?.title || '') ||
+                      editingText !== (editingChunk?.text || ''))
                       ? 'bg-purple-30 hover:bg-violets-are-blue cursor-pointer'
-                      : 'bg-gray-400 cursor-not-allowed'
+                      : 'cursor-not-allowed bg-gray-400'
                   }`}
                 >
                   {t('modals.chunk.save')}
@@ -378,7 +409,7 @@ const Chunks: React.FC<ChunksProps> = ({
             <>
               <button
                 onClick={() => setIsAddingChunk(false)}
-                className="dark:text-light-gray cursor-pointer rounded-full px-4 py-1 text-sm font-medium hover:bg-gray-100 dark:bg-transparent dark:hover:bg-[#767183]/50 text-nowrap h-[38px] min-w-[108px] flex items-center justify-center"
+                className="dark:text-light-gray flex h-[38px] min-w-[108px] cursor-pointer items-center justify-center rounded-full px-4 py-1 text-sm font-medium text-nowrap hover:bg-gray-100 dark:bg-transparent dark:hover:bg-[#767183]/50"
               >
                 {t('modals.chunk.cancel')}
               </button>
@@ -390,10 +421,10 @@ const Chunks: React.FC<ChunksProps> = ({
                   }
                 }}
                 disabled={!editingText.trim()}
-                className={`text-nowrap rounded-full px-4 py-1 text-[14px] text-white transition-all flex items-center justify-center h-[38px] min-w-[108px] font-medium ${
+                className={`flex h-[38px] min-w-[108px] items-center justify-center rounded-full px-4 py-1 text-[14px] font-medium text-nowrap text-white transition-all ${
                   editingText.trim()
                     ? 'bg-purple-30 hover:bg-violets-are-blue cursor-pointer'
-                    : 'bg-gray-400 cursor-not-allowed'
+                    : 'cursor-not-allowed bg-gray-400'
                 }`}
               >
                 {t('modals.chunk.add')}
@@ -437,33 +468,30 @@ const Chunks: React.FC<ChunksProps> = ({
       setFileSearchResults([]);
     },
     [], // No additional dependencies
-    false // Don't handle escape key
+    false, // Don't handle escape key
   );
 
   const renderFileSearch = () => {
     return (
       <div className="relative" ref={searchDropdownRef}>
         <div className="relative flex items-center">
-          <div className="absolute left-3 pointer-events-none">
-            <img src={SearchIcon} alt="Search" className="w-4 h-4" />
+          <div className="pointer-events-none absolute left-3">
+            <img src={SearchIcon} alt="Search" className="h-4 w-4" />
           </div>
           <input
             type="text"
             value={fileSearchQuery}
             onChange={(e) => handleFileSearchChange(e.target.value)}
             placeholder={t('settings.sources.searchFiles')}
-            className={`w-full h-[38px] border border-[#D1D9E0] pl-10 pr-4 py-2 dark:border-[#6A6A6A] 
-              ${fileSearchQuery 
-                ? 'rounded-t-[6px]' 
-                : 'rounded-[6px]'
-              } 
-              bg-transparent focus:outline-none dark:text-[#E0E0E0] transition-all duration-200`}
+            className={`h-[38px] w-full border border-[#D1D9E0] py-2 pr-4 pl-10 dark:border-[#6A6A6A] ${
+              fileSearchQuery ? 'rounded-t-[6px]' : 'rounded-[6px]'
+            } bg-transparent transition-all duration-200 focus:outline-none dark:text-[#E0E0E0]`}
           />
         </div>
 
         {fileSearchQuery && (
           <div className="absolute z-10 max-h-[calc(100vh-200px)] w-full overflow-hidden rounded-b-[6px] border border-t-0 border-[#D1D9E0] bg-white shadow-lg dark:border-[#6A6A6A] dark:bg-[#1F2023]">
-            <div className="max-h-[calc(100vh-200px)] overflow-y-auto overflow-x-hidden">
+            <div className="max-h-[calc(100vh-200px)] overflow-x-hidden overflow-y-auto">
               {fileSearchResults.length === 0 ? (
                 <div className="py-2 text-center text-sm text-gray-500 dark:text-gray-400">
                   {t('settings.sources.noResults')}
@@ -485,7 +513,7 @@ const Chunks: React.FC<ChunksProps> = ({
                       alt={result.isFile ? 'File' : 'Folder'}
                       className="mr-2 h-4 w-4 flex-shrink-0"
                     />
-                    <span className="text-sm dark:text-[#E0E0E0] truncate">
+                    <span className="truncate text-sm dark:text-[#E0E0E0]">
                       {result.path.split('/').pop() || result.path}
                     </span>
                   </div>
@@ -500,42 +528,39 @@ const Chunks: React.FC<ChunksProps> = ({
 
   return (
     <div className="flex flex-col">
-      <div className="mb-2">
-        {renderPathNavigation()}
-      </div>
+      <div className="mb-2">{renderPathNavigation()}</div>
       <div className="flex gap-4">
         {onFileSearch && onFileSelect && (
-          <div className="hidden lg:block w-[198px]">
-            {renderFileSearch()}
-          </div>
+          <div className="hidden w-[198px] lg:block">{renderFileSearch()}</div>
         )}
 
         {/* Right side: Chunks content */}
         <div className="flex-1">
           {!editingChunk && !isAddingChunk ? (
             <>
-              <div className="mb-3 flex flex-col sm:flex-row items-start sm:items-center justify-between gap-3">
-                <div className="flex-1 w-full flex items-center border border-[#D1D9E0] dark:border-[#6A6A6A] rounded-md overflow-hidden h-[38px]">
-                  <div className="px-4 flex items-center text-gray-700 dark:text-[#E0E0E0] font-medium whitespace-nowrap h-full">
+              <div className="mb-3 flex flex-col items-start justify-between gap-3 sm:flex-row sm:items-center">
+                <div className="flex h-[38px] w-full flex-1 items-center overflow-hidden rounded-md border border-[#D1D9E0] dark:border-[#6A6A6A]">
+                  <div className="flex h-full items-center px-4 font-medium whitespace-nowrap text-gray-700 dark:text-[#E0E0E0]">
                     {totalChunks > 999999
                       ? `${(totalChunks / 1000000).toFixed(2)}M`
                       : totalChunks > 999
                         ? `${(totalChunks / 1000).toFixed(2)}K`
-                        : totalChunks} {t('settings.sources.chunks')}
+                        : totalChunks}{' '}
+                    {t('settings.sources.chunks')}
                   </div>
                   <div className="h-full w-[1px] bg-[#D1D9E0] dark:bg-[#6A6A6A]"></div>
-                  <div className="flex-1 h-full">
+                  <div className="h-full flex-1">
                     <input
                       type="text"
                       placeholder={t('settings.sources.searchPlaceholder')}
                       value={searchTerm}
                       onChange={(e) => setSearchTerm(e.target.value)}
-                      className="w-full h-full px-3 py-2 bg-transparent border-none outline-none font-normal text-[13.56px] leading-[100%] dark:text-[#E0E0E0]"
+                      className="h-full w-full border-none bg-transparent px-3 py-2 text-[13.56px] leading-[100%] font-normal outline-none dark:text-[#E0E0E0]"
                     />
                   </div>
                 </div>
                 <button
-                  className="bg-purple-30 hover:bg-violets-are-blue flex h-[38px] w-full sm:w-auto min-w-[108px] items-center justify-center rounded-full px-4 text-[14px] whitespace-normal text-white shrink-0 font-medium"
+                  className="bg-purple-30 hover:bg-violets-are-blue flex h-[38px] w-full min-w-[108px] shrink-0 items-center justify-center rounded-full px-4 text-[14px] font-medium whitespace-normal text-white sm:w-auto"
                   title={t('settings.sources.addChunk')}
                   onClick={() => {
                     setIsAddingChunk(true);
@@ -547,13 +572,13 @@ const Chunks: React.FC<ChunksProps> = ({
                 </button>
               </div>
               {loading ? (
-                <div className="w-full grid grid-cols-1 sm:[grid-template-columns:repeat(auto-fit,minmax(400px,1fr))] gap-4 justify-items-start">
+                <div className="grid w-full grid-cols-1 justify-items-start gap-4 sm:[grid-template-columns:repeat(auto-fit,minmax(400px,1fr))]">
                   <SkeletonLoader component="chunkCards" count={perPage} />
                 </div>
               ) : (
-                <div className="w-full grid grid-cols-1 sm:[grid-template-columns:repeat(auto-fit,minmax(400px,1fr))] gap-4 justify-items-start">
+                <div className="grid w-full grid-cols-1 justify-items-start gap-4 sm:[grid-template-columns:repeat(auto-fit,minmax(400px,1fr))]">
                   {filteredChunks.length === 0 ? (
-                    <div className="col-span-full w-full min-h-[50vh] flex flex-col items-center justify-center text-center text-gray-500 dark:text-gray-400">
+                    <div className="col-span-full flex min-h-[50vh] w-full flex-col items-center justify-center text-center text-gray-500 dark:text-gray-400">
                       <img
                         src={isDarkTheme ? NoFilesDarkIcon : NoFilesIcon}
                         alt={t('settings.sources.noChunksAlt')}
@@ -565,7 +590,7 @@ const Chunks: React.FC<ChunksProps> = ({
                     filteredChunks.map((chunk, index) => (
                       <div
                         key={index}
-                        className="transform transition-transform duration-200 hover:scale-105 relative flex h-[197px] flex-col justify-between rounded-[5.86px] border border-[#D1D9E0] dark:border-[#6A6A6A] overflow-hidden cursor-pointer w-full max-w-[487px]"
+                        className="relative flex h-[197px] w-full max-w-[487px] transform cursor-pointer flex-col justify-between overflow-hidden rounded-[5.86px] border border-[#D1D9E0] transition-transform duration-200 hover:scale-105 dark:border-[#6A6A6A]"
                         onClick={() => {
                           setEditingChunk(chunk);
                           setEditingTitle(chunk.metadata?.title || '');
@@ -573,13 +598,16 @@ const Chunks: React.FC<ChunksProps> = ({
                         }}
                       >
                         <div className="w-full">
-                          <div className="flex w-full items-center justify-between border-b border-[#D1D9E0] bg-[#F6F8FA] dark:bg-[#27282D] dark:border-[#6A6A6A] px-4 py-3">
-                            <div className="text-[#59636E] text-sm dark:text-[#E0E0E0]">
-                              {chunk.metadata.token_count ? chunk.metadata.token_count.toLocaleString() : '-'} {t('settings.sources.tokensUnit')}
+                          <div className="flex w-full items-center justify-between border-b border-[#D1D9E0] bg-[#F6F8FA] px-4 py-3 dark:border-[#6A6A6A] dark:bg-[#27282D]">
+                            <div className="text-sm text-[#59636E] dark:text-[#E0E0E0]">
+                              {chunk.metadata.token_count
+                                ? chunk.metadata.token_count.toLocaleString()
+                                : '-'}{' '}
+                              {t('settings.sources.tokensUnit')}
                             </div>
                           </div>
                           <div className="px-4 pt-3 pb-6">
-                            <p className="font-['Inter'] text-[13.68px] leading-[19.93px] text-[#18181B] dark:text-[#E0E0E0] line-clamp-6 font-normal">
+                            <p className="line-clamp-6 font-['Inter'] text-[13.68px] leading-[19.93px] font-normal text-[#18181B] dark:text-[#E0E0E0]">
                               {chunk.text}
                             </p>
                           </div>
@@ -592,7 +620,7 @@ const Chunks: React.FC<ChunksProps> = ({
             </>
           ) : isAddingChunk ? (
             <div className="w-full">
-              <div className="relative border border-[#D1D9E0] dark:border-[#6A6A6A] rounded-lg overflow-hidden">
+              <div className="relative overflow-hidden rounded-lg border border-[#D1D9E0] dark:border-[#6A6A6A]">
                 <LineNumberedTextarea
                   value={editingText}
                   onChange={setEditingText}
@@ -601,45 +629,53 @@ const Chunks: React.FC<ChunksProps> = ({
                 />
               </div>
             </div>
-          ) : editingChunk && (
-            <div className="w-full">
-              <div className="relative flex flex-col rounded-[5.86px] border border-[#D1D9E0] dark:border-[#6A6A6A] overflow-hidden w-full">
-                <div className="flex w-full items-center justify-between border-b border-[#D1D9E0] bg-[#F6F8FA] dark:bg-[#27282D] dark:border-[#6A6A6A] px-4 py-3">
-                  <div className="text-[#59636E] text-sm dark:text-[#E0E0E0]">
-                    {editingChunk.metadata.token_count ? editingChunk.metadata.token_count.toLocaleString() : '-'} {t('settings.sources.tokensUnit')}
+          ) : (
+            editingChunk && (
+              <div className="w-full">
+                <div className="relative flex w-full flex-col overflow-hidden rounded-[5.86px] border border-[#D1D9E0] dark:border-[#6A6A6A]">
+                  <div className="flex w-full items-center justify-between border-b border-[#D1D9E0] bg-[#F6F8FA] px-4 py-3 dark:border-[#6A6A6A] dark:bg-[#27282D]">
+                    <div className="text-sm text-[#59636E] dark:text-[#E0E0E0]">
+                      {editingChunk.metadata.token_count
+                        ? editingChunk.metadata.token_count.toLocaleString()
+                        : '-'}{' '}
+                      {t('settings.sources.tokensUnit')}
+                    </div>
+                  </div>
+                  <div className="overflow-hidden p-4">
+                    <LineNumberedTextarea
+                      value={isEditing ? editingText : editingChunk.text}
+                      onChange={setEditingText}
+                      ariaLabel={t('modals.chunk.promptText')}
+                      editable={isEditing}
+                      onDoubleClick={() => {
+                        if (!isEditing) {
+                          setIsEditing(true);
+                          setEditingTitle(editingChunk.metadata.title || '');
+                          setEditingText(editingChunk.text);
+                        }
+                      }}
+                    />
                   </div>
                 </div>
-                <div className="p-4 overflow-hidden">
-                  <LineNumberedTextarea
-                    value={isEditing ? editingText : editingChunk.text}
-                    onChange={setEditingText}
-                    ariaLabel={t('modals.chunk.promptText')}
-                    editable={isEditing}
-                    onDoubleClick={() => {
-                      if (!isEditing) {
-                        setIsEditing(true);
-                        setEditingTitle(editingChunk.metadata.title || '');
-                        setEditingText(editingChunk.text);
-                      }
-                    }}
-                  />
-                </div>
               </div>
-            </div>
+            )
           )}
 
-          {!loading && totalChunks > perPage && !editingChunk && !isAddingChunk && (
-            <Pagination
-              currentPage={page}
-              totalPages={Math.ceil(totalChunks / perPage)}
-              rowsPerPage={perPage}
-              onPageChange={setPage}
-              onRowsPerPageChange={(rows) => {
-                setPerPage(rows);
-                setPage(1);
-              }}
-            />
-          )}
+          {!loading &&
+            totalChunks > perPage &&
+            !editingChunk &&
+            !isAddingChunk && (
+              <Pagination
+                currentPage={page}
+                totalPages={Math.ceil(totalChunks / perPage)}
+                rowsPerPage={perPage}
+                onPageChange={setPage}
+                onRowsPerPageChange={(rows) => {
+                  setPerPage(rows);
+                  setPage(1);
+                }}
+              />
+            )}
         </div>
       </div>
 

frontend/src/components/ConnectorAuth.tsx

@@ -1,5 +1,6 @@
 import React, { useRef } from 'react';
 import { useSelector } from 'react-redux';
+import { useTranslation } from 'react-i18next';
 import { useDarkTheme } from '../hooks';
 import { selectToken } from '../preferences/preferenceSlice';
 
@@ -24,6 +25,7 @@ const ConnectorAuth: React.FC<ConnectorAuthProps> = ({
   onDisconnect,
   errorMessage,
 }) => {
+  const { t } = useTranslation();
   const token = useSelector(selectToken);
   const [isDarkTheme] = useDarkTheme();
   const completedRef = useRef(false);
@@ -47,12 +49,16 @@ const ConnectorAuth: React.FC<ConnectorAuthProps> = ({
       cleanup();
       onSuccess({
         session_token: event.data.session_token,
-        user_email: event.data.user_email || 'Connected User',
+        user_email:
+          event.data.user_email ||
+          t('modals.uploadDoc.connectors.auth.connectedUser'),
       });
     } else if (errorProvider) {
       completedRef.current = true;
       cleanup();
-      onError(event.data.error || 'Authentication failed');
+      onError(
+        event.data.error || t('modals.uploadDoc.connectors.auth.authFailed'),
+      );
     }
   };
 
@@ -71,13 +77,15 @@ const ConnectorAuth: React.FC<ConnectorAuthProps> = ({
 
       if (!authResponse.ok) {
         throw new Error(
-          `Failed to get authorization URL: ${authResponse.status}`,
+          `${t('modals.uploadDoc.connectors.auth.authUrlFailed')}: ${authResponse.status}`,
         );
       }
 
       const authData = await authResponse.json();
       if (!authData.success || !authData.authorization_url) {
-        throw new Error(authData.error || 'Failed to get authorization URL');
+        throw new Error(
+          authData.error || t('modals.uploadDoc.connectors.auth.authUrlFailed'),
+        );
       }
 
       const authWindow = window.open(
@@ -86,9 +94,7 @@ const ConnectorAuth: React.FC<ConnectorAuthProps> = ({
         'width=500,height=600,scrollbars=yes,resizable=yes',
       );
       if (!authWindow) {
-        throw new Error(
-          'Failed to open authentication window. Please allow popups.',
-        );
+        throw new Error(t('modals.uploadDoc.connectors.auth.popupBlocked'));
       }
 
       window.addEventListener('message', handleAuthMessage as any);
@@ -98,28 +104,44 @@ const ConnectorAuth: React.FC<ConnectorAuthProps> = ({
           clearInterval(checkClosed);
           window.removeEventListener('message', handleAuthMessage as any);
           if (!completedRef.current) {
-            onError('Authentication was cancelled');
+            onError(t('modals.uploadDoc.connectors.auth.authCancelled'));
           }
         }
       }, 1000);
       intervalRef.current = checkClosed;
     } catch (error) {
-      onError(error instanceof Error ? error.message : 'Authentication failed');
+      onError(
+        error instanceof Error
+          ? error.message
+          : t('modals.uploadDoc.connectors.auth.authFailed'),
+      );
     }
   };
 
   return (
     <>
       {errorMessage && (
-        <div className="mb-4 flex items-center gap-2 rounded-lg border border-[#E60000] dark:border-[#D42626] bg-transparent dark:bg-[#D426261A] p-2">
-          <svg width="30" height="30" viewBox="0 0 30 30" fill="none" xmlns="http://www.w3.org/2000/svg">
-            <path d="M7.09974 24.5422H22.9C24.5156 24.5422 25.5228 22.7901 24.715 21.3947L16.8149 7.74526C16.007 6.34989 13.9927 6.34989 13.1848 7.74526L5.28471 21.3947C4.47686 22.7901 5.48405 24.5422 7.09974 24.5422ZM14.9998 17.1981C14.4228 17.1981 13.9507 16.726 13.9507 16.149V14.0507C13.9507 13.4736 14.4228 13.0015 14.9998 13.0015C15.5769 13.0015 16.049 13.4736 16.049 14.0507V16.149C16.049 16.726 15.5769 17.1981 14.9998 17.1981ZM16.049 21.3947H13.9507V19.2964H16.049V21.3947Z" fill={isDarkTheme ? '#EECF56' : '#E60000'} />
+        <div className="mb-4 flex items-center gap-2 rounded-lg border border-[#E60000] bg-transparent p-2 dark:border-[#D42626] dark:bg-[#D426261A]">
+          <svg
+            width="30"
+            height="30"
+            viewBox="0 0 30 30"
+            fill="none"
+            xmlns="http://www.w3.org/2000/svg"
+          >
+            <path
+              d="M7.09974 24.5422H22.9C24.5156 24.5422 25.5228 22.7901 24.715 21.3947L16.8149 7.74526C16.007 6.34989 13.9927 6.34989 13.1848 7.74526L5.28471 21.3947C4.47686 22.7901 5.48405 24.5422 7.09974 24.5422ZM14.9998 17.1981C14.4228 17.1981 13.9507 16.726 13.9507 16.149V14.0507C13.9507 13.4736 14.4228 13.0015 14.9998 13.0015C15.5769 13.0015 16.049 13.4736 16.049 14.0507V16.149C16.049 16.726 15.5769 17.1981 14.9998 17.1981ZM16.049 21.3947H13.9507V19.2964H16.049V21.3947Z"
+              fill={isDarkTheme ? '#EECF56' : '#E60000'}
+            />
           </svg>
 
-          <span className='text-[#E60000] dark:text-[#E37064] text-sm' style={{
-            fontFamily: 'Inter',
-            lineHeight: '100%'
-          }}>
+          <span
+            className="text-sm text-[#E60000] dark:text-[#E37064]"
+            style={{
+              fontFamily: 'Inter',
+              lineHeight: '100%',
+            }}
+          >
             {errorMessage}
           </span>
         </div>
@@ -127,19 +149,26 @@ const ConnectorAuth: React.FC<ConnectorAuthProps> = ({
 
       {isConnected ? (
         <div className="mb-4">
-          <div className="w-full flex items-center justify-between rounded-[10px] bg-[#8FDD51] px-4 py-2 text-[#212121] font-medium text-sm">
+          <div className="flex w-full items-center justify-between rounded-[10px] bg-[#8FDD51] px-4 py-2 text-sm font-medium text-[#212121]">
             <div className="flex items-center gap-2">
               <svg className="h-4 w-4" viewBox="0 0 24 24">
-                <path fill="currentColor" d="M9 16.17L4.83 12l-1.42 1.41L9 19 21 7l-1.41-1.41z" />
+                <path
+                  fill="currentColor"
+                  d="M9 16.17L4.83 12l-1.42 1.41L9 19 21 7l-1.41-1.41z"
+                />
               </svg>
-              <span>Connected as {userEmail}</span>
+              <span>
+                {t('modals.uploadDoc.connectors.auth.connectedAs', {
+                  email: userEmail,
+                })}
+              </span>
             </div>
             {onDisconnect && (
               <button
                 onClick={onDisconnect}
-                className="text-[#212121] hover:text-gray-700 font-medium text-xs underline"
+                className="text-xs font-medium text-[#212121] underline hover:text-gray-700"
               >
-                Disconnect
+                {t('modals.uploadDoc.connectors.auth.disconnect')}
               </button>
             )}
           </div>
@@ -162,4 +191,4 @@ const ConnectorAuth: React.FC<ConnectorAuthProps> = ({
   );
 };
 
-export default ConnectorAuth;
+export default ConnectorAuth;

frontend/src/components/ConnectorTreeComponent.tsx

@@ -76,7 +76,8 @@ const ConnectorTreeComponent: React.FC<ConnectorTreeComponentProps> = ({
   const [syncProgress, setSyncProgress] = useState<number>(0);
   const [sourceProvider, setSourceProvider] = useState<string>('');
   const [syncDone, setSyncDone] = useState<boolean>(false);
-  const [syncConfirmationModal, setSyncConfirmationModal] = useState<ActiveState>('INACTIVE');
+  const [syncConfirmationModal, setSyncConfirmationModal] =
+    useState<ActiveState>('INACTIVE');
 
   useOutsideAlerter(
     searchDropdownRef,
@@ -392,31 +393,26 @@ const ConnectorTreeComponent: React.FC<ConnectorTreeComponentProps> = ({
     const parentRow =
       currentPath.length > 0
         ? [
-          <TableRow
-            key="parent-dir"
-            onClick={navigateUp}
-          >
-            <TableCell width="40%" align="left">
-              <div className="flex items-center">
-                <img
-                  src={FolderIcon}
-                  alt={t('settings.sources.parentFolderAlt')}
-                  className="mr-2 h-4 w-4 flex-shrink-0"
-                />
-                <span className="truncate">
-                  ..
-                </span>
-              </div>
-            </TableCell>
-            <TableCell width="30%" align="left">
-              -
-            </TableCell>
-            <TableCell width="20%" align="left">
-              -
-            </TableCell>
-            <TableCell width="10%" align="right"></TableCell>
-          </TableRow>,
-        ]
+            <TableRow key="parent-dir" onClick={navigateUp}>
+              <TableCell width="40%" align="left">
+                <div className="flex items-center">
+                  <img
+                    src={FolderIcon}
+                    alt={t('settings.sources.parentFolderAlt')}
+                    className="mr-2 h-4 w-4 flex-shrink-0"
+                  />
+                  <span className="truncate">..</span>
+                </div>
+              </TableCell>
+              <TableCell width="30%" align="left">
+                -
+              </TableCell>
+              <TableCell width="20%" align="left">
+                -
+              </TableCell>
+              <TableCell width="10%" align="right"></TableCell>
+            </TableRow>,
+          ]
         : [];
 
     // Sort entries: directories first, then files, both alphabetically
@@ -444,10 +440,7 @@ const ConnectorTreeComponent: React.FC<ConnectorTreeComponentProps> = ({
         const dirStats = calculateDirectoryStats(node as DirectoryStructure);
 
         return (
-          <TableRow
-            key={itemId}
-            onClick={() => navigateToDirectory(name)}
-          >
+          <TableRow key={itemId} onClick={() => navigateToDirectory(name)}>
             <TableCell width="40%" align="left">
               <div className="flex min-w-0 items-center">
                 <img
@@ -455,9 +448,7 @@ const ConnectorTreeComponent: React.FC<ConnectorTreeComponentProps> = ({
                   alt={t('settings.sources.folderAlt')}
                   className="mr-2 h-4 w-4 flex-shrink-0"
                 />
-                <span className="truncate">
-                  {name}
-                </span>
+                <span className="truncate">{name}</span>
               </div>
             </TableCell>
             <TableCell width="30%" align="left">
@@ -472,7 +463,7 @@ const ConnectorTreeComponent: React.FC<ConnectorTreeComponentProps> = ({
               <div ref={menuRef} className="relative">
                 <button
                   onClick={(e) => handleMenuClick(e, itemId)}
-                  className="inline-flex h-[35px] w-[24px] shrink-0 items-center justify-center rounded-md transition-colors hover:bg-[#EBEBEB] dark:hover:bg-[#26272E] font-medium"
+                  className="inline-flex h-[35px] w-[24px] shrink-0 items-center justify-center rounded-md font-medium transition-colors hover:bg-[#EBEBEB] dark:hover:bg-[#26272E]"
                   aria-label={t('settings.sources.menuAlt')}
                 >
                   <img
@@ -505,10 +496,7 @@ const ConnectorTreeComponent: React.FC<ConnectorTreeComponentProps> = ({
         const menuRef = getMenuRef(itemId);
 
         return (
-          <TableRow
-            key={itemId}
-            onClick={() => handleFileClick(name)}
-          >
+          <TableRow key={itemId} onClick={() => handleFileClick(name)}>
             <TableCell width="40%" align="left">
               <div className="flex min-w-0 items-center">
                 <img
@@ -516,9 +504,7 @@ const ConnectorTreeComponent: React.FC<ConnectorTreeComponentProps> = ({
                   alt={t('settings.sources.fileAlt')}
                   className="mr-2 h-4 w-4 flex-shrink-0"
                 />
-                <span className="truncate">
-                  {name}
-                </span>
+                <span className="truncate">{name}</span>
               </div>
             </TableCell>
             <TableCell width="30%" align="left">
@@ -730,9 +716,7 @@ const ConnectorTreeComponent: React.FC<ConnectorTreeComponentProps> = ({
                     </TableHeader>
                   </TableRow>
                 </TableHead>
-                <TableBody>
-                  {renderFileTree(getCurrentDirectory())}
-                </TableBody>
+                <TableBody>{renderFileTree(getCurrentDirectory())}</TableBody>
               </Table>
             </TableContainer>
           </div>

frontend/src/components/CopyButton.tsx

@@ -8,10 +8,6 @@ import CopyIcon from '../assets/copy.svg?react';
 
 type CopyButtonProps = {
   textToCopy: string;
-  bgColorLight?: string;
-  bgColorDark?: string;
-  hoverBgColorLight?: string;
-  hoverBgColorDark?: string;
   iconSize?: string;
   padding?: string;
   showText?: boolean;
@@ -27,14 +23,11 @@ const DEFAULT_COPIED_DURATION = 2000;
 const DEFAULT_BG_LIGHT = '#FFFFFF';
 const DEFAULT_BG_DARK = 'transparent';
 const DEFAULT_HOVER_BG_LIGHT = '#EEEEEE';
-const DEFAULT_HOVER_BG_DARK = '#4A4A4A';
+const DEFAULT_HOVER_BG_DARK = '#464152';
 
 export default function CopyButton({
   textToCopy,
-  bgColorLight = DEFAULT_BG_LIGHT,
-  bgColorDark = DEFAULT_BG_DARK,
-  hoverBgColorLight = DEFAULT_HOVER_BG_LIGHT,
-  hoverBgColorDark = DEFAULT_HOVER_BG_DARK,
+
   iconSize = DEFAULT_ICON_SIZE,
   padding = DEFAULT_PADDING,
   showText = false,
@@ -50,9 +43,10 @@ export default function CopyButton({
   const iconWrapperClasses = clsx(
     'flex items-center justify-center rounded-full transition-colors duration-150 ease-in-out',
     padding,
-    `bg-[${bgColorLight}] dark:bg-[${bgColorDark}]`,
-    `hover:bg-[${hoverBgColorLight}] dark:hover:bg-[${hoverBgColorDark}]`,
+    `bg-[${DEFAULT_BG_LIGHT}] dark:bg-[${DEFAULT_BG_DARK}]`,
     {
+      [`hover:bg-[${DEFAULT_HOVER_BG_LIGHT}] dark:hover:bg-[${DEFAULT_HOVER_BG_DARK}]`]:
+        !isCopied,
       'bg-green-100 dark:bg-green-900 hover:bg-green-100 dark:hover:bg-green-900':
         isCopied,
     },

frontend/src/components/DocumentPagination.tsx

@@ -61,12 +61,12 @@ const Pagination: React.FC<PaginationProps> = ({
         <div className="relative">
           <button
             onClick={toggleDropdown}
-            className="rounded border px-3 py-1 hover:bg-gray-200 dark:bg-dark-charcoal dark:text-light-gray dark:hover:bg-neutral-700"
+            className="dark:bg-dark-charcoal dark:text-light-gray rounded border px-3 py-1 hover:bg-gray-200 dark:hover:bg-neutral-700"
           >
             {rowsPerPage}
           </button>
           <div
-            className={`absolute right-0 z-50 mt-1 w-28 transform bg-white shadow-lg ring-1 ring-black ring-opacity-5 transition-all duration-200 ease-in-out dark:bg-dark-charcoal ${
+            className={`ring-opacity-5 dark:bg-dark-charcoal absolute right-0 z-50 mt-1 w-28 transform bg-white shadow-lg ring-1 ring-black transition-all duration-200 ease-in-out ${
               isDropdownOpen
                 ? 'block scale-100 opacity-100'
                 : 'hidden scale-95 opacity-0'
@@ -78,8 +78,8 @@ const Pagination: React.FC<PaginationProps> = ({
                 onClick={() => handleSelectRowsPerPage(option)}
                 className={`cursor-pointer px-4 py-2 text-xs hover:bg-gray-100 dark:hover:bg-neutral-700 ${
                   rowsPerPage === option
-                    ? 'bg-gray-100 dark:bg-neutral-700 dark:text-light-gray'
-                    : 'bg-white dark:bg-dark-charcoal dark:text-light-gray'
+                    ? 'dark:text-light-gray bg-gray-100 dark:bg-neutral-700'
+                    : 'dark:bg-dark-charcoal dark:text-light-gray bg-white'
                 }`}
               >
                 {option}

frontend/src/components/DropdownMenu.tsx

@@ -88,7 +88,7 @@ export default function DropdownMenu({
       onClick={(e) => e.stopPropagation()}
     >
       <div
-        className={`w-28 transform rounded-md bg-white shadow-lg ring-1 ring-black ring-opacity-5 transition-all duration-200 ease-in-out dark:bg-dark-charcoal ${className}`}
+        className={`ring-opacity-5 dark:bg-dark-charcoal w-28 transform rounded-md bg-white shadow-lg ring-1 ring-black transition-all duration-200 ease-in-out ${className}`}
       >
         <div
           role="menu"
@@ -99,10 +99,10 @@ export default function DropdownMenu({
           {options.map((option, idx) => (
             <div
               id={`option-${idx}`}
-              className={`cursor-pointer px-4 py-2 text-xs hover:bg-gray-100 dark:text-light-gray dark:hover:bg-purple-taupe ${
+              className={`dark:text-light-gray dark:hover:bg-purple-taupe cursor-pointer px-4 py-2 text-xs hover:bg-gray-100 ${
                 selectedOption.value === option.value
-                  ? 'bg-gray-100 dark:bg-purple-taupe'
-                  : 'bg-white dark:bg-dark-charcoal'
+                  ? 'dark:bg-purple-taupe bg-gray-100'
+                  : 'dark:bg-dark-charcoal bg-white'
               }`}
               role="menuitem"
               key={option.value}

frontend/src/components/FilePicker.tsx

@@ -1,7 +1,11 @@
 import React, { useState, useEffect, useCallback, useRef } from 'react';
 import { formatBytes } from '../utils/stringUtils';
 import { formatDate } from '../utils/dateTimeUtils';
-import { getSessionToken, setSessionToken, removeSessionToken } from '../utils/providerUtils';
+import {
+  getSessionToken,
+  setSessionToken,
+  removeSessionToken,
+} from '../utils/providerUtils';
 import ConnectorAuth from '../components/ConnectorAuth';
 import FileIcon from '../assets/file.svg';
 import FolderIcon from '../assets/folder.svg';
@@ -28,7 +32,10 @@ interface CloudFile {
 }
 
 interface CloudFilePickerProps {
-  onSelectionChange: (selectedFileIds: string[], selectedFolderIds?: string[]) => void;
+  onSelectionChange: (
+    selectedFileIds: string[],
+    selectedFolderIds?: string[],
+  ) => void;
   onDisconnect?: () => void;
   provider: string;
   token: string | null;
@@ -51,23 +58,30 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
   } as const;
 
   const getProviderConfig = (provider: string) => {
-    return PROVIDER_CONFIG[provider as keyof typeof PROVIDER_CONFIG] || {
-      displayName: provider,
-      rootName: 'Root',
-    };
+    return (
+      PROVIDER_CONFIG[provider as keyof typeof PROVIDER_CONFIG] || {
+        displayName: provider,
+        rootName: 'Root',
+      }
+    );
   };
 
   const [files, setFiles] = useState<CloudFile[]>([]);
-  const [selectedFiles, setSelectedFiles] = useState<string[]>(initialSelectedFiles);
+  const [selectedFiles, setSelectedFiles] =
+    useState<string[]>(initialSelectedFiles);
   const [selectedFolders, setSelectedFolders] = useState<string[]>([]);
   const [isLoading, setIsLoading] = useState(false);
   const [hasMoreFiles, setHasMoreFiles] = useState(false);
   const [nextPageToken, setNextPageToken] = useState<string | null>(null);
   const [currentFolderId, setCurrentFolderId] = useState<string | null>(null);
-  const [folderPath, setFolderPath] = useState<Array<{ id: string | null, name: string }>>([{
-    id: null,
-    name: getProviderConfig(provider).rootName
-  }]);
+  const [folderPath, setFolderPath] = useState<
+    Array<{ id: string | null; name: string }>
+  >([
+    {
+      id: null,
+      name: getProviderConfig(provider).rootName,
+    },
+  ]);
   const [searchQuery, setSearchQuery] = useState<string>('');
   const [authError, setAuthError] = useState<string>('');
   const [isConnected, setIsConnected] = useState(false);
@@ -77,9 +91,11 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
   const searchTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null);
 
   const isFolder = (file: CloudFile) => {
-    return file.isFolder ||
+    return (
+      file.isFolder ||
       file.type === 'application/vnd.google-apps.folder' ||
-      file.type === 'folder';
+      file.type === 'folder'
+    );
   };
 
   const loadCloudFiles = useCallback(
@@ -87,7 +103,7 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
       sessionToken: string,
       folderId: string | null,
       pageToken?: string,
-      searchQuery: string = ''
+      searchQuery = '',
     ) => {
       setIsLoading(true);
 
@@ -101,7 +117,7 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
           method: 'POST',
           headers: {
             'Content-Type': 'application/json',
-            'Authorization': `Bearer ${token}`
+            Authorization: `Bearer ${token}`,
           },
           body: JSON.stringify({
             provider: provider,
@@ -109,13 +125,15 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
             folder_id: folderId,
             limit: 10,
             page_token: pageToken,
-            search_query: searchQuery
-          })
+            search_query: searchQuery,
+          }),
         });
 
         const data = await response.json();
         if (data.success) {
-          setFiles(prev => pageToken ? [...prev, ...data.files] : data.files);
+          setFiles((prev) =>
+            pageToken ? [...prev, ...data.files] : data.files,
+          );
           setNextPageToken(data.next_page_token);
           setHasMoreFiles(!!data.next_page_token);
         } else {
@@ -133,7 +151,7 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
         setIsLoading(false);
       }
     },
-    [token, provider]
+    [token, provider],
   );
 
   const validateAndLoadFiles = useCallback(async () => {
@@ -145,14 +163,20 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
 
     try {
       const apiHost = import.meta.env.VITE_API_HOST;
-      const validateResponse = await fetch(`${apiHost}/api/connectors/validate-session`, {
-        method: 'POST',
-        headers: {
-          'Content-Type': 'application/json',
-          'Authorization': `Bearer ${token}`
+      const validateResponse = await fetch(
+        `${apiHost}/api/connectors/validate-session`,
+        {
+          method: 'POST',
+          headers: {
+            'Content-Type': 'application/json',
+            Authorization: `Bearer ${token}`,
+          },
+          body: JSON.stringify({
+            provider: provider,
+            session_token: sessionToken,
+          }),
         },
-        body: JSON.stringify({ provider: provider, session_token: sessionToken })
-      });
+      );
 
       if (!validateResponse.ok) {
         removeSessionToken(provider);
@@ -171,14 +195,20 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
         setNextPageToken(null);
         setHasMoreFiles(false);
         setCurrentFolderId(null);
-        setFolderPath([{
-          id: null, name: getProviderConfig(provider).rootName
-        }]);
+        setFolderPath([
+          {
+            id: null,
+            name: getProviderConfig(provider).rootName,
+          },
+        ]);
         loadCloudFiles(sessionToken, null, undefined, '');
       } else {
         removeSessionToken(provider);
         setIsConnected(false);
-        setAuthError(validateData.error || 'Session expired. Please reconnect your account.');
+        setAuthError(
+          validateData.error ||
+            'Session expired. Please reconnect your account.',
+        );
       }
     } catch (error) {
       console.error('Error validating session:', error);
@@ -201,10 +231,23 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
     if (isNearBottom && hasMoreFiles && !isLoading && nextPageToken) {
       const sessionToken = getSessionToken(provider);
       if (sessionToken) {
-        loadCloudFiles(sessionToken, currentFolderId, nextPageToken, searchQuery);
+        loadCloudFiles(
+          sessionToken,
+          currentFolderId,
+          nextPageToken,
+          searchQuery,
+        );
       }
     }
-  }, [hasMoreFiles, isLoading, nextPageToken, currentFolderId, searchQuery, provider, loadCloudFiles]);
+  }, [
+    hasMoreFiles,
+    isLoading,
+    nextPageToken,
+    currentFolderId,
+    searchQuery,
+    provider,
+    loadCloudFiles,
+  ]);
 
   useEffect(() => {
     const scrollContainer = scrollContainerRef.current;
@@ -245,7 +288,7 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
     setIsLoading(true);
 
     setCurrentFolderId(folderId);
-    setFolderPath(prev => [...prev, { id: folderId, name: folderName }]);
+    setFolderPath((prev) => [...prev, { id: folderId, name: folderName }]);
     setSearchQuery('');
 
     const sessionToken = getSessionToken(provider);
@@ -273,13 +316,13 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
   const handleFileSelect = (fileId: string, isFolder: boolean) => {
     if (isFolder) {
       const newSelectedFolders = selectedFolders.includes(fileId)
-        ? selectedFolders.filter(id => id !== fileId)
+        ? selectedFolders.filter((id) => id !== fileId)
         : [...selectedFolders, fileId];
       setSelectedFolders(newSelectedFolders);
       onSelectionChange(selectedFiles, newSelectedFolders);
     } else {
       const newSelectedFiles = selectedFiles.includes(fileId)
-        ? selectedFiles.filter(id => id !== fileId)
+        ? selectedFiles.filter((id) => id !== fileId)
         : [...selectedFiles, fileId];
       setSelectedFiles(newSelectedFiles);
       onSelectionChange(newSelectedFiles, selectedFolders);
@@ -287,11 +330,11 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
   };
 
   return (
-    <div className=''>
+    <div className="">
       {authError && (
-        <div className="text-red-500 text-sm mb-4 text-center">{authError}</div>
+        <div className="mb-4 text-center text-sm text-red-500">{authError}</div>
       )}
-      
+
       <ConnectorAuth
         provider={provider}
         onSuccess={(data) => {
@@ -318,10 +361,18 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
               method: 'POST',
               headers: {
                 'Content-Type': 'application/json',
-                'Authorization': `Bearer ${token}`
+                Authorization: `Bearer ${token}`,
               },
-              body: JSON.stringify({ provider: provider, session_token: sessionToken })
-            }).catch(err => console.error(`Error disconnecting from ${getProviderConfig(provider).displayName}:`, err));
+              body: JSON.stringify({
+                provider: provider,
+                session_token: sessionToken,
+              }),
+            }).catch((err) =>
+              console.error(
+                `Error disconnecting from ${getProviderConfig(provider).displayName}:`,
+                err,
+              ),
+            );
           }
 
           removeSessionToken(provider);
@@ -337,13 +388,16 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
       />
 
       {isConnected && (
-        <div className="border border-[#D7D7D7] rounded-lg dark:border-[#6A6A6A] mt-3">
-          <div className="border-[#EEE6FF78] dark:border-[#6A6A6A] rounded-t-lg">
+        <div className="mt-3 rounded-lg border border-[#D7D7D7] dark:border-[#6A6A6A]">
+          <div className="rounded-t-lg border-[#EEE6FF78] dark:border-[#6A6A6A]">
             {/* Breadcrumb navigation */}
-            <div className="px-4 pt-4 bg-[#EEE6FF78] dark:bg-[#2A262E] rounded-t-lg">
-              <div className="flex items-center gap-1 mb-2">
+            <div className="rounded-t-lg bg-[#EEE6FF78] px-4 pt-4 dark:bg-[#2A262E]">
+              <div className="mb-2 flex items-center gap-1">
                 {folderPath.map((path, index) => (
-                  <div key={path.id || 'root'} className="flex items-center gap-1">
+                  <div
+                    key={path.id || 'root'}
+                    className="flex items-center gap-1"
+                  >
                     {index > 0 && <span className="text-gray-400">/</span>}
                     <button
                       onClick={() => navigateBack(index)}
@@ -369,7 +423,9 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
                   colorVariant="silver"
                   borderVariant="thin"
                   labelBgClassName="bg-[#EEE6FF78] dark:bg-[#2A262E]"
-                  leftIcon={<img src={SearchIcon} alt="Search" width={16} height={16} />}
+                  leftIcon={
+                    <img src={SearchIcon} alt="Search" width={16} height={16} />
+                  }
                 />
               </div>
 
@@ -386,7 +442,7 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
                 className="scrollbar-thin md:w-4xl lg:w-5xl"
                 bordered={false}
               >
-                {(
+                {
                   <>
                     <Table minWidth="1200px">
                       <TableHead>
@@ -411,13 +467,16 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
                           >
                             <TableCell width="40px" align="center">
                               <div
-                                className="flex h-5 w-5 text-sm shrink-0 items-center justify-center border border-[#EEE6FF78] p-[0.5px] dark:border-[#6A6A6A] cursor-pointer mx-auto"
+                                className="mx-auto flex h-5 w-5 shrink-0 cursor-pointer items-center justify-center border border-[#EEE6FF78] p-[0.5px] text-sm dark:border-[#6A6A6A]"
                                 onClick={(e) => {
                                   e.stopPropagation();
                                   handleFileSelect(file.id, isFolder(file));
                                 }}
                               >
-                                {(isFolder(file) ? selectedFolders : selectedFiles).includes(file.id) && (
+                                {(isFolder(file)
+                                  ? selectedFolders
+                                  : selectedFiles
+                                ).includes(file.id) && (
                                   <img
                                     src={CheckIcon}
                                     alt="Selected"
@@ -427,21 +486,21 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
                               </div>
                             </TableCell>
                             <TableCell>
-                              <div className="flex items-center gap-3 min-w-0">
+                              <div className="flex min-w-0 items-center gap-3">
                                 <div className="flex-shrink-0">
                                   <img
                                     src={isFolder(file) ? FolderIcon : FileIcon}
-                                    alt={isFolder(file) ? "Folder" : "File"}
+                                    alt={isFolder(file) ? 'Folder' : 'File'}
                                     className="h-6 w-6"
                                   />
                                 </div>
                                 <span className="truncate">{file.name}</span>
                               </div>
                             </TableCell>
-                            <TableCell className='text-xs'>
+                            <TableCell className="text-xs">
                               {formatDate(file.modifiedTime)}
                             </TableCell>
-                            <TableCell className='text-xs'>
+                            <TableCell className="text-xs">
                               {file.size ? formatBytes(file.size) : '-'}
                             </TableCell>
                           </TableRow>
@@ -450,7 +509,7 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
                     </Table>
 
                     {isLoading && (
-                      <div className="flex items-center justify-center p-4 border-t border-[#EEE6FF78] dark:border-[#6A6A6A]">
+                      <div className="flex items-center justify-center border-t border-[#EEE6FF78] p-4 dark:border-[#6A6A6A]">
                         <div className="inline-flex items-center gap-2 text-sm text-gray-600 dark:text-gray-400">
                           <div className="h-4 w-4 animate-spin rounded-full border-2 border-blue-500 border-t-transparent"></div>
                           Loading more files...
@@ -458,7 +517,7 @@ export const FilePicker: React.FC<CloudFilePickerProps> = ({
                       </div>
                     )}
                   </>
-                )}
+                }
               </TableContainer>
             </div>
           </div>

frontend/src/components/FileTreeComponent.tsx

@@ -542,31 +542,26 @@ const FileTreeComponent: React.FC<FileTreeComponentProps> = ({
     const parentRow =
       currentPath.length > 0
         ? [
-          <TableRow
-            key="parent-dir"
-            onClick={navigateUp}
-          >
-            <TableCell width="40%" align="left">
-              <div className="flex items-center">
-                <img
-                  src={FolderIcon}
-                  alt={t('settings.sources.parentFolderAlt')}
-                  className="mr-2 h-4 w-4 flex-shrink-0"
-                />
-                <span className="truncate">
-                  ..
-                </span>
-              </div>
-            </TableCell>
-            <TableCell width="30%" align="left">
-              -
-            </TableCell>
-            <TableCell width="20%" align="right">
-              -
-            </TableCell>
-            <TableCell width="10%" align="right"></TableCell>
-          </TableRow>,
-        ]
+            <TableRow key="parent-dir" onClick={navigateUp}>
+              <TableCell width="40%" align="left">
+                <div className="flex items-center">
+                  <img
+                    src={FolderIcon}
+                    alt={t('settings.sources.parentFolderAlt')}
+                    className="mr-2 h-4 w-4 flex-shrink-0"
+                  />
+                  <span className="truncate">..</span>
+                </div>
+              </TableCell>
+              <TableCell width="30%" align="left">
+                -
+              </TableCell>
+              <TableCell width="20%" align="right">
+                -
+              </TableCell>
+              <TableCell width="10%" align="right"></TableCell>
+            </TableRow>,
+          ]
         : [];
 
     // Render directories first, then files
@@ -578,10 +573,7 @@ const FileTreeComponent: React.FC<FileTreeComponentProps> = ({
         const dirStats = calculateDirectoryStats(node as DirectoryStructure);
 
         return (
-          <TableRow
-            key={itemId}
-            onClick={() => navigateToDirectory(name)}
-          >
+          <TableRow key={itemId} onClick={() => navigateToDirectory(name)}>
             <TableCell width="40%" align="left">
               <div className="flex min-w-0 items-center">
                 <img
@@ -589,9 +581,7 @@ const FileTreeComponent: React.FC<FileTreeComponentProps> = ({
                   alt={t('settings.sources.folderAlt')}
                   className="mr-2 h-4 w-4 flex-shrink-0"
                 />
-                <span className="truncate">
-                  {name}
-                </span>
+                <span className="truncate">{name}</span>
               </div>
             </TableCell>
             <TableCell width="30%" align="left">
@@ -635,10 +625,7 @@ const FileTreeComponent: React.FC<FileTreeComponentProps> = ({
         const menuRef = getMenuRef(itemId);
 
         return (
-          <TableRow
-            key={itemId}
-            onClick={() => handleFileClick(name)}
-          >
+          <TableRow key={itemId} onClick={() => handleFileClick(name)}>
             <TableCell width="40%" align="left">
               <div className="flex min-w-0 items-center">
                 <img
@@ -646,9 +633,7 @@ const FileTreeComponent: React.FC<FileTreeComponentProps> = ({
                   alt={t('settings.sources.fileAlt')}
                   className="mr-2 h-4 w-4 flex-shrink-0"
                 />
-                <span className="truncate">
-                  {name}
-                </span>
+                <span className="truncate">{name}</span>
               </div>
             </TableCell>
             <TableCell width="30%" align="left">
@@ -854,9 +839,7 @@ const FileTreeComponent: React.FC<FileTreeComponentProps> = ({
                     </TableHeader>
                   </TableRow>
                 </TableHead>
-                <TableBody>
-                  {renderFileTree(currentDirectory)}
-                </TableBody>
+                <TableBody>{renderFileTree(currentDirectory)}</TableBody>
               </Table>
             </TableContainer>
           </div>

frontend/src/components/FileUpload.tsx

@@ -159,7 +159,7 @@ export const FileUpload = ({
           e.stopPropagation();
           handleRemove();
         }}
-        className="absolute -right-2 -top-2 rounded-full bg-[#7D54D1] p-1 transition-colors hover:bg-[#714cbc]"
+        className="absolute -top-2 -right-2 rounded-full bg-[#7D54D1] p-1 transition-colors hover:bg-[#714cbc]"
       >
         <img src={Cross} alt="remove" className="h-3 w-3" />
       </button>
@@ -215,7 +215,7 @@ export const FileUpload = ({
         <input {...getInputProps()} />
         {children || defaultContent}
         {errors.length > 0 && (
-          <div className="absolute left-0 right-0 mt-[2px] px-4 text-xs text-red-600">
+          <div className="absolute right-0 left-0 mt-[2px] px-4 text-xs text-red-600">
             {errors.map((error, i) => (
               <p key={i} className="truncate">
                 {error}

frontend/src/components/GoogleDrivePicker.tsx

@@ -1,9 +1,13 @@
 import React, { useState, useEffect } from 'react';
+import { useTranslation } from 'react-i18next';
 import useDrivePicker from 'react-google-drive-picker';
 
 import ConnectorAuth from './ConnectorAuth';
-import { getSessionToken, setSessionToken, removeSessionToken } from '../utils/providerUtils';
-
+import {
+  getSessionToken,
+  setSessionToken,
+  removeSessionToken,
+} from '../utils/providerUtils';
 
 interface PickerFile {
   id: string;
@@ -23,6 +27,7 @@ const GoogleDrivePicker: React.FC<GoogleDrivePickerProps> = ({
   token,
   onSelectionChange,
 }) => {
+  const { t } = useTranslation();
   const [selectedFiles, setSelectedFiles] = useState<PickerFile[]>([]);
   const [selectedFolders, setSelectedFolders] = useState<PickerFile[]>([]);
   const [isLoading, setIsLoading] = useState(false);
@@ -31,9 +36,9 @@ const GoogleDrivePicker: React.FC<GoogleDrivePickerProps> = ({
   const [authError, setAuthError] = useState<string>('');
   const [accessToken, setAccessToken] = useState<string | null>(null);
   const [isValidating, setIsValidating] = useState(false);
-  
+
   const [openPicker] = useDrivePicker();
-  
+
   useEffect(() => {
     const sessionToken = getSessionToken('google_drive');
     if (sessionToken) {
@@ -46,25 +51,36 @@ const GoogleDrivePicker: React.FC<GoogleDrivePickerProps> = ({
   const validateSession = async (sessionToken: string) => {
     try {
       const apiHost = import.meta.env.VITE_API_HOST;
-      const validateResponse = await fetch(`${apiHost}/api/connectors/validate-session`, {
-        method: 'POST',
-        headers: {
-          'Content-Type': 'application/json',
-          'Authorization': `Bearer ${token}`
+      const validateResponse = await fetch(
+        `${apiHost}/api/connectors/validate-session`,
+        {
+          method: 'POST',
+          headers: {
+            'Content-Type': 'application/json',
+            Authorization: `Bearer ${token}`,
+          },
+          body: JSON.stringify({
+            provider: 'google_drive',
+            session_token: sessionToken,
+          }),
         },
-        body: JSON.stringify({ provider: 'google_drive', session_token: sessionToken })
-      });
+      );
 
       if (!validateResponse.ok) {
         setIsConnected(false);
-        setAuthError('Session expired. Please reconnect to Google Drive.');
+        setAuthError(
+          t('modals.uploadDoc.connectors.googleDrive.sessionExpired'),
+        );
         setIsValidating(false);
         return false;
       }
 
       const validateData = await validateResponse.json();
       if (validateData.success) {
-        setUserEmail(validateData.user_email || 'Connected User');
+        setUserEmail(
+          validateData.user_email ||
+            t('modals.uploadDoc.connectors.auth.connectedUser'),
+        );
         setIsConnected(true);
         setAuthError('');
         setAccessToken(validateData.access_token || null);
@@ -72,13 +88,16 @@ const GoogleDrivePicker: React.FC<GoogleDrivePickerProps> = ({
         return true;
       } else {
         setIsConnected(false);
-        setAuthError(validateData.error || 'Session expired. Please reconnect your account.');
+        setAuthError(
+          validateData.error ||
+            t('modals.uploadDoc.connectors.googleDrive.sessionExpiredGeneric'),
+        );
         setIsValidating(false);
         return false;
       }
     } catch (error) {
       console.error('Error validating session:', error);
-      setAuthError('Failed to validate session. Please reconnect.');
+      setAuthError(t('modals.uploadDoc.connectors.googleDrive.validateFailed'));
       setIsConnected(false);
       setIsValidating(false);
       return false;
@@ -87,21 +106,21 @@ const GoogleDrivePicker: React.FC<GoogleDrivePickerProps> = ({
 
   const handleOpenPicker = async () => {
     setIsLoading(true);
-    
+
     const sessionToken = getSessionToken('google_drive');
-    
+
     if (!sessionToken) {
-      setAuthError('No valid session found. Please reconnect to Google Drive.');
+      setAuthError(t('modals.uploadDoc.connectors.googleDrive.noSession'));
       setIsLoading(false);
       return;
     }
-    
+
     if (!accessToken) {
-      setAuthError('No access token available. Please reconnect to Google Drive.');
+      setAuthError(t('modals.uploadDoc.connectors.googleDrive.noAccessToken'));
       setIsLoading(false);
       return;
     }
-    
+
     try {
       const clientId: string = import.meta.env.VITE_GOOGLE_CLIENT_ID;
 
@@ -117,17 +136,18 @@ const GoogleDrivePicker: React.FC<GoogleDrivePickerProps> = ({
 
       openPicker({
         clientId: clientId,
-        developerKey: "",
+        developerKey: '',
         appId: appId,
         setSelectFolderEnabled: false,
-        viewId: "DOCS",
+        viewId: 'DOCS',
         showUploadView: false,
         showUploadFolders: false,
         supportDrives: false,
         multiselect: true,
         token: accessToken,
-        viewMimeTypes: 'application/vnd.google-apps.document,application/vnd.google-apps.presentation,application/vnd.google-apps.spreadsheet,application/pdf,application/vnd.openxmlformats-officedocument.wordprocessingml.document,application/vnd.openxmlformats-officedocument.presentationml.presentation,application/vnd.openxmlformats-officedocument.spreadsheetml.sheet,application/msword,application/vnd.ms-powerpoint,application/vnd.ms-excel,text/plain,text/csv,text/html,text/markdown,text/x-rst,application/json,application/epub+zip,application/rtf,image/jpeg,image/jpg,image/png',
-        callbackFunction: (data:any) => {
+        viewMimeTypes:
+          'application/vnd.google-apps.document,application/vnd.google-apps.presentation,application/vnd.google-apps.spreadsheet,application/pdf,application/vnd.openxmlformats-officedocument.wordprocessingml.document,application/vnd.openxmlformats-officedocument.presentationml.presentation,application/vnd.openxmlformats-officedocument.spreadsheetml.sheet,application/msword,application/vnd.ms-powerpoint,application/vnd.ms-excel,text/plain,text/csv,text/html,text/markdown,text/x-rst,application/json,application/epub+zip,application/rtf,image/jpeg,image/jpg,image/png',
+        callbackFunction: (data: any) => {
           setIsLoading(false);
           if (data.action === 'picked') {
             const docs = data.docs;
@@ -136,14 +156,14 @@ const GoogleDrivePicker: React.FC<GoogleDrivePickerProps> = ({
             const newFolders: PickerFile[] = [];
 
             docs.forEach((doc: any) => {
-                const item = {
-                  id: doc.id,
-                  name: doc.name,
-                  mimeType: doc.mimeType,
-                  iconUrl: doc.iconUrl || '',
-                  description: doc.description,
-                  sizeBytes: doc.sizeBytes
-                };
+              const item = {
+                id: doc.id,
+                name: doc.name,
+                mimeType: doc.mimeType,
+                iconUrl: doc.iconUrl || '',
+                description: doc.description,
+                sizeBytes: doc.sizeBytes,
+              };
 
               if (doc.mimeType === 'application/vnd.google-apps.folder') {
                 newFolders.push(item);
@@ -152,27 +172,33 @@ const GoogleDrivePicker: React.FC<GoogleDrivePickerProps> = ({
               }
             });
 
-            setSelectedFiles(prevFiles => {
-              const existingFileIds = new Set(prevFiles.map(file => file.id));
-              const uniqueNewFiles = newFiles.filter(file => !existingFileIds.has(file.id));
+            setSelectedFiles((prevFiles) => {
+              const existingFileIds = new Set(prevFiles.map((file) => file.id));
+              const uniqueNewFiles = newFiles.filter(
+                (file) => !existingFileIds.has(file.id),
+              );
               return [...prevFiles, ...uniqueNewFiles];
             });
 
-            setSelectedFolders(prevFolders => {
-              const existingFolderIds = new Set(prevFolders.map(folder => folder.id));
-              const uniqueNewFolders = newFolders.filter(folder => !existingFolderIds.has(folder.id));
+            setSelectedFolders((prevFolders) => {
+              const existingFolderIds = new Set(
+                prevFolders.map((folder) => folder.id),
+              );
+              const uniqueNewFolders = newFolders.filter(
+                (folder) => !existingFolderIds.has(folder.id),
+              );
               return [...prevFolders, ...uniqueNewFolders];
             });
             onSelectionChange(
-              [...selectedFiles, ...newFiles].map(file => file.id),
-              [...selectedFolders, ...newFolders].map(folder => folder.id)
+              [...selectedFiles, ...newFiles].map((file) => file.id),
+              [...selectedFolders, ...newFolders].map((folder) => folder.id),
             );
           }
         },
       });
     } catch (error) {
       console.error('Error opening picker:', error);
-      setAuthError('Failed to open file picker. Please try again.');
+      setAuthError(t('modals.uploadDoc.connectors.googleDrive.pickerFailed'));
       setIsLoading(false);
     }
   };
@@ -186,9 +212,12 @@ const GoogleDrivePicker: React.FC<GoogleDrivePickerProps> = ({
           method: 'POST',
           headers: {
             'Content-Type': 'application/json',
-            'Authorization': `Bearer ${token}`
+            Authorization: `Bearer ${token}`,
           },
-          body: JSON.stringify({ provider: 'google_drive', session_token: sessionToken })
+          body: JSON.stringify({
+            provider: 'google_drive',
+            session_token: sessionToken,
+          }),
         });
       } catch (err) {
         console.error('Error disconnecting from Google Drive:', err);
@@ -207,24 +236,24 @@ const GoogleDrivePicker: React.FC<GoogleDrivePickerProps> = ({
 
   const ConnectedStateSkeleton = () => (
     <div className="mb-4">
-      <div className="w-full flex items-center justify-between rounded-[10px] bg-gray-200 dark:bg-gray-700 px-4 py-2 animate-pulse">
+      <div className="flex w-full animate-pulse items-center justify-between rounded-[10px] bg-gray-200 px-4 py-2 dark:bg-gray-700">
         <div className="flex items-center gap-2">
-          <div className="h-4 w-4 bg-gray-300 dark:bg-gray-600 rounded"></div>
-          <div className="h-4 w-32 bg-gray-300 dark:bg-gray-600 rounded"></div>
+          <div className="h-4 w-4 rounded bg-gray-300 dark:bg-gray-600"></div>
+          <div className="h-4 w-32 rounded bg-gray-300 dark:bg-gray-600"></div>
         </div>
-        <div className="h-4 w-16 bg-gray-300 dark:bg-gray-600 rounded"></div>
+        <div className="h-4 w-16 rounded bg-gray-300 dark:bg-gray-600"></div>
       </div>
     </div>
   );
 
   const FilesSectionSkeleton = () => (
-    <div className="border border-[#EEE6FF78] rounded-lg dark:border-[#6A6A6A]">
+    <div className="rounded-lg border border-[#EEE6FF78] dark:border-[#6A6A6A]">
       <div className="p-4">
-        <div className="flex justify-between items-center mb-4">
-          <div className="h-5 w-24 bg-gray-200 dark:bg-gray-700 rounded animate-pulse"></div>
-          <div className="h-8 w-24 bg-gray-200 dark:bg-gray-700 rounded animate-pulse"></div>
+        <div className="mb-4 flex items-center justify-between">
+          <div className="h-5 w-24 animate-pulse rounded bg-gray-200 dark:bg-gray-700"></div>
+          <div className="h-8 w-24 animate-pulse rounded bg-gray-200 dark:bg-gray-700"></div>
         </div>
-        <div className="h-4 w-40 bg-gray-200 dark:bg-gray-700 rounded animate-pulse"></div>
+        <div className="h-4 w-40 animate-pulse rounded bg-gray-200 dark:bg-gray-700"></div>
       </div>
     </div>
   );
@@ -240,9 +269,12 @@ const GoogleDrivePicker: React.FC<GoogleDrivePickerProps> = ({
         <>
           <ConnectorAuth
             provider="google_drive"
-            label="Connect to Google Drive"
+            label={t('modals.uploadDoc.connectors.googleDrive.connect')}
             onSuccess={(data) => {
-              setUserEmail(data.user_email || 'Connected User');
+              setUserEmail(
+                data.user_email ||
+                  t('modals.uploadDoc.connectors.auth.connectedUser'),
+              );
               setIsConnected(true);
               setAuthError('');
 
@@ -262,42 +294,70 @@ const GoogleDrivePicker: React.FC<GoogleDrivePickerProps> = ({
           />
 
           {isConnected && (
-            <div className="border border-[#EEE6FF78] rounded-lg dark:border-[#6A6A6A]">
+            <div className="rounded-lg border border-[#EEE6FF78] dark:border-[#6A6A6A]">
               <div className="p-4">
-                <div className="flex justify-between items-center mb-4">
-                  <h3 className="text-sm font-medium">Selected Files</h3>
+                <div className="mb-4 flex items-center justify-between">
+                  <h3 className="text-sm font-medium">
+                    {t('modals.uploadDoc.connectors.googleDrive.selectedFiles')}
+                  </h3>
                   <button
                     onClick={() => handleOpenPicker()}
-                    className="bg-[#A076F6] hover:bg-[#8A5FD4] text-white text-sm py-1 px-3 rounded-md"
+                    className="rounded-md bg-[#A076F6] px-3 py-1 text-sm text-white hover:bg-[#8A5FD4]"
                     disabled={isLoading}
                   >
-                    {isLoading ? 'Loading...' : 'Select Files'}
+                    {isLoading
+                      ? t('modals.uploadDoc.connectors.googleDrive.loading')
+                      : t(
+                          'modals.uploadDoc.connectors.googleDrive.selectFiles',
+                        )}
                   </button>
                 </div>
 
                 {selectedFiles.length === 0 && selectedFolders.length === 0 ? (
-                  <p className="text-gray-600 dark:text-gray-400 text-sm">No files or folders selected</p>
+                  <p className="text-sm text-gray-600 dark:text-gray-400">
+                    {t(
+                      'modals.uploadDoc.connectors.googleDrive.noFilesSelected',
+                    )}
+                  </p>
                 ) : (
                   <div className="max-h-60 overflow-y-auto">
                     {selectedFolders.length > 0 && (
                       <div className="mb-2">
-                        <h4 className="text-xs font-medium text-gray-500 mb-1">Folders</h4>
+                        <h4 className="mb-1 text-xs font-medium text-gray-500">
+                          {t('modals.uploadDoc.connectors.googleDrive.folders')}
+                        </h4>
                         {selectedFolders.map((folder) => (
-                          <div key={folder.id} className="flex items-center p-2 border-b border-gray-200 dark:border-gray-700">
-                            <img src={folder.iconUrl} alt="Folder" className="w-5 h-5 mr-2" />
-                            <span className="text-sm truncate flex-1">{folder.name}</span>
+                          <div
+                            key={folder.id}
+                            className="flex items-center border-b border-gray-200 p-2 dark:border-gray-700"
+                          >
+                            <img
+                              src={folder.iconUrl}
+                              alt={t(
+                                'modals.uploadDoc.connectors.googleDrive.folderAlt',
+                              )}
+                              className="mr-2 h-5 w-5"
+                            />
+                            <span className="flex-1 truncate text-sm">
+                              {folder.name}
+                            </span>
                             <button
                               onClick={() => {
-                                const newSelectedFolders = selectedFolders.filter(f => f.id !== folder.id);
+                                const newSelectedFolders =
+                                  selectedFolders.filter(
+                                    (f) => f.id !== folder.id,
+                                  );
                                 setSelectedFolders(newSelectedFolders);
                                 onSelectionChange(
-                                  selectedFiles.map(f => f.id),
-                                  newSelectedFolders.map(f => f.id)
+                                  selectedFiles.map((f) => f.id),
+                                  newSelectedFolders.map((f) => f.id),
                                 );
                               }}
-                              className="text-red-500 hover:text-red-700 text-sm ml-2"
+                              className="ml-2 text-sm text-red-500 hover:text-red-700"
                             >
-                              Remove
+                              {t(
+                                'modals.uploadDoc.connectors.googleDrive.remove',
+                              )}
                             </button>
                           </div>
                         ))}
@@ -306,23 +366,40 @@ const GoogleDrivePicker: React.FC<GoogleDrivePickerProps> = ({
 
                     {selectedFiles.length > 0 && (
                       <div>
-                        <h4 className="text-xs font-medium text-gray-500 mb-1">Files</h4>
+                        <h4 className="mb-1 text-xs font-medium text-gray-500">
+                          {t('modals.uploadDoc.connectors.googleDrive.files')}
+                        </h4>
                         {selectedFiles.map((file) => (
-                          <div key={file.id} className="flex items-center p-2 border-b border-gray-200 dark:border-gray-700">
-                            <img src={file.iconUrl} alt="File" className="w-5 h-5 mr-2" />
-                            <span className="text-sm truncate flex-1">{file.name}</span>
+                          <div
+                            key={file.id}
+                            className="flex items-center border-b border-gray-200 p-2 dark:border-gray-700"
+                          >
+                            <img
+                              src={file.iconUrl}
+                              alt={t(
+                                'modals.uploadDoc.connectors.googleDrive.fileAlt',
+                              )}
+                              className="mr-2 h-5 w-5"
+                            />
+                            <span className="flex-1 truncate text-sm">
+                              {file.name}
+                            </span>
                             <button
                               onClick={() => {
-                                const newSelectedFiles = selectedFiles.filter(f => f.id !== file.id);
+                                const newSelectedFiles = selectedFiles.filter(
+                                  (f) => f.id !== file.id,
+                                );
                                 setSelectedFiles(newSelectedFiles);
                                 onSelectionChange(
-                                  newSelectedFiles.map(f => f.id),
-                                  selectedFolders.map(f => f.id)
+                                  newSelectedFiles.map((f) => f.id),
+                                  selectedFolders.map((f) => f.id),
                                 );
                               }}
-                              className="text-red-500 hover:text-red-700 text-sm ml-2"
+                              className="ml-2 text-sm text-red-500 hover:text-red-700"
                             >
-                              Remove
+                              {t(
+                                'modals.uploadDoc.connectors.googleDrive.remove',
+                              )}
                             </button>
                           </div>
                         ))}

frontend/src/components/Input.tsx

@@ -59,7 +59,7 @@ const Input = ({
         {children}
       </input>
       {leftIcon && (
-        <div className="absolute left-3 top-1/2 transform -translate-y-1/2 flex items-center justify-center">
+        <div className="absolute top-1/2 left-3 flex -translate-y-1/2 transform items-center justify-center">
           {leftIcon}
         </div>
       )}
@@ -69,7 +69,9 @@ const Input = ({
           className={`absolute select-none ${
             hasValue ? '-top-2.5 left-3 text-xs' : ''
           } px-2 transition-all peer-placeholder-shown:top-2.5 ${
-            leftIcon ? 'peer-placeholder-shown:left-7' : 'peer-placeholder-shown:left-3'
+            leftIcon
+              ? 'peer-placeholder-shown:left-7'
+              : 'peer-placeholder-shown:left-3'
           } peer-placeholder-shown:${
             textSizeStyles[textSize]
           } text-gray-4000 pointer-events-none cursor-none peer-focus:-top-2.5 peer-focus:left-3 peer-focus:text-xs dark:text-gray-400 ${labelBgClassName} max-w-[calc(100%-24px)] overflow-hidden text-ellipsis whitespace-nowrap`}

frontend/src/components/ShareButton.tsx

@@ -15,7 +15,7 @@ export default function ShareButton({ conversationId }: ShareButtonProps) {
         onClick={() => {
           setShareModalState(true);
         }}
-        className="absolute right-20 top-4 z-20 rounded-full hover:bg-bright-gray dark:hover:bg-[#28292E]"
+        className="hover:bg-bright-gray absolute top-4 right-20 z-20 rounded-full dark:hover:bg-[#28292E]"
       >
         <img
           className="m-2 h-5 w-5 filter dark:invert"

frontend/src/components/SkeletonLoader.tsx

@@ -189,19 +189,19 @@ const SkeletonLoader: React.FC<SkeletonLoaderProps> = ({
       {Array.from({ length: count }).map((_, index) => (
         <div
           key={`chunk-skel-${index}`}
-          className="relative flex h-[197px] flex-col rounded-[5.86px] border border-[#D1D9E0] dark:border-[#6A6A6A] overflow-hidden w-full max-w-[487px] animate-pulse"
+          className="relative flex h-[197px] w-full max-w-[487px] animate-pulse flex-col overflow-hidden rounded-[5.86px] border border-[#D1D9E0] dark:border-[#6A6A6A]"
         >
           <div className="w-full">
-            <div className="flex w-full items-center justify-between border-b border-[#D1D9E0] bg-[#F6F8FA] dark:bg-[#27282D] dark:border-[#6A6A6A] px-4 py-3">
-              <div className="h-4 bg-gray-300 dark:bg-gray-600 rounded w-20"></div>
+            <div className="flex w-full items-center justify-between border-b border-[#D1D9E0] bg-[#F6F8FA] px-4 py-3 dark:border-[#6A6A6A] dark:bg-[#27282D]">
+              <div className="h-4 w-20 rounded bg-gray-300 dark:bg-gray-600"></div>
             </div>
-            <div className="px-4 pt-4 pb-6 space-y-3">
-              <div className="h-3 bg-gray-200 dark:bg-gray-700 rounded w-full"></div>
-              <div className="h-3 bg-gray-200 dark:bg-gray-700 rounded w-11/12"></div>
-              <div className="h-3 bg-gray-200 dark:bg-gray-700 rounded w-5/6"></div>
-              <div className="h-3 bg-gray-200 dark:bg-gray-700 rounded w-4/5"></div>
-              <div className="h-3 bg-gray-200 dark:bg-gray-700 rounded w-3/4"></div>
-              <div className="h-3 bg-gray-200 dark:bg-gray-700 rounded w-2/3"></div>
+            <div className="space-y-3 px-4 pt-4 pb-6">
+              <div className="h-3 w-full rounded bg-gray-200 dark:bg-gray-700"></div>
+              <div className="h-3 w-11/12 rounded bg-gray-200 dark:bg-gray-700"></div>
+              <div className="h-3 w-5/6 rounded bg-gray-200 dark:bg-gray-700"></div>
+              <div className="h-3 w-4/5 rounded bg-gray-200 dark:bg-gray-700"></div>
+              <div className="h-3 w-3/4 rounded bg-gray-200 dark:bg-gray-700"></div>
+              <div className="h-3 w-2/3 rounded bg-gray-200 dark:bg-gray-700"></div>
             </div>
           </div>
         </div>
@@ -214,24 +214,24 @@ const SkeletonLoader: React.FC<SkeletonLoaderProps> = ({
       {Array.from({ length: count }).map((_, idx) => (
         <div
           key={`source-skel-${idx}`}
-          className="flex h-[130px] w-full flex-col rounded-2xl bg-[#F9F9F9] dark:bg-[#383838] p-3 animate-pulse"
+          className="flex h-[130px] w-full animate-pulse flex-col rounded-2xl bg-[#F9F9F9] p-3 dark:bg-[#383838]"
         >
           <div className="w-full flex-1">
             <div className="flex w-full items-center justify-between gap-2">
               <div className="flex-1">
                 <div className="h-[13px] w-full rounded bg-gray-200 dark:bg-gray-700"></div>
               </div>
-              <div className="w-6 h-6 rounded bg-gray-200 dark:bg-gray-700"></div>
+              <div className="h-6 w-6 rounded bg-gray-200 dark:bg-gray-700"></div>
             </div>
           </div>
 
           <div className="flex flex-col items-start justify-start gap-1 pt-3">
-            <div className="flex items-center gap-2 mb-1">
-              <div className="w-3 h-3 rounded bg-gray-200 dark:bg-gray-700"></div>
+            <div className="mb-1 flex items-center gap-2">
+              <div className="h-3 w-3 rounded bg-gray-200 dark:bg-gray-700"></div>
               <div className="h-[12px] w-20 rounded bg-gray-200 dark:bg-gray-700"></div>
             </div>
             <div className="flex items-center gap-2">
-              <div className="w-3 h-3 rounded bg-gray-200 dark:bg-gray-700"></div>
+              <div className="h-3 w-3 rounded bg-gray-200 dark:bg-gray-700"></div>
               <div className="h-[12px] w-16 rounded bg-gray-200 dark:bg-gray-700"></div>
             </div>
           </div>
@@ -251,7 +251,6 @@ const SkeletonLoader: React.FC<SkeletonLoaderProps> = ({
     sourceCards: renderSourceCards,
   };
 
-
   const render = componentMap[component] || componentMap.default;
 
   return <>{render()}</>;

frontend/src/components/Table.tsx

@@ -6,11 +6,9 @@ interface TableProps {
   minWidth?: string;
 }
 
-
 interface TableContainerProps {
   children: React.ReactNode;
   className?: string;
-  ref?: React.Ref<HTMLDivElement>;
   height?: string;
   bordered?: boolean;
 }
@@ -34,45 +32,51 @@ interface TableCellProps {
   align?: 'left' | 'right' | 'center';
 }
 
-const TableContainer = React.forwardRef<HTMLDivElement, TableContainerProps>(({
-  children,
-  className = '',
-  height = 'auto',
-  bordered = true
-}, ref) => {
-  return (
-    <div className={`relative rounded-[6px] ${className}`}>
-      <div
-        ref={ref}
-        className={`w-full overflow-x-auto rounded-[6px] bg-transparent ${bordered ? 'border border-[#D7D7D7] dark:border-[#6A6A6A]' : ''}`}
-        style={{
-          maxHeight: height === 'auto' ? undefined : height,
-          overflowY: height === 'auto' ? 'hidden' : 'auto'
-        }}
-      >
-        {children}
+const TableContainer = React.forwardRef<HTMLDivElement, TableContainerProps>(
+  function TableContainer(
+    {
+      children,
+      className = '',
+      height = 'auto',
+      bordered = true,
+    }: TableContainerProps,
+    ref: React.ForwardedRef<HTMLDivElement>,
+  ) {
+    return (
+      <div className={`relative rounded-[6px] ${className}`}>
+        <div
+          ref={ref}
+          className={`w-full overflow-x-auto rounded-[6px] bg-transparent ${bordered ? 'border border-[#D7D7D7] dark:border-[#6A6A6A]' : ''}`}
+          style={{
+            maxHeight: height === 'auto' ? undefined : height,
+            overflowY: height === 'auto' ? 'hidden' : 'auto',
+          }}
+        >
+          {children}
+        </div>
       </div>
-    </div>
-  );
-});;
+    );
+  },
+);
+
 const Table: React.FC<TableProps> = ({
   children,
   className = '',
-  minWidth = 'min-w-[600px]'
+  minWidth = 'min-w-[600px]',
 }) => {
   return (
-    <table className={`w-full table-auto border-collapse bg-transparent ${minWidth} ${className}`}>
+    <table
+      className={`w-full table-auto border-collapse bg-transparent ${minWidth} ${className}`}
+    >
       {children}
     </table>
   );
 };
 const TableHead: React.FC<TableHeadProps> = ({ children, className = '' }) => {
   return (
-    <thead className={`
-      sticky top-0 z-10
-      bg-gray-100 dark:bg-[#27282D]
-      ${className}
-    `}>
+    <thead
+      className={`sticky top-0 z-10 bg-gray-100 dark:bg-[#27282D] ${className} `}
+    >
       {children}
     </thead>
   );
@@ -86,12 +90,20 @@ const TableBody: React.FC<TableHeadProps> = ({ children, className = '' }) => {
   );
 };
 
-const TableRow: React.FC<TableRowProps> = ({ children, className = '', onClick }) => {
-  const baseClasses = "border-b border-[#D7D7D7] hover:bg-[#ECEEEF] dark:border-[#6A6A6A] dark:hover:bg-[#27282D]";
-  const cursorClass = onClick ? "cursor-pointer" : "";
+const TableRow: React.FC<TableRowProps> = ({
+  children,
+  className = '',
+  onClick,
+}) => {
+  const baseClasses =
+    'border-b border-[#D7D7D7] hover:bg-[#ECEEEF] dark:border-[#6A6A6A] dark:hover:bg-[#27282D]';
+  const cursorClass = onClick ? 'cursor-pointer' : '';
 
   return (
-    <tr className={`${baseClasses} ${cursorClass} ${className}`} onClick={onClick}>
+    <tr
+      className={`${baseClasses} ${cursorClass} ${className}`}
+      onClick={onClick}
+    >
       {children}
     </tr>
   );
@@ -102,7 +114,7 @@ const TableHeader: React.FC<TableCellProps> = ({
   className = '',
   minWidth,
   width,
-  align = 'left'
+  align = 'left',
 }) => {
   const getAlignmentClass = () => {
     switch (align) {
@@ -133,7 +145,7 @@ const TableCell: React.FC<TableCellProps> = ({
   className = '',
   minWidth,
   width,
-  align = 'left'
+  align = 'left',
 }) => {
   const getAlignmentClass = () => {
     switch (align) {

frontend/src/components/TextToSpeechButton.tsx

@@ -68,7 +68,7 @@ export default function SpeakButton({
     <div
       className={`flex items-center justify-center rounded-full p-2 ${
         isSpeakHovered
-          ? `bg-[#EEEEEE] dark:bg-purple-taupe`
+          ? `dark:bg-purple-taupe bg-[#EEEEEE]`
           : `bg-[${colorLight ? colorLight : '#FFFFFF'}] dark:bg-[${colorDark ? colorDark : 'transparent'}]`
       }`}
     >

frontend/src/components/UploadToast.tsx

@@ -0,0 +1,229 @@
+import { useState } from 'react';
+
+import { useDispatch, useSelector } from 'react-redux';
+import { useTranslation } from 'react-i18next';
+import { selectUploadTasks, dismissUploadTask } from '../upload/uploadSlice';
+import ChevronDown from '../assets/chevron-down.svg';
+import CheckCircleFilled from '../assets/check-circle-filled.svg';
+import WarnIcon from '../assets/warn.svg';
+
+const PROGRESS_RADIUS = 10;
+const PROGRESS_CIRCUMFERENCE = 2 * Math.PI * PROGRESS_RADIUS;
+
+export default function UploadToast() {
+  const [collapsedTasks, setCollapsedTasks] = useState<Record<string, boolean>>(
+    {},
+  );
+
+  const toggleTaskCollapse = (taskId: string) => {
+    setCollapsedTasks((prev) => ({
+      ...prev,
+      [taskId]: !prev[taskId],
+    }));
+  };
+
+  const { t } = useTranslation();
+  const dispatch = useDispatch();
+  const uploadTasks = useSelector(selectUploadTasks);
+
+  const getStatusHeading = (status: string) => {
+    switch (status) {
+      case 'preparing':
+        return t('modals.uploadDoc.progress.wait');
+      case 'uploading':
+        return t('modals.uploadDoc.progress.upload');
+      case 'training':
+        return t('modals.uploadDoc.progress.upload');
+      case 'completed':
+        return t('modals.uploadDoc.progress.completed');
+      case 'failed':
+        return t('attachments.uploadFailed');
+      default:
+        return t('modals.uploadDoc.progress.preparing');
+    }
+  };
+
+  return (
+    <div className="fixed right-4 bottom-4 z-50 flex max-w-md flex-col gap-2">
+      {uploadTasks
+        .filter((task) => !task.dismissed)
+        .map((task) => {
+          const shouldShowProgress = [
+            'preparing',
+            'uploading',
+            'training',
+          ].includes(task.status);
+          const rawProgress = Math.min(Math.max(task.progress ?? 0, 0), 100);
+          const formattedProgress = Math.round(rawProgress);
+          const progressOffset =
+            PROGRESS_CIRCUMFERENCE * (1 - rawProgress / 100);
+          const isCollapsed = collapsedTasks[task.id] ?? false;
+
+          return (
+            <div
+              key={task.id}
+              className={`w-[271px] overflow-hidden rounded-2xl border border-[#00000021] shadow-[0px_24px_48px_0px_#00000029] transition-all duration-300 ${
+                task.status === 'completed'
+                  ? 'bg-[#FBFBFB] dark:bg-[#26272E]'
+                  : task.status === 'failed'
+                    ? 'bg-[#FBFBFB] dark:bg-[#26272E]'
+                    : 'bg-[#FBFBFB] dark:bg-[#26272E]'
+              }`}
+            >
+              <div className="flex flex-col">
+                <div
+                  className={`flex items-center justify-between px-4 py-3 ${
+                    task.status !== 'failed'
+                      ? 'bg-[#FBF2FE] dark:bg-transparent'
+                      : ''
+                  }`}
+                >
+                  <h3 className="font-inter text-[14px] leading-[16.5px] font-medium text-black dark:text-[#DCDCDC]">
+                    {getStatusHeading(task.status)}
+                  </h3>
+                  <div className="flex items-center gap-1">
+                    <button
+                      type="button"
+                      onClick={() => toggleTaskCollapse(task.id)}
+                      aria-label={
+                        isCollapsed
+                          ? t('modals.uploadDoc.progress.expandDetails')
+                          : t('modals.uploadDoc.progress.collapseDetails')
+                      }
+                      className="flex h-8 items-center justify-center p-0 text-black opacity-70 transition-opacity hover:opacity-100 dark:text-white"
+                    >
+                      <img
+                        src={ChevronDown}
+                        alt=""
+                        className={`h-4 w-4 transform transition-transform duration-200 dark:invert ${
+                          isCollapsed ? 'rotate-180' : ''
+                        }`}
+                      />
+                    </button>
+                    <button
+                      type="button"
+                      onClick={() => dispatch(dismissUploadTask(task.id))}
+                      className="flex h-8 items-center justify-center p-0 text-black opacity-70 transition-opacity hover:opacity-100 dark:text-white"
+                      aria-label={t('modals.uploadDoc.progress.dismiss')}
+                    >
+                      <svg
+                        width="16"
+                        height="16"
+                        viewBox="0 0 24 24"
+                        fill="none"
+                        xmlns="http://www.w3.org/2000/svg"
+                        className="h-4 w-4"
+                      >
+                        <path
+                          d="M18 6L6 18"
+                          stroke="currentColor"
+                          strokeWidth="2"
+                          strokeLinecap="round"
+                          strokeLinejoin="round"
+                        />
+                        <path
+                          d="M6 6L18 18"
+                          stroke="currentColor"
+                          strokeWidth="2"
+                          strokeLinecap="round"
+                          strokeLinejoin="round"
+                        />
+                      </svg>
+                    </button>
+                  </div>
+                </div>
+
+                <div
+                  className="grid overflow-hidden transition-[grid-template-rows] duration-300 ease-out"
+                  style={{ gridTemplateRows: isCollapsed ? '0fr' : '1fr' }}
+                >
+                  <div
+                    className={`min-h-0 overflow-hidden transition-opacity duration-300 ${
+                      isCollapsed ? 'opacity-0' : 'opacity-100'
+                    }`}
+                  >
+                    <div className="flex items-center justify-between px-5 py-3">
+                      <p
+                        className="font-inter max-w-[200px] truncate text-[13px] leading-[16.5px] font-normal text-black dark:text-[#B7BAB8]"
+                        title={task.fileName}
+                      >
+                        {task.fileName}
+                      </p>
+
+                      <div className="flex items-center gap-2">
+                        {shouldShowProgress && (
+                          <svg
+                            width="24"
+                            height="24"
+                            viewBox="0 0 24 24"
+                            className="h-6 w-6 flex-shrink-0 text-[#7D54D1]"
+                            role="progressbar"
+                            aria-valuemin={0}
+                            aria-valuemax={100}
+                            aria-valuenow={formattedProgress}
+                            aria-label={t(
+                              'modals.uploadDoc.progress.uploadProgress',
+                              {
+                                progress: formattedProgress,
+                              },
+                            )}
+                          >
+                            <circle
+                              className="text-gray-300 dark:text-gray-700"
+                              stroke="currentColor"
+                              strokeWidth="2"
+                              cx="12"
+                              cy="12"
+                              r={PROGRESS_RADIUS}
+                              fill="none"
+                            />
+                            <circle
+                              className="text-[#7D54D1]"
+                              stroke="currentColor"
+                              strokeWidth="2"
+                              strokeLinecap="round"
+                              strokeDasharray={PROGRESS_CIRCUMFERENCE}
+                              strokeDashoffset={progressOffset}
+                              cx="12"
+                              cy="12"
+                              r={PROGRESS_RADIUS}
+                              fill="none"
+                              transform="rotate(-90 12 12)"
+                            />
+                          </svg>
+                        )}
+
+                        {task.status === 'completed' && (
+                          <img
+                            src={CheckCircleFilled}
+                            alt=""
+                            className="h-6 w-6 flex-shrink-0"
+                            aria-hidden="true"
+                          />
+                        )}
+
+                        {task.status === 'failed' && (
+                          <img
+                            src={WarnIcon}
+                            alt=""
+                            className="h-6 w-6 flex-shrink-0"
+                            aria-hidden="true"
+                          />
+                        )}
+                      </div>
+                    </div>
+
+                    {task.status === 'failed' && task.errorMessage && (
+                      <span className="block px-5 pb-3 text-xs text-red-500">
+                        {task.errorMessage}
+                      </span>
+                    )}
+                  </div>
+                </div>
+              </div>
+            </div>
+          );
+        })}
+    </div>
+  );
+}

frontend/src/conversation/ConversationBubble.tsx

@@ -188,7 +188,7 @@ const ConversationBubble = forwardRef<
                     setIsEditClicked(true);
                     setEditInputBox(message ?? '');
                   }}
-                  className={`hover:bg-light-silver mt-3 flex h-fit shrink-0 cursor-pointer items-center rounded-full p-2 dark:hover:bg-[#35363B] ${isQuestionHovered || isEditClicked ? 'visible' : 'invisible'}`}
+                  className={`hover:bg-light-silver mt-3 flex h-fit shrink-0 cursor-pointer items-center rounded-full p-2 pt-1.5 pl-1.5 dark:hover:bg-[#35363B] ${isQuestionHovered || isEditClicked ? 'visible' : 'invisible'}`}
                 >
                   <img src={Edit} alt="Edit" className="cursor-pointer" />
                 </button>
@@ -407,7 +407,7 @@ const ConversationBubble = forwardRef<
               </p>
             </div>
             <div
-              className={`fade-in-bubble bg-gray-1000 dark:bg-gun-metal mr-5 flex max-w-full rounded-[28px] px-7 py-[18px] ${
+              className={`fade-in-bubble bg-gray-1000 dark:bg-gun-metal mr-5 flex max-w-full rounded-[18px] px-6 py-4.5 ${
                 type === 'ERROR'
                   ? 'text-red-3000 dark:border-red-2000 relative flex-row items-center rounded-full border border-transparent bg-[#FFE7E7] p-2 py-5 text-sm font-normal dark:text-white'
                   : 'flex-col rounded-3xl'

frontend/src/conversation/ConversationMessages.tsx

@@ -157,7 +157,7 @@ export default function ConversationMessages({
     if (query.error) {
       const retryButton = (
         <button
-          className="flex items-center justify-center gap-3 self-center rounded-full px-5 py-3 text-lg text-gray-500 transition-colors delay-100 hover:border-gray-500 disabled:cursor-not-allowed dark:text-bright-gray"
+          className="dark:text-bright-gray flex items-center justify-center gap-3 self-center rounded-full px-5 py-3 text-lg text-gray-500 transition-colors delay-100 hover:border-gray-500 disabled:cursor-not-allowed"
           disabled={status === 'loading'}
           onClick={() => {
             const questionToRetry = queries[index].prompt;
@@ -199,12 +199,12 @@ export default function ConversationMessages({
             scrollConversationToBottom();
           }}
           aria-label={t('Scroll to bottom') || 'Scroll to bottom'}
-          className="fixed bottom-40 right-14 z-10 flex h-7 w-7 items-center justify-center rounded-full border-[0.5px] border-gray-alpha bg-gray-100 bg-opacity-50 dark:bg-gunmetal md:h-9 md:w-9 md:bg-opacity-100"
+          className="border-gray-alpha bg-opacity-50 dark:bg-gunmetal md:bg-opacity-100 fixed right-14 bottom-40 z-10 flex h-7 w-7 items-center justify-center rounded-full border-[0.5px] bg-gray-100 md:h-9 md:w-9"
         >
           <img
             src={ArrowDown}
             alt="arrow down"
-            className="h-4 w-4 opacity-50 filter dark:invert md:h-5 md:w-5"
+            className="h-4 w-4 opacity-50 filter md:h-5 md:w-5 dark:invert"
           />
         </button>
       )}

frontend/src/locale/en.json

@@ -229,6 +229,9 @@
     "uploadDoc": {
       "label": "Upload new document",
       "select": "Choose how to upload your document to DocsGPT",
+      "selectSource": "Select the way to add your source",
+      "selectedFiles": "Selected Files",
+      "noFilesSelected": "No files selected",
       "file": "Upload from device",
       "back": "Back",
       "wait": "Please wait ...",
@@ -257,13 +260,74 @@
       },
       "progress": {
         "upload": "Upload is in progress",
-        "training": "Training is in progress",
-        "completed": "Training completed",
+        "training": "Upload is in progress",
+        "completed": "Upload completed",
         "wait": "This may take several minutes",
-        "tokenLimit": "Over the token limit, please consider uploading smaller document"
+        "preparing": "Preparing upload",
+        "tokenLimit": "Over the token limit, please consider uploading smaller document",
+        "expandDetails": "Expand upload details",
+        "collapseDetails": "Collapse upload details",
+        "dismiss": "Dismiss upload toast",
+        "uploadProgress": "Upload progress {{progress}}%",
+        "clear": "Clear"
       },
       "showAdvanced": "Show advanced options",
-      "hideAdvanced": "Hide advanced options"
+      "hideAdvanced": "Hide advanced options",
+      "ingestors": {
+        "local_file": {
+          "label": "Upload File",
+          "heading": "Upload new document"
+        },
+        "crawler": {
+          "label": "Crawler",
+          "heading": "Add content with Web Crawler"
+        },
+        "url": {
+          "label": "Link",
+          "heading": "Add content from URL"
+        },
+        "github": {
+          "label": "GitHub",
+          "heading": "Add content from GitHub"
+        },
+        "reddit": {
+          "label": "Reddit",
+          "heading": "Add content from Reddit"
+        },
+        "google_drive": {
+          "label": "Google Drive",
+          "heading": "Upload from Google Drive"
+        }
+      },
+      "connectors": {
+        "auth": {
+          "connectedUser": "Connected User",
+          "authFailed": "Authentication failed",
+          "authUrlFailed": "Failed to get authorization URL",
+          "popupBlocked": "Failed to open authentication window. Please allow popups.",
+          "authCancelled": "Authentication was cancelled",
+          "connectedAs": "Connected as {{email}}",
+          "disconnect": "Disconnect"
+        },
+        "googleDrive": {
+          "connect": "Connect to Google Drive",
+          "sessionExpired": "Session expired. Please reconnect to Google Drive.",
+          "sessionExpiredGeneric": "Session expired. Please reconnect your account.",
+          "validateFailed": "Failed to validate session. Please reconnect.",
+          "noSession": "No valid session found. Please reconnect to Google Drive.",
+          "noAccessToken": "No access token available. Please reconnect to Google Drive.",
+          "pickerFailed": "Failed to open file picker. Please try again.",
+          "selectedFiles": "Selected Files",
+          "selectFiles": "Select Files",
+          "loading": "Loading...",
+          "noFilesSelected": "No files or folders selected",
+          "folders": "Folders",
+          "files": "Files",
+          "remove": "Remove",
+          "folderAlt": "Folder",
+          "fileAlt": "File"
+        }
+      }
     },
     "createAPIKey": {
       "label": "Create New API Key",

frontend/src/locale/es.json

@@ -192,6 +192,9 @@
     "uploadDoc": {
       "label": "Subir nuevo documento",
       "select": "Elige cómo cargar tu documento en DocsGPT",
+      "selectSource": "Selecciona la forma de agregar tu fuente",
+      "selectedFiles": "Archivos Seleccionados",
+      "noFilesSelected": "No hay archivos seleccionados",
       "file": "Subir desde el dispositivo",
       "back": "Atrás",
       "wait": "Por favor espera ...",
@@ -220,13 +223,74 @@
       },
       "progress": {
         "upload": "Subida en progreso",
-        "training": "Entrenamiento en progreso",
-        "completed": "Entrenamiento completado",
+        "training": "Subida en progreso",
+        "completed": "Subida completada",
         "wait": "Esto puede tardar varios minutos",
-        "tokenLimit": "Excede el límite de tokens, considere cargar un documento más pequeño"
+        "preparing": "Preparando subida",
+        "tokenLimit": "Excede el límite de tokens, considere cargar un documento más pequeño",
+        "expandDetails": "Expandir detalles de subida",
+        "collapseDetails": "Contraer detalles de subida",
+        "dismiss": "Descartar notificación de subida",
+        "uploadProgress": "Progreso de subida {{progress}}%",
+        "clear": "Limpiar"
       },
       "showAdvanced": "Mostrar opciones avanzadas",
-      "hideAdvanced": "Ocultar opciones avanzadas"
+      "hideAdvanced": "Ocultar opciones avanzadas",
+      "ingestors": {
+        "local_file": {
+          "label": "Subir archivo",
+          "heading": "Subir nuevo documento"
+        },
+        "crawler": {
+          "label": "Rastreador",
+          "heading": "Agregar contenido con rastreador web"
+        },
+        "url": {
+          "label": "Enlace",
+          "heading": "Agregar contenido desde URL"
+        },
+        "github": {
+          "label": "GitHub",
+          "heading": "Agregar contenido desde GitHub"
+        },
+        "reddit": {
+          "label": "Reddit",
+          "heading": "Agregar contenido desde Reddit"
+        },
+        "google_drive": {
+          "label": "Google Drive",
+          "heading": "Subir desde Google Drive"
+        }
+      },
+      "connectors": {
+        "auth": {
+          "connectedUser": "Usuario Conectado",
+          "authFailed": "Autenticación fallida",
+          "authUrlFailed": "Error al obtener la URL de autorización",
+          "popupBlocked": "Error al abrir la ventana de autenticación. Por favor, permita ventanas emergentes.",
+          "authCancelled": "Autenticación cancelada",
+          "connectedAs": "Conectado como {{email}}",
+          "disconnect": "Desconectar"
+        },
+        "googleDrive": {
+          "connect": "Conectar a Google Drive",
+          "sessionExpired": "Sesión expirada. Por favor, reconecte a Google Drive.",
+          "sessionExpiredGeneric": "Sesión expirada. Por favor, reconecte su cuenta.",
+          "validateFailed": "Error al validar la sesión. Por favor, reconecte.",
+          "noSession": "No se encontró una sesión válida. Por favor, reconecte a Google Drive.",
+          "noAccessToken": "No hay token de acceso disponible. Por favor, reconecte a Google Drive.",
+          "pickerFailed": "Error al abrir el selector de archivos. Por favor, inténtelo de nuevo.",
+          "selectedFiles": "Archivos Seleccionados",
+          "selectFiles": "Seleccionar Archivos",
+          "loading": "Cargando...",
+          "noFilesSelected": "No hay archivos o carpetas seleccionados",
+          "folders": "Carpetas",
+          "files": "Archivos",
+          "remove": "Eliminar",
+          "folderAlt": "Carpeta",
+          "fileAlt": "Archivo"
+        }
+      }
     },
     "createAPIKey": {
       "label": "Crear Nueva Clave de API",

frontend/src/locale/jp.json

@@ -192,6 +192,9 @@
     "uploadDoc": {
       "label": "新しい文書をアップロードする",
       "select": "ドキュメントを DocsGPT にアップロードする方法を選択します",
+      "selectSource": "ソースを追加する方法を選択してください",
+      "selectedFiles": "選択されたファイル",
+      "noFilesSelected": "ファイルが選択されていません",
       "file": "デバイスからアップロード",
       "back": "戻る",
       "wait": "お待ちください ...",
@@ -220,13 +223,74 @@
       },
       "progress": {
         "upload": "アップロード中",
-        "training": "トレーニング中",
-        "completed": "トレーニング完了",
+        "training": "アップロード中",
+        "completed": "アップロード完了",
         "wait": "数分かかる場合があります",
-        "tokenLimit": "トークン制限を超えています。より小さいドキュメントをアップロードしてください"
+        "preparing": "アップロードを準備中",
+        "tokenLimit": "トークン制限を超えています。より小さいドキュメントをアップロードしてください",
+        "expandDetails": "アップロードの詳細を展開",
+        "collapseDetails": "アップロードの詳細を折りたたむ",
+        "dismiss": "アップロード通知を閉じる",
+        "uploadProgress": "アップロード進行状況 {{progress}}%",
+        "clear": "クリア"
       },
       "showAdvanced": "詳細オプションを表示",
-      "hideAdvanced": "詳細オプションを非表示"
+      "hideAdvanced": "詳細オプションを非表示",
+      "ingestors": {
+        "local_file": {
+          "label": "ファイルをアップロード",
+          "heading": "新しいドキュメントをアップロード"
+        },
+        "crawler": {
+          "label": "クローラー",
+          "heading": "Webクローラーでコンテンツを追加"
+        },
+        "url": {
+          "label": "リンク",
+          "heading": "URLからコンテンツを追加"
+        },
+        "github": {
+          "label": "GitHub",
+          "heading": "GitHubからコンテンツを追加"
+        },
+        "reddit": {
+          "label": "Reddit",
+          "heading": "Redditからコンテンツを追加"
+        },
+        "google_drive": {
+          "label": "Google Drive",
+          "heading": "Google Driveからアップロード"
+        }
+      },
+      "connectors": {
+        "auth": {
+          "connectedUser": "接続されたユーザー",
+          "authFailed": "認証に失敗しました",
+          "authUrlFailed": "認証URLの取得に失敗しました",
+          "popupBlocked": "認証ウィンドウを開けませんでした。ポップアップを許可してください。",
+          "authCancelled": "認証がキャンセルされました",
+          "connectedAs": "{{email}}として接続",
+          "disconnect": "切断"
+        },
+        "googleDrive": {
+          "connect": "Google Driveに接続",
+          "sessionExpired": "セッションが期限切れです。Google Driveに再接続してください。",
+          "sessionExpiredGeneric": "セッションが期限切れです。アカウントに再接続してください。",
+          "validateFailed": "セッションの検証に失敗しました。再接続してください。",
+          "noSession": "有効なセッションが見つかりません。Google Driveに再接続してください。",
+          "noAccessToken": "アクセストークンが利用できません。Google Driveに再接続してください。",
+          "pickerFailed": "ファイルピッカーを開けませんでした。もう一度お試しください。",
+          "selectedFiles": "選択されたファイル",
+          "selectFiles": "ファイルを選択",
+          "loading": "読み込み中...",
+          "noFilesSelected": "ファイルまたはフォルダが選択されていません",
+          "folders": "フォルダ",
+          "files": "ファイル",
+          "remove": "削除",
+          "folderAlt": "フォルダ",
+          "fileAlt": "ファイル"
+        }
+      }
     },
     "createAPIKey": {
       "label": "新しいAPIキーを作成",

frontend/src/locale/ru.json

@@ -192,6 +192,9 @@
     "uploadDoc": {
       "label": "Загрузить новый документ",
       "select": "Выберите способ загрузки документа в DocsGPT",
+      "selectSource": "Выберите способ добавления источника",
+      "selectedFiles": "Выбранные файлы",
+      "noFilesSelected": "Файлы не выбраны",
       "file": "Загрузить с устройства",
       "back": "Назад",
       "wait": "Пожалуйста, подождите...",
@@ -220,13 +223,74 @@
       },
       "progress": {
         "upload": "Идет загрузка",
-        "training": "Идет обучение",
-        "completed": "Обучение завершено",
+        "training": "Идет загрузка",
+        "completed": "Загрузка завершена",
         "wait": "Это может занять несколько минут",
-        "tokenLimit": "Превышен лимит токенов, рассмотрите возможность загрузки документа меньшего размера"
+        "preparing": "Подготовка загрузки",
+        "tokenLimit": "Превышен лимит токенов, рассмотрите возможность загрузки документа меньшего размера",
+        "expandDetails": "Развернуть детали загрузки",
+        "collapseDetails": "Свернуть детали загрузки",
+        "dismiss": "Закрыть уведомление о загрузке",
+        "uploadProgress": "Прогресс загрузки {{progress}}%",
+        "clear": "Очистить"
       },
       "showAdvanced": "Показать расширенные настройки",
-      "hideAdvanced": "Скрыть расширенные настройки"
+      "hideAdvanced": "Скрыть расширенные настройки",
+      "ingestors": {
+        "local_file": {
+          "label": "Загрузить файл",
+          "heading": "Загрузить новый документ"
+        },
+        "crawler": {
+          "label": "Краулер",
+          "heading": "Добавить контент с помощью веб-краулера"
+        },
+        "url": {
+          "label": "Ссылка",
+          "heading": "Добавить контент из URL"
+        },
+        "github": {
+          "label": "GitHub",
+          "heading": "Добавить контент из GitHub"
+        },
+        "reddit": {
+          "label": "Reddit",
+          "heading": "Добавить контент из Reddit"
+        },
+        "google_drive": {
+          "label": "Google Drive",
+          "heading": "Загрузить из Google Drive"
+        }
+      },
+      "connectors": {
+        "auth": {
+          "connectedUser": "Подключенный пользователь",
+          "authFailed": "Ошибка аутентификации",
+          "authUrlFailed": "Не удалось получить URL авторизации",
+          "popupBlocked": "Не удалось открыть окно аутентификации. Пожалуйста, разрешите всплывающие окна.",
+          "authCancelled": "Аутентификация отменена",
+          "connectedAs": "Подключен как {{email}}",
+          "disconnect": "Отключить"
+        },
+        "googleDrive": {
+          "connect": "Подключиться к Google Drive",
+          "sessionExpired": "Сеанс истек. Пожалуйста, переподключитесь к Google Drive.",
+          "sessionExpiredGeneric": "Сеанс истек. Пожалуйста, переподключите свою учетную запись.",
+          "validateFailed": "Не удалось проверить сеанс. Пожалуйста, переподключитесь.",
+          "noSession": "Действительный сеанс не найден. Пожалуйста, переподключитесь к Google Drive.",
+          "noAccessToken": "Токен доступа недоступен. Пожалуйста, переподключитесь к Google Drive.",
+          "pickerFailed": "Не удалось открыть средство выбора файлов. Пожалуйста, попробуйте еще раз.",
+          "selectedFiles": "Выбранные файлы",
+          "selectFiles": "Выбрать файлы",
+          "loading": "Загрузка...",
+          "noFilesSelected": "Файлы или папки не выбраны",
+          "folders": "Папки",
+          "files": "Файлы",
+          "remove": "Удалить",
+          "folderAlt": "Папка",
+          "fileAlt": "Файл"
+        }
+      }
     },
     "createAPIKey": {
       "label": "Создать новый API ключ",

frontend/src/locale/zh-TW.json

@@ -192,6 +192,9 @@
     "uploadDoc": {
       "label": "上傳新文件",
       "select": "選擇如何將文件上傳到 DocsGPT",
+      "selectSource": "選擇新增來源的方式",
+      "selectedFiles": "已選擇的檔案",
+      "noFilesSelected": "未選擇檔案",
       "file": "從檔案",
       "remote": "遠端",
       "back": "返回",
@@ -220,13 +223,74 @@
       },
       "progress": {
         "upload": "正在上傳",
-        "training": "正在訓練",
-        "completed": "訓練完成",
+        "training": "正在上傳",
+        "completed": "上傳完成",
         "wait": "這可能需要幾分鐘",
-        "tokenLimit": "超出令牌限制，請考慮上傳較小的文檔"
+        "preparing": "準備上傳",
+        "tokenLimit": "超出令牌限制，請考慮上傳較小的文檔",
+        "expandDetails": "展開上傳詳情",
+        "collapseDetails": "摺疊上傳詳情",
+        "dismiss": "關閉上傳通知",
+        "uploadProgress": "上傳進度 {{progress}}%",
+        "clear": "清除"
       },
       "showAdvanced": "顯示進階選項",
-      "hideAdvanced": "隱藏進階選項"
+      "hideAdvanced": "隱藏進階選項",
+      "ingestors": {
+        "local_file": {
+          "label": "上傳檔案",
+          "heading": "上傳新文檔"
+        },
+        "crawler": {
+          "label": "爬蟲",
+          "heading": "使用網路爬蟲新增內容"
+        },
+        "url": {
+          "label": "連結",
+          "heading": "從URL新增內容"
+        },
+        "github": {
+          "label": "GitHub",
+          "heading": "從GitHub新增內容"
+        },
+        "reddit": {
+          "label": "Reddit",
+          "heading": "從Reddit新增內容"
+        },
+        "google_drive": {
+          "label": "Google Drive",
+          "heading": "從Google Drive上傳"
+        }
+      },
+      "connectors": {
+        "auth": {
+          "connectedUser": "已連接使用者",
+          "authFailed": "驗證失敗",
+          "authUrlFailed": "取得授權URL失敗",
+          "popupBlocked": "無法開啟驗證視窗。請允許彈出視窗。",
+          "authCancelled": "驗證已取消",
+          "connectedAs": "已連接為 {{email}}",
+          "disconnect": "中斷連接"
+        },
+        "googleDrive": {
+          "connect": "連接到 Google Drive",
+          "sessionExpired": "工作階段已過期。請重新連接到 Google Drive。",
+          "sessionExpiredGeneric": "工作階段已過期。請重新連接您的帳戶。",
+          "validateFailed": "驗證工作階段失敗。請重新連接。",
+          "noSession": "未找到有效工作階段。請重新連接到 Google Drive。",
+          "noAccessToken": "存取權杖不可用。請重新連接到 Google Drive。",
+          "pickerFailed": "無法開啟檔案選擇器。請重試。",
+          "selectedFiles": "已選擇的檔案",
+          "selectFiles": "選擇檔案",
+          "loading": "載入中...",
+          "noFilesSelected": "未選擇檔案或資料夾",
+          "folders": "資料夾",
+          "files": "檔案",
+          "remove": "移除",
+          "folderAlt": "資料夾",
+          "fileAlt": "檔案"
+        }
+      }
     },
     "createAPIKey": {
       "label": "建立新的 API 金鑰",

frontend/src/locale/zh.json

@@ -192,6 +192,9 @@
     "uploadDoc": {
       "label": "上传新文档",
       "select": "选择如何将文档上传到 DocsGPT",
+      "selectSource": "选择添加源的方式",
+      "selectedFiles": "已选择的文件",
+      "noFilesSelected": "未选择文件",
       "file": "从设备上传",
       "back": "后退",
       "wait": "请稍等 ...",
@@ -220,13 +223,74 @@
       },
       "progress": {
         "upload": "正在上传",
-        "training": "正在训练",
-        "completed": "训练完成",
+        "training": "正在上传",
+        "completed": "上传完成",
         "wait": "这可能需要几分钟",
-        "tokenLimit": "超出令牌限制，请考虑上传较小的文档"
+        "preparing": "准备上传",
+        "tokenLimit": "超出令牌限制，请考虑上传较小的文档",
+        "expandDetails": "展开上传详情",
+        "collapseDetails": "折叠上传详情",
+        "dismiss": "关闭上传通知",
+        "uploadProgress": "上传进度 {{progress}}%",
+        "clear": "清除"
       },
       "showAdvanced": "显示高级选项",
-      "hideAdvanced": "隐藏高级选项"
+      "hideAdvanced": "隐藏高级选项",
+      "ingestors": {
+        "local_file": {
+          "label": "上传文件",
+          "heading": "上传新文档"
+        },
+        "crawler": {
+          "label": "爬虫",
+          "heading": "使用网络爬虫添加内容"
+        },
+        "url": {
+          "label": "链接",
+          "heading": "从URL添加内容"
+        },
+        "github": {
+          "label": "GitHub",
+          "heading": "从GitHub添加内容"
+        },
+        "reddit": {
+          "label": "Reddit",
+          "heading": "从Reddit添加内容"
+        },
+        "google_drive": {
+          "label": "Google Drive",
+          "heading": "从Google Drive上传"
+        }
+      },
+      "connectors": {
+        "auth": {
+          "connectedUser": "已连接用户",
+          "authFailed": "身份验证失败",
+          "authUrlFailed": "获取授权URL失败",
+          "popupBlocked": "无法打开身份验证窗口。请允许弹出窗口。",
+          "authCancelled": "身份验证已取消",
+          "connectedAs": "已连接为 {{email}}",
+          "disconnect": "断开连接"
+        },
+        "googleDrive": {
+          "connect": "连接到 Google Drive",
+          "sessionExpired": "会话已过期。请重新连接到 Google Drive。",
+          "sessionExpiredGeneric": "会话已过期。请重新连接您的账户。",
+          "validateFailed": "验证会话失败。请重新连接。",
+          "noSession": "未找到有效会话。请重新连接到 Google Drive。",
+          "noAccessToken": "访问令牌不可用。请重新连接到 Google Drive。",
+          "pickerFailed": "无法打开文件选择器。请重试。",
+          "selectedFiles": "已选择的文件",
+          "selectFiles": "选择文件",
+          "loading": "加载中...",
+          "noFilesSelected": "未选择文件或文件夹",
+          "folders": "文件夹",
+          "files": "文件",
+          "remove": "删除",
+          "folderAlt": "文件夹",
+          "fileAlt": "文件"
+        }
+      }
     },
     "createAPIKey": {
       "label": "创建新的 API 密钥",

frontend/src/modals/AddActionModal.tsx

@@ -43,7 +43,7 @@ export default function AddActionModal({
       className="sm:w-[512px]"
     >
       <div>
-        <h2 className="px-3 text-xl font-semibold text-jet dark:text-bright-gray">
+        <h2 className="text-jet dark:text-bright-gray px-3 text-xl font-semibold">
           New Action
         </h2>
         <div className="relative mt-6 px-3">
@@ -61,7 +61,7 @@ export default function AddActionModal({
             required={true}
           />
           <p
-            className={`ml-1 mt-2 text-xs italic ${
+            className={`mt-2 ml-1 text-xs italic ${
               functionNameError ? 'text-red-500' : 'text-gray-500'
             }`}
           >
@@ -73,7 +73,7 @@ export default function AddActionModal({
         <div className="mt-3 flex flex-row-reverse gap-1 px-3">
           <button
             onClick={handleAddAction}
-            className="rounded-3xl bg-purple-30 px-5 py-2 text-sm text-white transition-all hover:bg-violets-are-blue"
+            className="bg-purple-30 hover:bg-violets-are-blue rounded-3xl px-5 py-2 text-sm text-white transition-all"
           >
             Add
           </button>
@@ -83,7 +83,7 @@ export default function AddActionModal({
               setModalState('INACTIVE');
               setActionName('');
             }}
-            className="cursor-pointer rounded-3xl px-5 py-2 text-sm font-medium hover:bg-gray-100 dark:bg-transparent dark:text-light-gray dark:hover:bg-[#767183]/50"
+            className="dark:text-light-gray cursor-pointer rounded-3xl px-5 py-2 text-sm font-medium hover:bg-gray-100 dark:bg-transparent dark:hover:bg-[#767183]/50"
           >
             {t('modals.configTool.closeButton')}
           </button>

frontend/src/modals/AgentDetailsModal.tsx

@@ -85,19 +85,19 @@ export default function AgentDetailsModal({
       }}
     >
       <div>
-        <h2 className="text-xl font-semibold text-jet dark:text-bright-gray">
+        <h2 className="text-jet dark:text-bright-gray text-xl font-semibold">
           Access Details
         </h2>
         <div className="mt-8 flex flex-col gap-6">
           <div className="flex flex-col gap-3">
             <div className="flex items-center gap-2">
-              <h2 className="text-base font-semibold text-jet dark:text-bright-gray">
+              <h2 className="text-jet dark:text-bright-gray text-base font-semibold">
                 Public Link
               </h2>
             </div>
             {sharedToken ? (
               <div className="flex flex-col gap-2">
-                <p className="inline break-all font-roboto text-[14px] font-medium leading-normal text-gray-700 dark:text-[#ECECF1]">
+                <p className="font-roboto inline text-[14px] leading-normal font-medium break-all text-gray-700 dark:text-[#ECECF1]">
                   <a
                     href={`${baseURL}/shared/agent/${sharedToken}`}
                     target="_blank"
@@ -113,7 +113,7 @@ export default function AgentDetailsModal({
                 </p>
                 <a
                   href="https://docs.docsgpt.cloud/Agents/basics#core-components-of-an-agent"
-                  className="flex w-fit items-center gap-1 text-purple-30 hover:underline"
+                  className="text-purple-30 flex w-fit items-center gap-1 hover:underline"
                   target="_blank"
                   rel="noopener noreferrer"
                 >
@@ -127,7 +127,7 @@ export default function AgentDetailsModal({
               </div>
             ) : (
               <button
-                className="flex w-28 items-center justify-center rounded-3xl border border-solid border-purple-30 px-5 py-2 text-sm font-medium text-purple-30 transition-colors hover:bg-purple-30 hover:text-white"
+                className="border-purple-30 text-purple-30 hover:bg-purple-30 flex w-28 items-center justify-center rounded-3xl border border-solid px-5 py-2 text-sm font-medium transition-colors hover:text-white"
                 onClick={handleGeneratePublicLink}
               >
                 {loadingStates.publicLink ? (
@@ -139,13 +139,13 @@ export default function AgentDetailsModal({
             )}
           </div>
           <div className="flex flex-col gap-3">
-            <h2 className="text-base font-semibold text-jet dark:text-bright-gray">
+            <h2 className="text-jet dark:text-bright-gray text-base font-semibold">
               API Key
             </h2>
             {apiKey ? (
               <div className="flex flex-col gap-2">
                 <div className="flex items-center gap-2">
-                  <div className="break-all font-roboto text-[14px] font-medium leading-normal text-gray-700 dark:text-[#ECECF1]">
+                  <div className="font-roboto text-[14px] leading-normal font-medium break-all text-gray-700 dark:text-[#ECECF1]">
                     {apiKey}
                     {!apiKey.includes('...') && (
                       <CopyButton
@@ -158,7 +158,7 @@ export default function AgentDetailsModal({
                   {!apiKey.includes('...') && (
                     <a
                       href={`https://widget.docsgpt.cloud/?api-key=${apiKey}`}
-                      className="group ml-8 flex w-[101px] items-center justify-center gap-1 rounded-[62px] border border-purple-30 py-1.5 text-sm font-medium text-purple-30 transition-colors hover:bg-purple-30 hover:text-white"
+                      className="group border-purple-30 text-purple-30 hover:bg-purple-30 ml-8 flex w-[101px] items-center justify-center gap-1 rounded-[62px] border py-1.5 text-sm font-medium transition-colors hover:text-white"
                       target="_blank"
                       rel="noopener noreferrer"
                     >
@@ -173,20 +173,20 @@ export default function AgentDetailsModal({
                 </div>
               </div>
             ) : (
-              <button className="w-28 rounded-3xl border border-solid border-purple-30 px-5 py-2 text-sm font-medium text-purple-30 transition-colors hover:bg-purple-30 hover:text-white">
+              <button className="border-purple-30 text-purple-30 hover:bg-purple-30 w-28 rounded-3xl border border-solid px-5 py-2 text-sm font-medium transition-colors hover:text-white">
                 Generate
               </button>
             )}
           </div>
           <div className="flex flex-col gap-3">
             <div className="flex items-center gap-2">
-              <h2 className="text-base font-semibold text-jet dark:text-bright-gray">
+              <h2 className="text-jet dark:text-bright-gray text-base font-semibold">
                 Webhook URL
               </h2>
             </div>
             {webhookUrl ? (
               <div className="flex flex-col gap-2">
-                <p className="break-all font-roboto text-[14px] font-medium leading-normal text-gray-700 dark:text-[#ECECF1]">
+                <p className="font-roboto text-[14px] leading-normal font-medium break-all text-gray-700 dark:text-[#ECECF1]">
                   <a href={webhookUrl} target="_blank" rel="noreferrer">
                     {webhookUrl}
                   </a>
@@ -198,7 +198,7 @@ export default function AgentDetailsModal({
                 </p>
                 <a
                   href="https://docs.docsgpt.cloud/Agents/basics#core-components-of-an-agent"
-                  className="flex w-fit items-center gap-1 text-purple-30 hover:underline"
+                  className="text-purple-30 flex w-fit items-center gap-1 hover:underline"
                   target="_blank"
                   rel="noopener noreferrer"
                 >
@@ -212,7 +212,7 @@ export default function AgentDetailsModal({
               </div>
             ) : (
               <button
-                className="flex w-28 items-center justify-center rounded-3xl border border-solid border-purple-30 px-5 py-2 text-sm font-medium text-purple-30 transition-colors hover:bg-purple-30 hover:text-white"
+                className="border-purple-30 text-purple-30 hover:bg-purple-30 flex w-28 items-center justify-center rounded-3xl border border-solid px-5 py-2 text-sm font-medium transition-colors hover:text-white"
                 onClick={handleGenerateWebhook}
               >
                 {loadingStates.webhook ? (

frontend/src/modals/ConfigToolModal.tsx

@@ -53,7 +53,7 @@ export default function ConfigToolModal({
   return (
     <WrapperModal close={() => setModalState('INACTIVE')}>
       <div>
-        <h2 className="px-3 text-xl font-semibold text-jet dark:text-bright-gray">
+        <h2 className="text-jet dark:text-bright-gray px-3 text-xl font-semibold">
           {t('modals.configTool.title')}
         </h2>
         <p className="mt-5 px-3 text-sm text-gray-600 dark:text-gray-400">
@@ -85,13 +85,13 @@ export default function ConfigToolModal({
             onClick={() => {
               tool && handleAddTool(tool);
             }}
-            className="rounded-3xl bg-purple-30 px-5 py-2 text-sm text-white transition-all hover:bg-violets-are-blue"
+            className="bg-purple-30 hover:bg-violets-are-blue rounded-3xl px-5 py-2 text-sm text-white transition-all"
           >
             {t('modals.configTool.addButton')}
           </button>
           <button
             onClick={() => setModalState('INACTIVE')}
-            className="cursor-pointer rounded-3xl px-5 py-2 text-sm font-medium hover:bg-gray-100 dark:bg-transparent dark:text-light-gray dark:hover:bg-[#767183]/50"
+            className="dark:text-light-gray cursor-pointer rounded-3xl px-5 py-2 text-sm font-medium hover:bg-gray-100 dark:bg-transparent dark:hover:bg-[#767183]/50"
           >
             {t('modals.configTool.closeButton')}
           </button>

frontend/src/modals/CreateAPIKeyModal.tsx

@@ -76,11 +76,11 @@ export default function CreateAPIKeyModal({
   return (
     <WrapperModal close={close} className="p-4">
       <div className="mb-6">
-        <span className="text-xl text-jet dark:text-bright-gray">
+        <span className="text-jet dark:text-bright-gray text-xl">
           {t('modals.createAPIKey.label')}
         </span>
       </div>
-      <div className="relative mb-4 mt-5">
+      <div className="relative mt-5 mb-4">
         <Input
           type="text"
           className="rounded-md"
@@ -117,7 +117,7 @@ export default function CreateAPIKeyModal({
         />
       </div>
       <div className="my-4">
-        <p className="mb-2 ml-2 font-semibold text-jet dark:text-bright-gray">
+        <p className="text-jet dark:text-bright-gray mb-2 ml-2 font-semibold">
           {t('modals.createAPIKey.chunks')}
         </p>
         <Dropdown
@@ -146,7 +146,7 @@ export default function CreateAPIKeyModal({
             createAPIKey(payload);
           }
         }}
-        className="float-right mt-4 rounded-full bg-purple-30 px-5 py-2 text-sm text-white hover:bg-violets-are-blue disabled:opacity-50"
+        className="bg-purple-30 hover:bg-violets-are-blue float-right mt-4 rounded-full px-5 py-2 text-sm text-white disabled:opacity-50"
       >
         {t('modals.createAPIKey.create')}
       </button>

frontend/src/modals/JWTModal.tsx

@@ -24,11 +24,11 @@ export default function JWTModal({
       close={() => undefined}
     >
       <div className="mb-6">
-        <span className="text-lg text-jet dark:text-bright-gray">
+        <span className="text-jet dark:text-bright-gray text-lg">
           Add JWT Token
         </span>
       </div>
-      <div className="relative mb-4 mt-5">
+      <div className="relative mt-5 mb-4">
         <Input
           name="JWT Token"
           type="text"
@@ -41,7 +41,7 @@ export default function JWTModal({
       <button
         disabled={jwtToken.length === 0}
         onClick={handleTokenSubmit.bind(null, jwtToken)}
-        className="float-right mt-4 rounded-full bg-purple-30 px-5 py-2 text-sm text-white hover:bg-[#6F3FD1] disabled:opacity-50"
+        className="bg-purple-30 float-right mt-4 rounded-full px-5 py-2 text-sm text-white hover:bg-[#6F3FD1] disabled:opacity-50"
       >
         Save Token
       </button>

frontend/src/modals/SaveAPIKeyModal.tsx

@@ -18,23 +18,23 @@ export default function SaveAPIKeyModal({
 
   return (
     <WrapperModal close={close}>
-      <h1 className="my-0 text-xl font-medium text-jet dark:text-bright-gray">
+      <h1 className="text-jet dark:text-bright-gray my-0 text-xl font-medium">
         {t('modals.saveKey.note')}
       </h1>
-      <h3 className="text-sm font-normal text-outer-space dark:text-silver">
+      <h3 className="text-outer-space dark:text-silver text-sm font-normal">
         {t('modals.saveKey.disclaimer')}
       </h3>
       <div className="flex justify-between py-2">
         <div>
-          <h2 className="text-base font-semibold text-jet dark:text-bright-gray">
+          <h2 className="text-jet dark:text-bright-gray text-base font-semibold">
             API Key
           </h2>
-          <span className="text-sm font-normal leading-7 text-jet dark:text-bright-gray">
+          <span className="text-jet dark:text-bright-gray text-sm leading-7 font-normal">
             {apiKey}
           </span>
         </div>
         <button
-          className="my-1 h-10 w-20 rounded-full border border-solid border-violets-are-blue p-2 text-sm text-violets-are-blue hover:bg-violets-are-blue hover:text-white"
+          className="border-violets-are-blue text-violets-are-blue hover:bg-violets-are-blue my-1 h-10 w-20 rounded-full border border-solid p-2 text-sm hover:text-white"
           onClick={handleCopyKey}
         >
           {isCopied ? t('modals.saveKey.copied') : t('modals.saveKey.copy')}
@@ -42,7 +42,7 @@ export default function SaveAPIKeyModal({
       </div>
       <button
         onClick={close}
-        className="rounded-full bg-philippine-yellow px-4 py-3 font-medium text-black hover:bg-[#E6B91A]"
+        className="bg-philippine-yellow rounded-full px-4 py-3 font-medium text-black hover:bg-[#E6B91A]"
       >
         {t('modals.saveKey.confirm')}
       </button>

frontend/src/modals/WrapperModal.tsx

@@ -45,7 +45,7 @@ export default function WrapperModal({
     <div className="fixed top-0 left-0 z-30 flex h-screen w-screen items-center justify-center">
       <div
         ref={modalRef}
-        className={`relative rounded-2xl bg-white dark:bg-[#26272E] p-8 shadow-[0px_4px_40px_-3px_#0000001A] ${className}`}
+        className={`relative rounded-2xl bg-white p-8 shadow-[0px_4px_40px_-3px_#0000001A] dark:bg-[#26272E] ${className}`}
       >
         {!isPerformingTask && (
           <button
@@ -55,7 +55,11 @@ export default function WrapperModal({
             <img className="filter dark:invert" src={Exit} alt="Close" />
           </button>
         )}
-        <div className={`overflow-y-auto no-scrollbar text-[#18181B] dark:text-[#ECECF1] ${contentClassName}`}>{children}</div>
+        <div
+          className={`no-scrollbar overflow-y-auto text-[#18181B] dark:text-[#ECECF1] ${contentClassName}`}
+        >
+          {children}
+        </div>
       </div>
     </div>
   );

frontend/src/settings/Analytics.tsx

@@ -177,9 +177,9 @@ export default function Analytics({ agentId }: AnalyticsProps) {
     <div className="mt-12">
       {/* Messages Analytics */}
       <div className="mt-8 flex w-full flex-col gap-3 [@media(min-width:1080px)]:flex-row">
-        <div className="h-[345px] w-full overflow-hidden rounded-2xl border border-silver px-6 py-5 dark:border-silver/40 [@media(min-width:1080px)]:w-1/2">
+        <div className="border-silver dark:border-silver/40 h-[345px] w-full overflow-hidden rounded-2xl border px-6 py-5 [@media(min-width:1080px)]:w-1/2">
           <div className="flex flex-row items-center justify-start gap-3">
-            <p className="font-bold text-jet dark:text-bright-gray">
+            <p className="text-jet dark:text-bright-gray font-bold">
               {t('settings.analytics.messages')}
             </p>
             <Dropdown
@@ -225,9 +225,9 @@ export default function Analytics({ agentId }: AnalyticsProps) {
         </div>
 
         {/* Token Usage Analytics */}
-        <div className="h-[345px] w-full overflow-hidden rounded-2xl border border-silver px-6 py-5 dark:border-silver/40 [@media(min-width:1080px)]:w-1/2">
+        <div className="border-silver dark:border-silver/40 h-[345px] w-full overflow-hidden rounded-2xl border px-6 py-5 [@media(min-width:1080px)]:w-1/2">
           <div className="flex flex-row items-center justify-start gap-3">
-            <p className="font-bold text-jet dark:text-bright-gray">
+            <p className="text-jet dark:text-bright-gray font-bold">
               {t('settings.analytics.tokenUsage')}
             </p>
             <Dropdown
@@ -275,9 +275,9 @@ export default function Analytics({ agentId }: AnalyticsProps) {
 
       {/* Feedback Analytics */}
       <div className="mt-8 flex w-full flex-col gap-3">
-        <div className="h-[345px] w-full overflow-hidden rounded-2xl border border-silver px-6 py-5 dark:border-silver/40">
+        <div className="border-silver dark:border-silver/40 h-[345px] w-full overflow-hidden rounded-2xl border px-6 py-5">
           <div className="flex flex-row items-center justify-start gap-3">
-            <p className="font-bold text-jet dark:text-bright-gray">
+            <p className="text-jet dark:text-bright-gray font-bold">
               {t('settings.analytics.userFeedback')}
             </p>
             <Dropdown

frontend/src/settings/Tools.tsx

@@ -171,7 +171,7 @@ export default function Tools() {
                 />
               </div>
               <button
-                className="flex h-[32px] min-w-[108px] items-center justify-center whitespace-normal rounded-full bg-purple-30 px-4 text-sm text-white hover:bg-violets-are-blue"
+                className="bg-purple-30 hover:bg-violets-are-blue flex h-[32px] min-w-[108px] items-center justify-center rounded-full px-4 text-sm whitespace-normal text-white"
                 onClick={() => {
                   setAddToolModalState('ACTIVE');
                 }}
@@ -179,7 +179,7 @@ export default function Tools() {
                 {t('settings.tools.addTool')}
               </button>
             </div>
-            <div className="mb-8 mt-5 border-b border-light-silver dark:border-dim-gray" />
+            <div className="border-light-silver dark:border-dim-gray mt-5 mb-8 border-b" />
             {loading ? (
               <div className="grid grid-cols-2 gap-6 lg:grid-cols-3">
                 <div className="col-span-2 mt-24 flex h-32 items-center justify-center lg:col-span-3">
@@ -219,7 +219,7 @@ export default function Tools() {
                               activeMenuId === tool.id ? null : tool.id,
                             );
                           }}
-                          className="absolute right-4 top-4 z-10 cursor-pointer"
+                          className="absolute top-4 right-4 z-10 cursor-pointer"
                         >
                           <img
                             src={ThreeDotsIcon}
@@ -248,16 +248,16 @@ export default function Tools() {
                           <div className="mt-[9px]">
                             <p
                               title={tool.customName || tool.displayName}
-                              className="truncate px-1 text-[13px] font-semibold capitalize leading-relaxed text-raisin-black-light dark:text-bright-gray"
+                              className="text-raisin-black-light dark:text-bright-gray truncate px-1 text-[13px] leading-relaxed font-semibold capitalize"
                             >
                               {tool.customName || tool.displayName}
                             </p>
-                            <p className="mt-1 h-24 overflow-auto px-1 text-[12px] leading-relaxed text-old-silver dark:text-sonic-silver-light">
+                            <p className="text-old-silver dark:text-sonic-silver-light mt-1 h-24 overflow-auto px-1 text-[12px] leading-relaxed">
                               {tool.description}
                             </p>
                           </div>
                         </div>
-                        <div className="absolute bottom-4 right-4">
+                        <div className="absolute right-4 bottom-4">
                           <ToggleSwitch
                             checked={tool.status}
                             onChange={(checked) =>

frontend/src/settings/Widgets.tsx

@@ -50,7 +50,7 @@ const Widgets: React.FC<{
   return (
     <div>
       <div className="mt-[59px]">
-        <p className="font-bold text-jet">Widget Source</p>
+        <p className="text-jet font-bold">Widget Source</p>
         <Dropdown
           options={widgetSources}
           selectedValue={selectedWidgetSource}
@@ -58,7 +58,7 @@ const Widgets: React.FC<{
         />
       </div>
       <div className="mt-5">
-        <p className="font-bold text-jet">Widget Method</p>
+        <p className="text-jet font-bold">Widget Method</p>
         <Dropdown
           options={widgetMethods}
           selectedValue={selectedWidgetMethod}
@@ -66,7 +66,7 @@ const Widgets: React.FC<{
         />
       </div>
       <div className="mt-5">
-        <p className="font-bold text-jet">Widget Type</p>
+        <p className="text-jet font-bold">Widget Type</p>
         <Dropdown
           options={widgetTypes}
           selectedValue={selectedWidgetType}
@@ -74,7 +74,7 @@ const Widgets: React.FC<{
         />
       </div>
       <div className="mt-6">
-        <p className="font-bold text-jet">Widget Code Snippet</p>
+        <p className="text-jet font-bold">Widget Code Snippet</p>
         <textarea
           rows={4}
           value={widgetCode}

frontend/src/settings/index.tsx

@@ -38,8 +38,7 @@ export default function Settings() {
 
   const getActiveTabFromPath = () => {
     const path = location.pathname;
-    if (path.includes('/settings/sources'))
-      return t('settings.sources.label');
+    if (path.includes('/settings/sources')) return t('settings.sources.label');
     if (path.includes('/settings/analytics'))
       return t('settings.analytics.label');
     if (path.includes('/settings/logs')) return t('settings.logs.label');
@@ -53,8 +52,7 @@ export default function Settings() {
   const handleTabChange = (tab: string) => {
     setActiveTab(tab);
     if (tab === t('settings.general.label')) navigate('/settings');
-    else if (tab === t('settings.sources.label'))
-      navigate('/settings/sources');
+    else if (tab === t('settings.sources.label')) navigate('/settings/sources');
     else if (tab === t('settings.analytics.label'))
       navigate('/settings/analytics');
     else if (tab === t('settings.logs.label')) navigate('/settings/logs');
@@ -103,7 +101,7 @@ export default function Settings() {
 
   return (
     <div className="h-full overflow-auto p-4 md:p-12">
-      <p className="text-2xl font-bold text-eerie-black dark:text-bright-gray">
+      <p className="text-eerie-black dark:text-bright-gray text-2xl font-bold">
         {t('settings.label')}
       </p>
       <SettingsBar

frontend/src/upload/Upload.tsx

@@ -1,4 +1,5 @@
-import { useCallback, useEffect, useRef, useState } from 'react';
+import { useCallback, useState } from 'react';
+import { nanoid } from '@reduxjs/toolkit';
 import { useDropzone } from 'react-dropzone';
 import { useTranslation } from 'react-i18next';
 import { useDispatch, useSelector } from 'react-redux';
@@ -9,7 +10,7 @@ import Dropdown from '../components/Dropdown';
 import Input from '../components/Input';
 import ToggleSwitch from '../components/ToggleSwitch';
 import WrapperModal from '../modals/WrapperModal';
-import { ActiveState, Doc } from '../models/misc';  
+import { ActiveState, Doc } from '../models/misc';
 
 import { getDocs } from '../preferences/preferenceApi';
 import {
@@ -18,14 +19,17 @@ import {
   setSelectedDocs,
   setSourceDocs,
 } from '../preferences/preferenceSlice';
-import { IngestorDefaultConfigs, IngestorFormSchemas, getIngestorSchema, IngestorOption } from '../upload/types/ingestor';
 import {
-  FormField,
-  IngestorConfig,
-  IngestorType,
-} from './types/ingestor';
+  IngestorDefaultConfigs,
+  IngestorFormSchemas,
+  getIngestorSchema,
+  IngestorOption,
+} from '../upload/types/ingestor';
+import { addUploadTask, updateUploadTask } from './uploadSlice';
 
-import {FilePicker}  from '../components/FilePicker';
+import { FormField, IngestorConfig, IngestorType } from './types/ingestor';
+
+import { FilePicker } from '../components/FilePicker';
 import GoogleDrivePicker from '../components/GoogleDrivePicker';
 
 import ChevronRight from '../assets/chevron-right.svg';
@@ -46,7 +50,7 @@ function Upload({
   onSuccessfulUpload?: () => void;
 }) {
   const token = useSelector(selectToken);
-  
+
   const [files, setfiles] = useState<File[]>(receivedFile);
   const [activeTab, setActiveTab] = useState<boolean>(true);
   const [showAdvancedOptions, setShowAdvancedOptions] = useState(false);
@@ -55,9 +59,6 @@ function Upload({
   const [selectedFiles, setSelectedFiles] = useState<string[]>([]);
   const [selectedFolders, setSelectedFolders] = useState<string[]>([]);
 
-
-  
-
   const renderFormFields = () => {
     if (!ingestor.type) return null;
     const ingestorSchema = getIngestorSchema(ingestor.type as IngestorType);
@@ -190,20 +191,20 @@ function Upload({
         return (
           <div key={field.name}>
             <div className="mb-3" {...getRootProps()}>
-              <span className="inline-block text-purple-30 dark:text-silver rounded-3xl border border-[#7F7F82] bg-transparent px-4 py-2 font-medium hover:cursor-pointer">
+              <span className="text-purple-30 dark:text-silver inline-block rounded-3xl border border-[#7F7F82] bg-transparent px-4 py-2 font-medium hover:cursor-pointer">
                 <input type="button" {...getInputProps()} />
-                Choose Files
+                {t('modals.uploadDoc.choose')}
               </span>
             </div>
             <div className="mt-4 max-w-full">
               <p className="text-eerie-black dark:text-light-gray mb-[14px] text-[14px] font-medium">
-                Selected Files
+                {t('modals.uploadDoc.selectedFiles')}
               </p>
               <div className="max-w-full overflow-hidden">
                 {files.map((file) => (
                   <p
                     key={file.name}
-                    className="text-gray-6000 dark:text-[#ececf1] truncate overflow-hidden text-ellipsis"
+                    className="text-gray-6000 truncate overflow-hidden text-ellipsis dark:text-[#ececf1]"
                     title={file.name}
                   >
                     {file.name}
@@ -211,7 +212,7 @@ function Upload({
                 ))}
                 {files.length === 0 && (
                   <p className="text-gray-6000 dark:text-light-gray text-[14px]">
-                    No files selected
+                    {t('modals.uploadDoc.noFilesSelected')}
                   </p>
                 )}
               </div>
@@ -222,7 +223,10 @@ function Upload({
         return (
           <FilePicker
             key={field.name}
-            onSelectionChange={(selectedFileIds: string[], selectedFolderIds: string[] = []) => {
+            onSelectionChange={(
+              selectedFileIds: string[],
+              selectedFolderIds: string[] = [],
+            ) => {
               setSelectedFiles(selectedFileIds);
               setSelectedFolders(selectedFolderIds);
             }}
@@ -236,7 +240,10 @@ function Upload({
         return (
           <GoogleDrivePicker
             key={field.name}
-            onSelectionChange={(selectedFileIds: string[], selectedFolderIds: string[] = []) => {
+            onSelectionChange={(
+              selectedFileIds: string[],
+              selectedFolderIds: string[] = [],
+            ) => {
               setSelectedFiles(selectedFileIds);
               setSelectedFolders(selectedFolderIds);
             }}
@@ -255,227 +262,156 @@ function Upload({
     config: {},
   }));
 
-  const [progress, setProgress] = useState<{
-    type: 'UPLOAD' | 'TRAINING';
-    percentage: number;
-    taskId?: string;
-    failed?: boolean;
-  }>();
-
   const { t } = useTranslation();
-  const setTimeoutRef = useRef<number | null>(null);
+  const dispatch = useDispatch();
 
-  const ingestorOptions: IngestorOption[] = IngestorFormSchemas
-    .filter(schema => schema.validate ? schema.validate() : true)
-    .map(schema => ({
-      label: schema.label,
-      value: schema.key,
-      icon: schema.icon,
-      heading: schema.heading
-    }));
+  const ingestorOptions: IngestorOption[] = IngestorFormSchemas.filter(
+    (schema) => (schema.validate ? schema.validate() : true),
+  ).map((schema) => ({
+    label: schema.label,
+    value: schema.key,
+    icon: schema.icon,
+    heading: schema.heading,
+  }));
 
   const sourceDocs = useSelector(selectSourceDocs);
-  useEffect(() => {
-    if (setTimeoutRef.current) {
-      clearTimeout(setTimeoutRef.current);
-    }
+
+  const resetUploaderState = useCallback(() => {
+    setIngestor({ type: null, name: '', config: {} });
+    setfiles([]);
+    setSelectedFiles([]);
+    setSelectedFolders([]);
+    setShowAdvancedOptions(false);
   }, []);
 
-  function ProgressBar({ progressPercent }: { progressPercent: number }) {
-    return (
-      <div className="my-8 flex h-full w-full items-center justify-center">
-        <div className="relative h-32 w-32 rounded-full">
-          <div className="absolute inset-0 rounded-full shadow-[0_0_10px_2px_rgba(0,0,0,0.3)_inset] dark:shadow-[0_0_10px_2px_rgba(0,0,0,0.3)_inset]"></div>
-          <div
-            className={`absolute inset-0 rounded-full ${progressPercent === 100 ? 'bg-linear-to-r from-white to-gray-400 shadow-xl shadow-lime-300/50 dark:bg-linear-to-br dark:from-gray-500 dark:to-gray-300 dark:shadow-lime-300/50' : 'shadow-[0_4px_0_#7D54D1] dark:shadow-[0_4px_0_#7D54D1]'}`}
-            style={{
-              animation: `${progressPercent === 100 ? 'none' : 'rotate 2s linear infinite'}`,
-            }}
-          ></div>
-          <div className="absolute inset-0 flex items-center justify-center">
-            <span className="text-2xl font-bold">{progressPercent}%</span>
-          </div>
-          <style>
-            {`@keyframes rotate {
-                0% { transform: rotate(0deg); }
-                100%{ transform: rotate(360deg); }
-              }`}
-          </style>
-        </div>
-      </div>
-    );
-  }
+  const handleTaskFailure = useCallback(
+    (clientTaskId: string, errorMessage?: string) => {
+      dispatch(
+        updateUploadTask({
+          id: clientTaskId,
+          updates: {
+            status: 'failed',
+            errorMessage: errorMessage || t('attachments.uploadFailed'),
+          },
+        }),
+      );
+    },
+    [dispatch, t],
+  );
 
-  function Progress({
-    title,
-    isCancellable = false,
-    isFailed = false,
-    isTraining = false,
-  }: {
-    title: string;
-    isCancellable?: boolean;
-    isFailed?: boolean;
-    isTraining?: boolean;
-  }) {
-    return (
-      <div className="text-gray-2000 dark:text-bright-gray mt-5 flex flex-col items-center gap-2">
-        <p className="text-gra text-xl tracking-[0.15px]">
-          {isTraining &&
-            (progress?.percentage === 100
-              ? t('modals.uploadDoc.progress.completed')
-              : title)}
-          {!isTraining && title}
-        </p>
-        <p className="text-sm">{t('modals.uploadDoc.progress.wait')}</p>
-        <p className={`ml-5 text-xl text-red-400 ${isFailed ? '' : 'hidden'}`}>
-          {t('modals.uploadDoc.progress.tokenLimit')}
-        </p>
-        {/* <p className="mt-10 text-2xl">{progress?.percentage || 0}%</p> */}
-        <ProgressBar progressPercent={progress?.percentage || 0} />
-        {isTraining &&
-          (progress?.percentage === 100 ? (
-            <button
-              onClick={() => {
-                setIngestor({ type: null, name: '', config: {} });
-                setfiles([]);
-                setProgress(undefined);
-                setModalState('INACTIVE');
-              }}
-              className="h-[42px] cursor-pointer rounded-3xl bg-[#7D54D1] px-[28px] py-[6px] text-sm text-white shadow-lg hover:bg-[#6F3FD1]"
-            >
-              {t('modals.uploadDoc.start')}
-            </button>
-          ) : (
-            <button
-              className="ml-2 h-[42px] cursor-pointer rounded-3xl bg-[#7D54D14D] px-[28px] py-[6px] text-sm text-white shadow-lg"
-              disabled
-            >
-              {t('modals.uploadDoc.wait')}
-            </button>
-          ))}
-      </div>
-    );
-  }
+  const trackTraining = useCallback(
+    (backendTaskId: string, clientTaskId: string) => {
+      let timeoutId: number | null = null;
 
-  function UploadProgress() {
-    return <Progress title={t('modals.uploadDoc.progress.upload')}></Progress>;
-  }
-
-  function TrainingProgress() {
-    const dispatch = useDispatch();
-
-    useEffect(() => {
-      let timeoutID: number | undefined;
-
-      if ((progress?.percentage ?? 0) < 100) {
-        timeoutID = setTimeout(() => {
-          userService
-            .getTaskStatus(progress?.taskId as string, null)
-            .then((data) => data.json())
-            .then((data) => {
-              if (data.status == 'SUCCESS') {
-                if (data.result.limited === true) {
-                  getDocs(token).then((data) => {
-                    dispatch(setSourceDocs(data));
-                    dispatch(
-                      setSelectedDocs(
-                        Array.isArray(data) &&
-                          data?.find(
-                            (d: Doc) => d.type?.toLowerCase() === 'local',
-                          ),
-                    ));
-                  });
-                  setProgress(
-                    (progress) =>
-                      progress && {
-                        ...progress,
-                        percentage: 100,
-                        failed: true,
-                      },
-                  );
-                } else {
-                  getDocs(token).then((data) => {
-                    dispatch(setSourceDocs(data));
-                    const docIds = new Set(
-                      (Array.isArray(sourceDocs) &&
-                        sourceDocs?.map((doc: Doc) =>
-                          doc.id ? doc.id : null,
-                        )) ||
-                        [],
-                    );
-                    if (data && Array.isArray(data)) {
-                      data.map((updatedDoc: Doc) => {
-                        if (updatedDoc.id && !docIds.has(updatedDoc.id)) {
-                          // Select the doc not present in the intersection of current Docs and fetched data
-                          dispatch(setSelectedDocs(updatedDoc));
-                          return;
-                        }
-                      });
-                    }
-                  });
-                  setProgress(
-                    (progress) =>
-                      progress && {
-                        ...progress,
-                        percentage: 100,
-                        failed: false,
-                      },
-                  );
-                  setIngestor({ type: null, name: '', config: {} });
-                  setfiles([]);
-                  setProgress(undefined);
-                  setModalState('INACTIVE');
-                  onSuccessfulUpload?.();
-                }
-              } else if (data.status == 'PROGRESS') {
-                setProgress(
-                  (progress) =>
-                    progress && {
-                      ...progress,
-                      percentage: data.result.current,
-                    },
-                );
+      const poll = () => {
+        userService
+          .getTaskStatus(backendTaskId, null)
+          .then((response) => response.json())
+          .then(async (data) => {
+            if (data.status === 'SUCCESS') {
+              if (timeoutId !== null) {
+                clearTimeout(timeoutId);
+                timeoutId = null;
               }
-            });
-        }, 5000);
-      }
 
-      // cleanup
-      return () => {
-        if (timeoutID !== undefined) {
-          clearTimeout(timeoutID);
-        }
+              const docs = await getDocs(token);
+              dispatch(setSourceDocs(docs));
+
+              if (Array.isArray(docs)) {
+                const existingDocIds = new Set(
+                  (Array.isArray(sourceDocs) ? sourceDocs : [])
+                    .map((doc: Doc) => doc?.id)
+                    .filter((id): id is string => Boolean(id)),
+                );
+                const newDoc = docs.find(
+                  (doc: Doc) => doc.id && !existingDocIds.has(doc.id),
+                );
+                if (newDoc) {
+                  dispatch(setSelectedDocs([newDoc]));
+                }
+              }
+
+              if (data.result?.limited) {
+                dispatch(
+                  updateUploadTask({
+                    id: clientTaskId,
+                    updates: {
+                      status: 'failed',
+                      progress: 100,
+                      errorMessage: t('modals.uploadDoc.progress.tokenLimit'),
+                    },
+                  }),
+                );
+              } else {
+                dispatch(
+                  updateUploadTask({
+                    id: clientTaskId,
+                    updates: {
+                      status: 'completed',
+                      progress: 100,
+                      errorMessage: undefined,
+                    },
+                  }),
+                );
+                onSuccessfulUpload?.();
+              }
+            } else if (data.status === 'FAILURE') {
+              if (timeoutId !== null) {
+                clearTimeout(timeoutId);
+                timeoutId = null;
+              }
+              handleTaskFailure(clientTaskId, data.result?.message);
+            } else if (data.status === 'PROGRESS') {
+              dispatch(
+                updateUploadTask({
+                  id: clientTaskId,
+                  updates: {
+                    status: 'training',
+                    progress: Math.min(100, data.result?.current ?? 0),
+                  },
+                }),
+              );
+              timeoutId = window.setTimeout(poll, 5000);
+            } else {
+              timeoutId = window.setTimeout(poll, 5000);
+            }
+          })
+          .catch(() => {
+            if (timeoutId !== null) {
+              clearTimeout(timeoutId);
+              timeoutId = null;
+            }
+            handleTaskFailure(clientTaskId);
+          });
       };
-    }, [progress, dispatch]);
-    return (
-      <Progress
-        title={t('modals.uploadDoc.progress.training')}
-        isCancellable={progress?.percentage === 100}
-        isFailed={progress?.failed === true}
-        isTraining={true}
-      ></Progress>
-    );
-  }
 
-  const onDrop = useCallback((acceptedFiles: File[]) => {
-    setfiles(acceptedFiles);
-    setIngestor(prev => ({ ...prev, name: acceptedFiles[0]?.name || '' }));
+      timeoutId = window.setTimeout(poll, 3000);
+    },
+    [dispatch, handleTaskFailure, onSuccessfulUpload, sourceDocs, t, token],
+  );
 
-    // If we're in local_file mode, update the ingestor config
-    if (ingestor.type === 'local_file') {
-      setIngestor((prevState) => ({
-        ...prevState,
-        config: {
-          ...prevState.config,
-          files: acceptedFiles,
-        },
-      }));
-    }
-  }, [ingestor.type]);
+  const onDrop = useCallback(
+    (acceptedFiles: File[]) => {
+      setfiles(acceptedFiles);
+      setIngestor((prev) => ({ ...prev, name: acceptedFiles[0]?.name || '' }));
+
+      // If we're in local_file mode, update the ingestor config
+      if (ingestor.type === 'local_file') {
+        setIngestor((prevState) => ({
+          ...prevState,
+          config: {
+            ...prevState.config,
+            files: acceptedFiles,
+          },
+        }));
+      }
+    },
+    [ingestor.type],
+  );
 
   const doNothing = () => undefined;
 
-  const uploadFile = () => {
+  const uploadFile = (clientTaskId: string) => {
     const formData = new FormData();
     files.forEach((file) => {
       formData.append('file', file);
@@ -483,44 +419,106 @@ function Upload({
 
     formData.append('name', ingestor.name);
     formData.append('user', 'local');
+
     const apiHost = import.meta.env.VITE_API_HOST;
     const xhr = new XMLHttpRequest();
+
+    dispatch(
+      updateUploadTask({
+        id: clientTaskId,
+        updates: { status: 'uploading', progress: 0 },
+      }),
+    );
+
     xhr.upload.addEventListener('progress', (event) => {
-      const progress = +((event.loaded / event.total) * 100).toFixed(2);
-      setProgress({ type: 'UPLOAD', percentage: progress });
+      if (!event.lengthComputable) return;
+      const progressPercentage = Number(
+        ((event.loaded / event.total) * 100).toFixed(2),
+      );
+      dispatch(
+        updateUploadTask({
+          id: clientTaskId,
+          updates: { progress: progressPercentage },
+        }),
+      );
     });
+
     xhr.onload = () => {
-      const { task_id } = JSON.parse(xhr.responseText);
-      setTimeoutRef.current = setTimeout(() => {
-        setProgress({ type: 'TRAINING', percentage: 0, taskId: task_id });
-      }, 3000);
+      if (xhr.status >= 200 && xhr.status < 300) {
+        try {
+          const parsed = JSON.parse(xhr.responseText) as { task_id?: string };
+          if (parsed.task_id) {
+            dispatch(
+              updateUploadTask({
+                id: clientTaskId,
+                updates: {
+                  taskId: parsed.task_id,
+                  status: 'training',
+                  progress: 0,
+                },
+              }),
+            );
+            trackTraining(parsed.task_id, clientTaskId);
+          } else {
+            dispatch(
+              updateUploadTask({
+                id: clientTaskId,
+                updates: { status: 'completed', progress: 100 },
+              }),
+            );
+            onSuccessfulUpload?.();
+          }
+        } catch (error) {
+          handleTaskFailure(clientTaskId);
+        }
+      } else {
+        handleTaskFailure(clientTaskId, xhr.statusText || undefined);
+      }
     };
-    xhr.open('POST', `${apiHost + '/api/upload'}`);
+
+    xhr.onerror = () => {
+      handleTaskFailure(clientTaskId);
+    };
+
+    xhr.open('POST', `${apiHost}/api/upload`);
     xhr.setRequestHeader('Authorization', `Bearer ${token}`);
     xhr.send(formData);
   };
 
-  const uploadRemote = () => {
-    if (!ingestor.type) return;
+  const uploadRemote = (clientTaskId: string) => {
+    if (!ingestor.type) {
+      handleTaskFailure(clientTaskId);
+      return;
+    }
+
     const formData = new FormData();
     formData.append('name', ingestor.name);
     formData.append('user', 'local');
     formData.append('source', ingestor.type as string);
 
-    let configData: any = {};
-
     const ingestorSchema = getIngestorSchema(ingestor.type as IngestorType);
-    if (!ingestorSchema) return;
+    if (!ingestorSchema) {
+      handleTaskFailure(clientTaskId);
+      return;
+    }
+
     const schema: FormField[] = ingestorSchema.fields;
-    const hasLocalFilePicker = schema.some((field: FormField) => field.type === 'local_file_picker');
-    const hasRemoteFilePicker = schema.some((field: FormField) => field.type === 'remote_file_picker');
-    const hasGoogleDrivePicker = schema.some((field: FormField) => field.type === 'google_drive_picker');
+    const hasLocalFilePicker = schema.some(
+      (field: FormField) => field.type === 'local_file_picker',
+    );
+    const hasRemoteFilePicker = schema.some(
+      (field: FormField) => field.type === 'remote_file_picker',
+    );
+    const hasGoogleDrivePicker = schema.some(
+      (field: FormField) => field.type === 'google_drive_picker',
+    );
+
+    let configData: Record<string, unknown> = { ...ingestor.config };
 
     if (hasLocalFilePicker) {
       files.forEach((file) => {
         formData.append('file', file);
       });
-      configData = { ...ingestor.config };
     } else if (hasRemoteFilePicker || hasGoogleDrivePicker) {
       const sessionToken = getSessionToken(ingestor.type as string);
       configData = {
@@ -529,42 +527,121 @@ function Upload({
         file_ids: selectedFiles,
         folder_ids: selectedFolders,
       };
-    } else {
-      configData = { ...ingestor.config };
     }
 
     formData.append('data', JSON.stringify(configData));
 
     const apiHost: string = import.meta.env.VITE_API_HOST;
+    const endpoint =
+      ingestor.type === 'local_file'
+        ? `${apiHost}/api/upload`
+        : `${apiHost}/api/remote`;
+
     const xhr = new XMLHttpRequest();
+
+    dispatch(
+      updateUploadTask({
+        id: clientTaskId,
+        updates: { status: 'uploading', progress: 0 },
+      }),
+    );
+
     xhr.upload.addEventListener('progress', (event: ProgressEvent) => {
-      if (event.lengthComputable) {
-        const progressPercentage = +(
-          (event.loaded / event.total) *
-          100
-        ).toFixed(2);
-        setProgress({ type: 'UPLOAD', percentage: progressPercentage });
-      }
+      if (!event.lengthComputable) return;
+      const progressPercentage = Number(
+        ((event.loaded / event.total) * 100).toFixed(2),
+      );
+      dispatch(
+        updateUploadTask({
+          id: clientTaskId,
+          updates: { progress: progressPercentage },
+        }),
+      );
     });
+
     xhr.onload = () => {
-      const response = JSON.parse(xhr.responseText) as { task_id: string };
-      setTimeoutRef.current = window.setTimeout(() => {
-        setProgress({
-          type: 'TRAINING',
-          percentage: 0,
-          taskId: response.task_id,
-        });
-      }, 3000);
+      if (xhr.status >= 200 && xhr.status < 300) {
+        try {
+          const response = JSON.parse(xhr.responseText) as { task_id?: string };
+          if (response.task_id) {
+            dispatch(
+              updateUploadTask({
+                id: clientTaskId,
+                updates: {
+                  taskId: response.task_id,
+                  status: 'training',
+                  progress: 0,
+                },
+              }),
+            );
+            trackTraining(response.task_id, clientTaskId);
+          } else {
+            dispatch(
+              updateUploadTask({
+                id: clientTaskId,
+                updates: { status: 'completed', progress: 100 },
+              }),
+            );
+            onSuccessfulUpload?.();
+          }
+        } catch (error) {
+          handleTaskFailure(clientTaskId);
+        }
+      } else {
+        handleTaskFailure(clientTaskId, xhr.statusText || undefined);
+      }
     };
 
-    const endpoint = ingestor.type === 'local_file' ? `${apiHost}/api/upload` : `${apiHost}/api/remote`;
+    xhr.onerror = () => {
+      handleTaskFailure(clientTaskId);
+    };
 
     xhr.open('POST', endpoint);
     xhr.setRequestHeader('Authorization', `Bearer ${token}`);
     xhr.send(formData);
   };
 
-  
+  const handleClose = useCallback(() => {
+    resetUploaderState();
+    setModalState('INACTIVE');
+    close();
+  }, [close, resetUploaderState, setModalState]);
+
+  const handleUpload = () => {
+    if (!ingestor.type) return;
+
+    const ingestorSchemaForUpload = getIngestorSchema(
+      ingestor.type as IngestorType,
+    );
+    if (!ingestorSchemaForUpload) return;
+
+    const schema: FormField[] = ingestorSchemaForUpload.fields;
+    const hasLocalFilePicker = schema.some(
+      (field: FormField) => field.type === 'local_file_picker',
+    );
+
+    const displayName =
+      ingestor.name?.trim() || files[0]?.name || t('modals.uploadDoc.label');
+
+    const clientTaskId = nanoid();
+
+    dispatch(
+      addUploadTask({
+        id: clientTaskId,
+        fileName: displayName,
+        progress: 0,
+        status: 'preparing',
+      }),
+    );
+
+    if (hasLocalFilePicker) {
+      uploadFile(clientTaskId);
+    } else {
+      uploadRemote(clientTaskId);
+    }
+
+    handleClose();
+  };
 
   const { getRootProps, getInputProps } = useDropzone({
     onDrop,
@@ -604,12 +681,20 @@ function Upload({
     }
 
     if (!ingestor.type) return true;
-    const ingestorSchemaForValidation = getIngestorSchema(ingestor.type as IngestorType);
+    const ingestorSchemaForValidation = getIngestorSchema(
+      ingestor.type as IngestorType,
+    );
     if (!ingestorSchemaForValidation) return true;
     const schema: FormField[] = ingestorSchemaForValidation.fields;
-    const hasLocalFilePicker = schema.some((field: FormField) => field.type === 'local_file_picker');
-    const hasRemoteFilePicker = schema.some((field: FormField) => field.type === 'remote_file_picker');
-    const hasGoogleDrivePicker = schema.some((field: FormField) => field.type === 'google_drive_picker');
+    const hasLocalFilePicker = schema.some(
+      (field: FormField) => field.type === 'local_file_picker',
+    );
+    const hasRemoteFilePicker = schema.some(
+      (field: FormField) => field.type === 'remote_file_picker',
+    );
+    const hasGoogleDrivePicker = schema.some(
+      (field: FormField) => field.type === 'google_drive_picker',
+    );
 
     if (hasLocalFilePicker) {
       if (files.length === 0) {
@@ -621,7 +706,9 @@ function Upload({
       }
     }
 
-    const ingestorSchemaForFields = getIngestorSchema(ingestor.type as IngestorType);
+    const ingestorSchemaForFields = getIngestorSchema(
+      ingestor.type as IngestorType,
+    );
     if (!ingestorSchemaForFields) return false;
     const formFields: FormField[] = ingestorSchemaForFields.fields;
     for (const field of formFields) {
@@ -686,27 +773,29 @@ function Upload({
 
   const renderIngestorSelection = () => {
     return (
-      <div className="grid grid-cols-1 sm:grid-cols-2 md:grid-cols-3 gap-4 w-full">
+      <div className="grid w-full grid-cols-1 gap-4 sm:grid-cols-2 md:grid-cols-3">
         {ingestorOptions.map((option) => (
           <div
             key={option.value}
-            className={`relative flex flex-col justify-between rounded-2xl cursor-pointer w-full h-[91.2px] border border-solid pt-[21.1px] pr-[21px] pb-[15px] pl-[21px] gap-2 transition-colors duration-300 ease-out mx-auto ${
-              ingestor.type === option.value 
-                ? 'bg-[#7D54D1] text-white border-[#7D54D1]' 
-                : 'bg-transparent hover:bg-[#ECECEC]/30 dark:hover:bg-[#383838]/30 border-[#D7D7D7] dark:border-[#4A4A4A] hover:shadow-[0_0_15px_0_#00000026] transition-shadow duration-300'
+            className={`relative mx-auto flex h-[91.2px] w-full cursor-pointer flex-col justify-between gap-2 rounded-2xl border border-solid pt-[21.1px] pr-[21px] pb-[15px] pl-[21px] transition-colors duration-300 ease-out ${
+              ingestor.type === option.value
+                ? 'border-[#7D54D1] bg-[#7D54D1] text-white'
+                : 'border-[#D7D7D7] bg-transparent transition-shadow duration-300 hover:bg-[#ECECEC]/30 hover:shadow-[0_0_15px_0_#00000026] dark:border-[#4A4A4A] dark:hover:bg-[#383838]/30'
             }`}
-            onClick={() => handleIngestorTypeChange(option.value as IngestorType)}
+            onClick={() =>
+              handleIngestorTypeChange(option.value as IngestorType)
+            }
           >
-            <div className="flex flex-col justify-between h-full">
-              <div className="w-6 h-6">
-                <img 
-                  src={option.icon} 
-                  alt={option.label} 
-                  className={`${ingestor.type === option.value ? 'filter invert' : ''} dark:filter dark:invert`}
+            <div className="flex h-full flex-col justify-between">
+              <div className="h-6 w-6">
+                <img
+                  src={option.icon}
+                  alt={option.label}
+                  className={`${ingestor.type === option.value ? 'invert filter' : ''} dark:invert dark:filter`}
                 />
               </div>
-              <p className="font-inter font-semibold text-[13px] leading-[18px] self-start">
-                {option.label}
+              <p className="font-inter self-start text-[13px] leading-[18px] font-semibold">
+                {t(`modals.uploadDoc.ingestors.${option.value}.label`)}
               </p>
             </div>
           </div>
@@ -714,21 +803,19 @@ function Upload({
       </div>
     );
   };
-  let view;
-
-  if (progress?.type === 'UPLOAD') {
-    view = <UploadProgress></UploadProgress>;
-  } else if (progress?.type === 'TRAINING') {
-    view = <TrainingProgress></TrainingProgress>;
-  }   else {
-    view = (
+  return (
+    <WrapperModal
+      close={handleClose}
+      className="max-h-[90vh] w-11/12 sm:max-h-none sm:w-auto sm:min-w-[600px] md:min-w-[700px]"
+      contentClassName="max-h-[80vh] sm:max-h-none"
+    >
       <div className="flex w-full flex-col gap-6">
         {!ingestor.type && (
-          <p className="text-[#18181B] dark:text-[#ECECF1] text-left font-inter font-semibold text-[20px] leading-[28px] tracking-[0.15px]">
-            Select the way to add your source
+          <p className="font-inter text-left text-[20px] leading-[28px] font-semibold tracking-[0.15px] text-[#18181B] dark:text-[#ECECF1]">
+            {t('modals.uploadDoc.selectSource')}
           </p>
         )}
-        
+
         {activeTab && (
           <>
             {!ingestor.type && renderIngestorSelection()}
@@ -736,18 +823,19 @@ function Upload({
               <div className="flex flex-col gap-4">
                 <button
                   onClick={() => handleIngestorTypeChange(null)}
-                  className="flex items-center gap-2 text-[#777777] hover:text-[#555555] w-fit"
+                  className="flex w-fit items-center gap-2 text-[#777777] hover:text-[#555555]"
                 >
-                  <img 
-                    src={ChevronRight} 
-                    alt="back" 
-                    className="h-3 w-3 transform rotate-180" 
+                  <img
+                    src={ChevronRight}
+                    alt="back"
+                    className="h-3 w-3 rotate-180 transform"
                   />
-                  <span>Back</span>
+                  <span>{t('modals.uploadDoc.back')}</span>
                 </button>
 
-                <h2 className="font-inter font-semibold text-[22px] leading-[28px] tracking-[0.15px] text-black dark:text-[#E0E0E0]">
-                  {ingestor.type && getIngestorSchema(ingestor.type as IngestorType)?.heading}
+                <h2 className="font-inter text-[22px] leading-[28px] font-semibold tracking-[0.15px] text-black dark:text-[#E0E0E0]">
+                  {ingestor.type &&
+                    t(`modals.uploadDoc.ingestors.${ingestor.type}.heading`)}
                 </h2>
 
                 <Input
@@ -761,7 +849,7 @@ function Upload({
                     }));
                   }}
                   borderVariant="thin"
-                  placeholder="Name"
+                  placeholder={t('modals.uploadDoc.name')}
                   required={true}
                   labelBgClassName="bg-white dark:bg-charleston-green-2"
                   className="w-full"
@@ -769,37 +857,26 @@ function Upload({
                 {renderFormFields()}
               </div>
             )}
-  
-            {ingestor.type && getIngestorSchema(ingestor.type as IngestorType)?.fields.some(
-              (field: FormField) => field.advanced,
-            ) && (
-              <button
-                onClick={() => setShowAdvancedOptions(!showAdvancedOptions)}
-                className="text-purple-30 bg-transparent py-2 pl-0 text-left text-sm font-normal hover:cursor-pointer"
-              >
-                {showAdvancedOptions
-                  ? t('modals.uploadDoc.hideAdvanced')
-                  : t('modals.uploadDoc.showAdvanced')}
-              </button>
-            )}
+
+            {ingestor.type &&
+              getIngestorSchema(ingestor.type as IngestorType)?.fields.some(
+                (field: FormField) => field.advanced,
+              ) && (
+                <button
+                  onClick={() => setShowAdvancedOptions(!showAdvancedOptions)}
+                  className="text-purple-30 bg-transparent py-2 pl-0 text-left text-sm font-normal hover:cursor-pointer"
+                >
+                  {showAdvancedOptions
+                    ? t('modals.uploadDoc.hideAdvanced')
+                    : t('modals.uploadDoc.showAdvanced')}
+                </button>
+              )}
           </>
         )}
         <div className="flex justify-end gap-4">
           {activeTab && ingestor.type && (
             <button
-              onClick={() => {
-                if (!ingestor.type) return;
-                const ingestorSchemaForUpload = getIngestorSchema(ingestor.type as IngestorType);
-                if (!ingestorSchemaForUpload) return;
-                const schema: FormField[] = ingestorSchemaForUpload.fields;
-                const hasLocalFilePicker = schema.some((field: FormField) => field.type === 'local_file_picker');
-
-                if (hasLocalFilePicker) {
-                  uploadFile();
-                } else {
-                  uploadRemote();
-                }
-              }}
+              onClick={handleUpload}
               disabled={isUploadDisabled()}
               className={`rounded-3xl px-4 py-2 text-[14px] font-medium ${
                 isUploadDisabled()
@@ -812,26 +889,6 @@ function Upload({
           )}
         </div>
       </div>
-    );
-  }
-
-  
-
-  
-
-  return (
-    <WrapperModal
-      isPerformingTask={progress !== undefined && progress.percentage < 100}
-      close={() => {
-        close();
-        setIngestor({ type: null, name: '', config: {} });
-        setfiles([]);
-        setModalState('INACTIVE');
-      }}
-      className="w-11/12 sm:w-auto sm:min-w-[600px] md:min-w-[700px] max-h-[90vh] sm:max-h-none"
-      contentClassName="max-h-[80vh] sm:max-h-none"
-    >
-      {view}
     </WrapperModal>
   );
 }

frontend/src/upload/types/ingestor.ts

@@ -5,7 +5,13 @@ import GithubIcon from '../../assets/github.svg';
 import RedditIcon from '../../assets/reddit.svg';
 import DriveIcon from '../../assets/drive.svg';
 
-export type IngestorType = 'crawler' | 'github' | 'reddit' | 'url' | 'google_drive' | 'local_file';
+export type IngestorType =
+  | 'crawler'
+  | 'github'
+  | 'reddit'
+  | 'url'
+  | 'google_drive'
+  | 'local_file';
 
 export interface IngestorConfig {
   type: IngestorType | null;
@@ -20,7 +26,14 @@ export type IngestorFormData = {
   data: string;
 };
 
-export type FieldType = 'string' | 'number' | 'enum' | 'boolean' | 'local_file_picker' | 'remote_file_picker' | 'google_drive_picker';
+export type FieldType =
+  | 'string'
+  | 'number'
+  | 'enum'
+  | 'boolean'
+  | 'local_file_picker'
+  | 'remote_file_picker'
+  | 'google_drive_picker';
 
 export interface FormField {
   name: string;
@@ -47,29 +60,41 @@ export const IngestorFormSchemas: IngestorSchema[] = [
     icon: FileUploadIcon,
     heading: 'Upload new document',
     fields: [
-      { name: 'files', label: 'Select files', type: 'local_file_picker', required: true },
-    ]
+      {
+        name: 'files',
+        label: 'Select files',
+        type: 'local_file_picker',
+        required: true,
+      },
+    ],
   },
   {
     key: 'crawler',
     label: 'Crawler',
     icon: CrawlerIcon,
     heading: 'Add content with Web Crawler',
-    fields: [{ name: 'url', label: 'URL', type: 'string', required: true }]
+    fields: [{ name: 'url', label: 'URL', type: 'string', required: true }],
   },
   {
     key: 'url',
     label: 'Link',
     icon: UrlIcon,
     heading: 'Add content from URL',
-    fields: [{ name: 'url', label: 'URL', type: 'string', required: true }]
+    fields: [{ name: 'url', label: 'URL', type: 'string', required: true }],
   },
   {
     key: 'github',
     label: 'GitHub',
     icon: GithubIcon,
     heading: 'Add content from GitHub',
-    fields: [{ name: 'repo_url', label: 'Repository URL', type: 'string', required: true }]
+    fields: [
+      {
+        name: 'repo_url',
+        label: 'Repository URL',
+        type: 'string',
+        required: true,
+      },
+    ],
   },
   {
     key: 'reddit',
@@ -78,11 +103,31 @@ export const IngestorFormSchemas: IngestorSchema[] = [
     heading: 'Add content from Reddit',
     fields: [
       { name: 'client_id', label: 'Client ID', type: 'string', required: true },
-      { name: 'client_secret', label: 'Client Secret', type: 'string', required: true },
-      { name: 'user_agent', label: 'User Agent', type: 'string', required: true },
-      { name: 'search_queries', label: 'Search Queries', type: 'string', required: true },
-      { name: 'number_posts', label: 'Number of Posts', type: 'number', required: true },
-    ]
+      {
+        name: 'client_secret',
+        label: 'Client Secret',
+        type: 'string',
+        required: true,
+      },
+      {
+        name: 'user_agent',
+        label: 'User Agent',
+        type: 'string',
+        required: true,
+      },
+      {
+        name: 'search_queries',
+        label: 'Search Queries',
+        type: 'string',
+        required: true,
+      },
+      {
+        name: 'number_posts',
+        label: 'Number of Posts',
+        type: 'number',
+        required: true,
+      },
+    ],
   },
   {
     key: 'google_drive',
@@ -91,7 +136,7 @@ export const IngestorFormSchemas: IngestorSchema[] = [
     heading: 'Upload from Google Drive',
     validate: () => {
       const googleClientId = import.meta.env.VITE_GOOGLE_CLIENT_ID;
-      return !!(googleClientId);
+      return !!googleClientId;
     },
     fields: [
       {
@@ -99,12 +144,15 @@ export const IngestorFormSchemas: IngestorSchema[] = [
         label: 'Select Files from Google Drive',
         type: 'google_drive_picker',
         required: true,
-      }
-    ]
+      },
+    ],
   },
 ];
 
-export const IngestorDefaultConfigs: Record<IngestorType, Omit<IngestorConfig, 'type'>> = {
+export const IngestorDefaultConfigs: Record<
+  IngestorType,
+  Omit<IngestorConfig, 'type'>
+> = {
   crawler: { name: '', config: { url: '' } },
   url: { name: '', config: { url: '' } },
   reddit: {
@@ -114,8 +162,8 @@ export const IngestorDefaultConfigs: Record<IngestorType, Omit<IngestorConfig, '
       client_secret: '',
       user_agent: '',
       search_queries: '',
-      number_posts: 10
-    }
+      number_posts: 10,
+    },
   },
   github: { name: '', config: { repo_url: '' } },
   google_drive: {
@@ -123,8 +171,8 @@ export const IngestorDefaultConfigs: Record<IngestorType, Omit<IngestorConfig, '
     config: {
       file_ids: '',
       folder_ids: '',
-      recursive: true
-    }
+      recursive: true,
+    },
   },
   local_file: { name: '', config: { files: [] } },
 };
@@ -136,8 +184,8 @@ export interface IngestorOption {
   heading: string;
 }
 
-export const getIngestorSchema = (key: IngestorType): IngestorSchema | undefined => {
-  return IngestorFormSchemas.find(schema => schema.key === key);
+export const getIngestorSchema = (
+  key: IngestorType,
+): IngestorSchema | undefined => {
+  return IngestorFormSchemas.find((schema) => schema.key === key);
 };
-
-

frontend/src/upload/uploadSlice.ts

@@ -10,12 +10,31 @@ export interface Attachment {
   token_count?: number;
 }
 
+export type UploadTaskStatus =
+  | 'preparing'
+  | 'uploading'
+  | 'training'
+  | 'completed'
+  | 'failed';
+
+export interface UploadTask {
+  id: string;
+  fileName: string;
+  progress: number;
+  status: UploadTaskStatus;
+  taskId?: string;
+  errorMessage?: string;
+  dismissed?: boolean;
+}
+
 interface UploadState {
   attachments: Attachment[];
+  tasks: UploadTask[];
 }
 
 const initialState: UploadState = {
   attachments: [],
+  tasks: [],
 };
 
 export const uploadSlice = createSlice({
@@ -52,6 +71,49 @@ export const uploadSlice = createSlice({
         (att) => att.status === 'uploading' || att.status === 'processing',
       );
     },
+    addUploadTask: (state, action: PayloadAction<UploadTask>) => {
+      state.tasks.unshift(action.payload);
+    },
+    updateUploadTask: (
+      state,
+      action: PayloadAction<{
+        id: string;
+        updates: Partial<UploadTask>;
+      }>,
+    ) => {
+      const index = state.tasks.findIndex(
+        (task) => task.id === action.payload.id,
+      );
+      if (index !== -1) {
+        const updates = action.payload.updates;
+
+        // When task completes or fails, set dismissed to false to notify user
+        if (updates.status === 'completed' || updates.status === 'failed') {
+          state.tasks[index] = {
+            ...state.tasks[index],
+            ...updates,
+            dismissed: false,
+          };
+        } else {
+          state.tasks[index] = {
+            ...state.tasks[index],
+            ...updates,
+          };
+        }
+      }
+    },
+    dismissUploadTask: (state, action: PayloadAction<string>) => {
+      const index = state.tasks.findIndex((task) => task.id === action.payload);
+      if (index !== -1) {
+        state.tasks[index] = {
+          ...state.tasks[index],
+          dismissed: true,
+        };
+      }
+    },
+    removeUploadTask: (state, action: PayloadAction<string>) => {
+      state.tasks = state.tasks.filter((task) => task.id !== action.payload);
+    },
   },
 });
 
@@ -60,10 +122,15 @@ export const {
   updateAttachment,
   removeAttachment,
   clearAttachments,
+  addUploadTask,
+  updateUploadTask,
+  dismissUploadTask,
+  removeUploadTask,
 } = uploadSlice.actions;
 
 export const selectAttachments = (state: RootState) => state.upload.attachments;
 export const selectCompletedAttachments = (state: RootState) =>
   state.upload.attachments.filter((att) => att.status === 'completed');
+export const selectUploadTasks = (state: RootState) => state.upload.tasks;
 
 export default uploadSlice.reducer;

tests/llm/handlers/test_base.py

@@ -0,0 +1,231 @@
+from unittest.mock import Mock, patch
+from typing import Any, Dict, Generator
+
+from application.llm.handlers.base import LLMHandler, LLMResponse, ToolCall
+
+
+class TestToolCall:
+    """Test ToolCall dataclass."""
+
+    def test_tool_call_creation(self):
+        """Test basic ToolCall creation."""
+        tool_call = ToolCall(
+            id="test_id",
+            name="test_function",
+            arguments={"arg1": "value1"},
+            index=0
+        )
+        assert tool_call.id == "test_id"
+        assert tool_call.name == "test_function"
+        assert tool_call.arguments == {"arg1": "value1"}
+        assert tool_call.index == 0
+
+    def test_tool_call_from_dict(self):
+        """Test ToolCall creation from dictionary."""
+        data = {
+            "id": "call_123",
+            "name": "get_weather",
+            "arguments": {"location": "New York"},
+            "index": 1
+        }
+        tool_call = ToolCall.from_dict(data)
+        assert tool_call.id == "call_123"
+        assert tool_call.name == "get_weather"
+        assert tool_call.arguments == {"location": "New York"}
+        assert tool_call.index == 1
+
+    def test_tool_call_from_dict_missing_fields(self):
+        """Test ToolCall creation with missing fields."""
+        data = {"name": "test_func"}
+        tool_call = ToolCall.from_dict(data)
+        assert tool_call.id == ""
+        assert tool_call.name == "test_func"
+        assert tool_call.arguments == {}
+        assert tool_call.index is None
+
+
+class TestLLMResponse:
+    """Test LLMResponse dataclass."""
+
+    def test_llm_response_creation(self):
+        """Test basic LLMResponse creation."""
+        tool_calls = [ToolCall(id="1", name="func", arguments={})]
+        response = LLMResponse(
+            content="Hello",
+            tool_calls=tool_calls,
+            finish_reason="tool_calls",
+            raw_response={"test": "data"}
+        )
+        assert response.content == "Hello"
+        assert len(response.tool_calls) == 1
+        assert response.finish_reason == "tool_calls"
+        assert response.raw_response == {"test": "data"}
+
+    def test_requires_tool_call_true(self):
+        """Test requires_tool_call property when tool calls are needed."""
+        tool_calls = [ToolCall(id="1", name="func", arguments={})]
+        response = LLMResponse(
+            content="",
+            tool_calls=tool_calls,
+            finish_reason="tool_calls",
+            raw_response={}
+        )
+        assert response.requires_tool_call is True
+
+    def test_requires_tool_call_false_no_tools(self):
+        """Test requires_tool_call property when no tool calls."""
+        response = LLMResponse(
+            content="Hello",
+            tool_calls=[],
+            finish_reason="stop",
+            raw_response={}
+        )
+        assert response.requires_tool_call is False
+
+    def test_requires_tool_call_false_wrong_finish_reason(self):
+        """Test requires_tool_call property with tools but wrong finish reason."""
+        tool_calls = [ToolCall(id="1", name="func", arguments={})]
+        response = LLMResponse(
+            content="Hello",
+            tool_calls=tool_calls,
+            finish_reason="stop",
+            raw_response={}
+        )
+        assert response.requires_tool_call is False
+
+
+class ConcreteHandler(LLMHandler):
+    """Concrete implementation for testing abstract base class."""
+
+    def parse_response(self, response: Any) -> LLMResponse:
+        return LLMResponse(
+            content=str(response),
+            tool_calls=[],
+            finish_reason="stop",
+            raw_response=response
+        )
+
+    def create_tool_message(self, tool_call: ToolCall, result: Any) -> Dict:
+        return {
+            "role": "tool",
+            "content": str(result),
+            "tool_call_id": tool_call.id
+        }
+
+    def _iterate_stream(self, response: Any) -> Generator:
+        for chunk in response:
+            yield chunk
+
+
+class TestLLMHandler:
+    """Test LLMHandler base class."""
+
+    def test_handler_initialization(self):
+        """Test handler initialization."""
+        handler = ConcreteHandler()
+        assert handler.llm_calls == []
+        assert handler.tool_calls == []
+
+    def test_prepare_messages_no_attachments(self):
+        """Test prepare_messages with no attachments."""
+        handler = ConcreteHandler()
+        messages = [{"role": "user", "content": "Hello"}]
+        
+        mock_agent = Mock()
+        result = handler.prepare_messages(mock_agent, messages, None)
+        assert result == messages
+
+    def test_prepare_messages_with_supported_attachments(self):
+        """Test prepare_messages with supported attachments."""
+        handler = ConcreteHandler()
+        messages = [{"role": "user", "content": "Hello"}]
+        attachments = [{"mime_type": "image/png", "path": "/test.png"}]
+        
+        mock_agent = Mock()
+        mock_agent.llm.get_supported_attachment_types.return_value = ["image/png"]
+        mock_agent.llm.prepare_messages_with_attachments.return_value = messages
+        
+        result = handler.prepare_messages(mock_agent, messages, attachments)
+        mock_agent.llm.prepare_messages_with_attachments.assert_called_once_with(
+            messages, attachments
+        )
+        assert result == messages
+
+    @patch('application.llm.handlers.base.logger')
+    def test_prepare_messages_with_unsupported_attachments(self, mock_logger):
+        """Test prepare_messages with unsupported attachments."""
+        handler = ConcreteHandler()
+        messages = [{"role": "user", "content": "Hello"}]
+        attachments = [{"mime_type": "text/plain", "path": "/test.txt"}]
+        
+        mock_agent = Mock()
+        mock_agent.llm.get_supported_attachment_types.return_value = ["image/png"]
+        
+        with patch.object(handler, '_append_unsupported_attachments', return_value=messages) as mock_append:
+            result = handler.prepare_messages(mock_agent, messages, attachments)
+            mock_append.assert_called_once_with(messages, attachments)
+            assert result == messages
+
+    def test_prepare_messages_mixed_attachments(self):
+        """Test prepare_messages with both supported and unsupported attachments."""
+        handler = ConcreteHandler()
+        messages = [{"role": "user", "content": "Hello"}]
+        attachments = [
+            {"mime_type": "image/png", "path": "/test.png"},
+            {"mime_type": "text/plain", "path": "/test.txt"}
+        ]
+        
+        mock_agent = Mock()
+        mock_agent.llm.get_supported_attachment_types.return_value = ["image/png"]
+        mock_agent.llm.prepare_messages_with_attachments.return_value = messages
+        
+        with patch.object(handler, '_append_unsupported_attachments', return_value=messages) as mock_append:
+            result = handler.prepare_messages(mock_agent, messages, attachments)
+            
+            # Should call both methods
+            mock_agent.llm.prepare_messages_with_attachments.assert_called_once()
+            mock_append.assert_called_once()
+            assert result == messages
+
+    def test_process_message_flow_non_streaming(self):
+        """Test process_message_flow for non-streaming."""
+        handler = ConcreteHandler()
+        mock_agent = Mock()
+        initial_response = "test response"
+        tools_dict = {}
+        messages = [{"role": "user", "content": "Hello"}]
+        
+        with patch.object(handler, 'prepare_messages', return_value=messages) as mock_prepare:
+            with patch.object(handler, 'handle_non_streaming', return_value="final") as mock_handle:
+                result = handler.process_message_flow(
+                    mock_agent, initial_response, tools_dict, messages, stream=False
+                )
+                
+                mock_prepare.assert_called_once_with(mock_agent, messages, None)
+                mock_handle.assert_called_once_with(mock_agent, initial_response, tools_dict, messages)
+                assert result == "final"
+
+    def test_process_message_flow_streaming(self):
+        """Test process_message_flow for streaming."""
+        handler = ConcreteHandler()
+        mock_agent = Mock()
+        initial_response = "test response"
+        tools_dict = {}
+        messages = [{"role": "user", "content": "Hello"}]
+        
+        def mock_generator():
+            yield "chunk1"
+            yield "chunk2"
+        
+        with patch.object(handler, 'prepare_messages', return_value=messages) as mock_prepare:
+            with patch.object(handler, 'handle_streaming', return_value=mock_generator()) as mock_handle:
+                result = handler.process_message_flow(
+                    mock_agent, initial_response, tools_dict, messages, stream=True
+                )
+                
+                mock_prepare.assert_called_once_with(mock_agent, messages, None)
+                mock_handle.assert_called_once_with(mock_agent, initial_response, tools_dict, messages)
+                
+                # Verify it's a generator
+                chunks = list(result)
+                assert chunks == ["chunk1", "chunk2"]

tests/llm/handlers/test_google.py

@@ -0,0 +1,270 @@
+from unittest.mock import Mock, patch
+from types import SimpleNamespace
+import uuid
+
+from application.llm.handlers.google import GoogleLLMHandler
+from application.llm.handlers.base import ToolCall, LLMResponse
+
+
+class TestGoogleLLMHandler:
+    """Test GoogleLLMHandler class."""
+
+    def test_handler_initialization(self):
+        """Test handler initialization."""
+        handler = GoogleLLMHandler()
+        assert handler.llm_calls == []
+        assert handler.tool_calls == []
+
+    def test_parse_response_string_input(self):
+        """Test parsing string response."""
+        handler = GoogleLLMHandler()
+        response = "Hello from Google!"
+        
+        result = handler.parse_response(response)
+        
+        assert isinstance(result, LLMResponse)
+        assert result.content == "Hello from Google!"
+        assert result.tool_calls == []
+        assert result.finish_reason == "stop"
+        assert result.raw_response == "Hello from Google!"
+
+    def test_parse_response_with_candidates_text_only(self):
+        """Test parsing response with candidates containing only text."""
+        handler = GoogleLLMHandler()
+        
+        mock_part = SimpleNamespace(text="Google response text")
+        mock_content = SimpleNamespace(parts=[mock_part])
+        mock_candidate = SimpleNamespace(content=mock_content)
+        mock_response = SimpleNamespace(candidates=[mock_candidate])
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == "Google response text"
+        assert result.tool_calls == []
+        assert result.finish_reason == "stop"
+        assert result.raw_response == mock_response
+
+    def test_parse_response_with_multiple_text_parts(self):
+        """Test parsing response with multiple text parts."""
+        handler = GoogleLLMHandler()
+        
+        mock_part1 = SimpleNamespace(text="First part")
+        mock_part2 = SimpleNamespace(text="Second part")
+        mock_content = SimpleNamespace(parts=[mock_part1, mock_part2])
+        mock_candidate = SimpleNamespace(content=mock_content)
+        mock_response = SimpleNamespace(candidates=[mock_candidate])
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == "First part Second part"
+        assert result.tool_calls == []
+        assert result.finish_reason == "stop"
+
+    @patch('uuid.uuid4')
+    def test_parse_response_with_function_call(self, mock_uuid):
+        """Test parsing response with function call."""
+        mock_uuid.return_value = Mock(spec=uuid.UUID)
+        mock_uuid.return_value.__str__ = Mock(return_value="test-uuid-123")
+        
+        handler = GoogleLLMHandler()
+        
+        mock_function_call = SimpleNamespace(
+            name="get_weather",
+            args={"location": "San Francisco"}
+        )
+        mock_part = SimpleNamespace(function_call=mock_function_call)
+        mock_content = SimpleNamespace(parts=[mock_part])
+        mock_candidate = SimpleNamespace(content=mock_content)
+        mock_response = SimpleNamespace(candidates=[mock_candidate])
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == ""
+        assert len(result.tool_calls) == 1
+        assert result.tool_calls[0].id == "test-uuid-123"
+        assert result.tool_calls[0].name == "get_weather"
+        assert result.tool_calls[0].arguments == {"location": "San Francisco"}
+        assert result.finish_reason == "tool_calls"
+
+    @patch('uuid.uuid4')
+    def test_parse_response_with_mixed_parts(self, mock_uuid):
+        """Test parsing response with both text and function call parts."""
+        mock_uuid.return_value = Mock(spec=uuid.UUID)
+        mock_uuid.return_value.__str__ = Mock(return_value="test-uuid-456")
+        
+        handler = GoogleLLMHandler()
+        
+        mock_text_part = SimpleNamespace(text="I'll check the weather for you.")
+        mock_function_call = SimpleNamespace(
+            name="get_weather",
+            args={"location": "NYC"}
+        )
+        mock_function_part = SimpleNamespace(function_call=mock_function_call)
+        
+        mock_content = SimpleNamespace(parts=[mock_text_part, mock_function_part])
+        mock_candidate = SimpleNamespace(content=mock_content)
+        mock_response = SimpleNamespace(candidates=[mock_candidate])
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == "I'll check the weather for you."
+        assert len(result.tool_calls) == 1
+        assert result.tool_calls[0].name == "get_weather"
+        assert result.finish_reason == "tool_calls"
+
+    def test_parse_response_empty_candidates(self):
+        """Test parsing response with empty candidates."""
+        handler = GoogleLLMHandler()
+        
+        mock_response = SimpleNamespace(candidates=[])
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == ""
+        assert result.tool_calls == []
+        assert result.finish_reason == "stop"
+
+    def test_parse_response_parts_with_none_text(self):
+        """Test parsing response with parts that have None text."""
+        handler = GoogleLLMHandler()
+        
+        mock_part1 = SimpleNamespace(text=None)
+        mock_part2 = SimpleNamespace(text="Valid text")
+        mock_content = SimpleNamespace(parts=[mock_part1, mock_part2])
+        mock_candidate = SimpleNamespace(content=mock_content)
+        mock_response = SimpleNamespace(candidates=[mock_candidate])
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == "Valid text"
+
+    def test_parse_response_parts_without_text_attribute(self):
+        """Test parsing response with parts missing text attribute."""
+        handler = GoogleLLMHandler()
+        
+        mock_part1 = SimpleNamespace() 
+        mock_part2 = SimpleNamespace(text="Valid text")
+        mock_content = SimpleNamespace(parts=[mock_part1, mock_part2])
+        mock_candidate = SimpleNamespace(content=mock_content)
+        mock_response = SimpleNamespace(candidates=[mock_candidate])
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == "Valid text"
+
+    @patch('uuid.uuid4')
+    def test_parse_response_direct_function_call(self, mock_uuid):
+        """Test parsing response with direct function call (not in candidates)."""
+        mock_uuid.return_value = Mock(spec=uuid.UUID)
+        mock_uuid.return_value.__str__ = Mock(return_value="direct-uuid-789")
+        
+        handler = GoogleLLMHandler()
+        
+        mock_function_call = SimpleNamespace(
+            name="calculate",
+            args={"expression": "2+2"}
+        )
+        mock_response = SimpleNamespace(
+            function_call=mock_function_call,
+            text="The calculation result is:"
+        )
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == "The calculation result is:"
+        assert len(result.tool_calls) == 1
+        assert result.tool_calls[0].id == "direct-uuid-789"
+        assert result.tool_calls[0].name == "calculate"
+        assert result.tool_calls[0].arguments == {"expression": "2+2"}
+        assert result.finish_reason == "tool_calls"
+
+    def test_parse_response_direct_function_call_no_text(self):
+        """Test parsing response with direct function call and no text."""
+        handler = GoogleLLMHandler()
+        
+        mock_function_call = SimpleNamespace(
+            name="get_data",
+            args={"id": 123}
+        )
+        mock_response = SimpleNamespace(function_call=mock_function_call)
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == ""
+        assert len(result.tool_calls) == 1
+        assert result.tool_calls[0].name == "get_data"
+        assert result.finish_reason == "tool_calls"
+
+    def test_create_tool_message(self):
+        """Test creating tool message."""
+        handler = GoogleLLMHandler()
+        
+        tool_call = ToolCall(
+            id="call_123",
+            name="get_weather",
+            arguments={"location": "Tokyo"},
+            index=0
+        )
+        result = {"temperature": "25C", "condition": "cloudy"}
+        
+        message = handler.create_tool_message(tool_call, result)
+        
+        expected = {
+            "role": "model",
+            "content": [
+                {
+                    "function_response": {
+                        "name": "get_weather",
+                        "response": {"result": result},
+                    }
+                }
+            ],
+        }
+        
+        assert message == expected
+
+    def test_create_tool_message_string_result(self):
+        """Test creating tool message with string result."""
+        handler = GoogleLLMHandler()
+        
+        tool_call = ToolCall(id="call_456", name="get_time", arguments={})
+        result = "2023-12-01 15:30:00 JST"
+        
+        message = handler.create_tool_message(tool_call, result)
+        
+        assert message["role"] == "model"
+        assert message["content"][0]["function_response"]["response"]["result"] == result
+        assert message["content"][0]["function_response"]["name"] == "get_time"
+
+    def test_iterate_stream(self):
+        """Test stream iteration."""
+        handler = GoogleLLMHandler()
+        
+        mock_chunks = ["chunk1", "chunk2", "chunk3"]
+        
+        result = list(handler._iterate_stream(mock_chunks))
+        
+        assert result == mock_chunks
+
+    def test_iterate_stream_empty(self):
+        """Test stream iteration with empty response."""
+        handler = GoogleLLMHandler()
+        
+        result = list(handler._iterate_stream([]))
+        
+        assert result == []
+
+    def test_parse_response_parts_without_function_call_attribute(self):
+        """Test parsing response with parts missing function_call attribute."""
+        handler = GoogleLLMHandler()
+        
+        mock_part = SimpleNamespace(text="Normal text")
+        mock_content = SimpleNamespace(parts=[mock_part])
+        mock_candidate = SimpleNamespace(content=mock_content)
+        mock_response = SimpleNamespace(candidates=[mock_candidate])
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == "Normal text"
+        assert result.tool_calls == []
+        assert result.finish_reason == "stop"

tests/llm/handlers/test_handler_creator.py

@@ -0,0 +1,125 @@
+
+from application.llm.handlers.handler_creator import LLMHandlerCreator
+from application.llm.handlers.base import LLMHandler
+from application.llm.handlers.openai import OpenAILLMHandler
+from application.llm.handlers.google import GoogleLLMHandler
+
+
+class TestLLMHandlerCreator:
+    """Test LLMHandlerCreator class."""
+
+    def test_create_openai_handler(self):
+        """Test creating OpenAI handler."""
+        handler = LLMHandlerCreator.create_handler("openai")
+        
+        assert isinstance(handler, OpenAILLMHandler)
+        assert isinstance(handler, LLMHandler)
+
+    def test_create_openai_handler_case_insensitive(self):
+        """Test creating OpenAI handler with different cases."""
+        handler_upper = LLMHandlerCreator.create_handler("OPENAI")
+        handler_mixed = LLMHandlerCreator.create_handler("OpenAI")
+        
+        assert isinstance(handler_upper, OpenAILLMHandler)
+        assert isinstance(handler_mixed, OpenAILLMHandler)
+
+    def test_create_google_handler(self):
+        """Test creating Google handler."""
+        handler = LLMHandlerCreator.create_handler("google")
+        
+        assert isinstance(handler, GoogleLLMHandler)
+        assert isinstance(handler, LLMHandler)
+
+    def test_create_google_handler_case_insensitive(self):
+        """Test creating Google handler with different cases."""
+        handler_upper = LLMHandlerCreator.create_handler("GOOGLE")
+        handler_mixed = LLMHandlerCreator.create_handler("Google")
+
+        assert isinstance(handler_upper, GoogleLLMHandler)
+        assert isinstance(handler_mixed, GoogleLLMHandler)
+
+
+
+    def test_create_default_handler(self):
+        """Test creating default handler."""
+        handler = LLMHandlerCreator.create_handler("default")
+        
+        assert isinstance(handler, OpenAILLMHandler)
+
+    def test_create_unknown_handler_fallback(self):
+        """Test creating handler for unknown type falls back to OpenAI."""
+        handler = LLMHandlerCreator.create_handler("unknown_provider")
+
+        assert isinstance(handler, OpenAILLMHandler)
+
+    def test_create_anthropic_handler_fallback(self):
+        """Test creating Anthropic handler falls back to OpenAI (not supported in handlers)."""
+        handler = LLMHandlerCreator.create_handler("anthropic")
+
+        assert isinstance(handler, OpenAILLMHandler)
+
+    def test_create_empty_string_handler_fallback(self):
+        """Test creating handler with empty string falls back to OpenAI."""
+        handler = LLMHandlerCreator.create_handler("")
+        
+        assert isinstance(handler, OpenAILLMHandler)
+
+
+
+    def test_handlers_registry(self):
+        """Test the handlers registry contains expected mappings."""
+        expected_handlers = {
+            "openai": OpenAILLMHandler,
+            "google": GoogleLLMHandler,
+            "default": OpenAILLMHandler,
+        }
+
+        assert LLMHandlerCreator.handlers == expected_handlers
+
+    def test_create_handler_with_args(self):
+        """Test creating handler with additional arguments."""
+        handler = LLMHandlerCreator.create_handler("openai")
+        
+        assert isinstance(handler, OpenAILLMHandler)
+        assert handler.llm_calls == []
+        assert handler.tool_calls == []
+
+    def test_create_handler_with_kwargs(self):
+        """Test creating handler with keyword arguments."""
+        handler = LLMHandlerCreator.create_handler("google")
+        
+        assert isinstance(handler, GoogleLLMHandler)
+        assert handler.llm_calls == []
+        assert handler.tool_calls == []
+
+    def test_all_registered_handlers_are_valid(self):
+        """Test that all registered handlers can be instantiated."""
+        for handler_type in LLMHandlerCreator.handlers.keys():
+            handler = LLMHandlerCreator.create_handler(handler_type)
+            assert isinstance(handler, LLMHandler)
+            assert hasattr(handler, 'parse_response')
+            assert hasattr(handler, 'create_tool_message')
+            assert hasattr(handler, '_iterate_stream')
+
+    def test_handler_inheritance(self):
+        """Test that all created handlers inherit from LLMHandler."""
+        test_types = ["openai", "google", "default", "unknown"]
+        
+        for handler_type in test_types:
+            handler = LLMHandlerCreator.create_handler(handler_type)
+            assert isinstance(handler, LLMHandler)
+            
+            assert callable(getattr(handler, 'parse_response'))
+            assert callable(getattr(handler, 'create_tool_message'))
+            assert callable(getattr(handler, '_iterate_stream'))
+
+    def test_create_handler_preserves_handler_state(self):
+        """Test that each created handler has independent state."""
+        handler1 = LLMHandlerCreator.create_handler("openai")
+        handler2 = LLMHandlerCreator.create_handler("openai")
+        
+        handler1.llm_calls.append("test_call")
+
+        assert len(handler1.llm_calls) == 1
+        assert len(handler2.llm_calls) == 0
+        assert handler1 is not handler2

tests/llm/handlers/test_openai.py

@@ -0,0 +1,208 @@
+from types import SimpleNamespace
+
+from application.llm.handlers.openai import OpenAILLMHandler
+from application.llm.handlers.base import ToolCall, LLMResponse
+
+
+class TestOpenAILLMHandler:
+    """Test OpenAILLMHandler class."""
+
+    def test_handler_initialization(self):
+        """Test handler initialization."""
+        handler = OpenAILLMHandler()
+        assert handler.llm_calls == []
+        assert handler.tool_calls == []
+
+    def test_parse_response_string_input(self):
+        """Test parsing string response."""
+        handler = OpenAILLMHandler()
+        response = "Hello, world!"
+        
+        result = handler.parse_response(response)
+        
+        assert isinstance(result, LLMResponse)
+        assert result.content == "Hello, world!"
+        assert result.tool_calls == []
+        assert result.finish_reason == "stop"
+        assert result.raw_response == "Hello, world!"
+
+    def test_parse_response_with_message_content(self):
+        """Test parsing response with message content."""
+        handler = OpenAILLMHandler()
+        
+        # Mock OpenAI response structure
+        mock_message = SimpleNamespace(content="Test content", tool_calls=None)
+        mock_response = SimpleNamespace(message=mock_message, finish_reason="stop")
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == "Test content"
+        assert result.tool_calls == []
+        assert result.finish_reason == "stop"
+        assert result.raw_response == mock_response
+
+    def test_parse_response_with_delta_content(self):
+        """Test parsing response with delta content (streaming)."""
+        handler = OpenAILLMHandler()
+        
+        # Mock streaming response structure
+        mock_delta = SimpleNamespace(content="Stream chunk", tool_calls=None)
+        mock_response = SimpleNamespace(delta=mock_delta, finish_reason="")
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == "Stream chunk"
+        assert result.tool_calls == []
+        assert result.finish_reason == ""
+        assert result.raw_response == mock_response
+
+    def test_parse_response_with_tool_calls(self):
+        """Test parsing response with tool calls."""
+        handler = OpenAILLMHandler()
+        
+        # Mock tool call structure
+        mock_function = SimpleNamespace(name="get_weather", arguments='{"location": "NYC"}')
+        mock_tool_call = SimpleNamespace(
+            id="call_123",
+            function=mock_function,
+            index=0
+        )
+        mock_message = SimpleNamespace(content="", tool_calls=[mock_tool_call])
+        mock_response = SimpleNamespace(message=mock_message, finish_reason="tool_calls")
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == ""
+        assert len(result.tool_calls) == 1
+        assert result.tool_calls[0].id == "call_123"
+        assert result.tool_calls[0].name == "get_weather"
+        assert result.tool_calls[0].arguments == '{"location": "NYC"}'
+        assert result.tool_calls[0].index == 0
+        assert result.finish_reason == "tool_calls"
+
+    def test_parse_response_with_multiple_tool_calls(self):
+        """Test parsing response with multiple tool calls."""
+        handler = OpenAILLMHandler()
+        
+        # Mock multiple tool calls
+        mock_function1 = SimpleNamespace(name="get_weather", arguments='{"location": "NYC"}')
+        mock_function2 = SimpleNamespace(name="get_time", arguments='{"timezone": "UTC"}')
+        
+        mock_tool_call1 = SimpleNamespace(id="call_1", function=mock_function1, index=0)
+        mock_tool_call2 = SimpleNamespace(id="call_2", function=mock_function2, index=1)
+        
+        mock_message = SimpleNamespace(content="", tool_calls=[mock_tool_call1, mock_tool_call2])
+        mock_response = SimpleNamespace(message=mock_message, finish_reason="tool_calls")
+        
+        result = handler.parse_response(mock_response)
+        
+        assert len(result.tool_calls) == 2
+        assert result.tool_calls[0].name == "get_weather"
+        assert result.tool_calls[1].name == "get_time"
+
+    def test_parse_response_empty_tool_calls(self):
+        """Test parsing response with empty tool_calls."""
+        handler = OpenAILLMHandler()
+        
+        mock_message = SimpleNamespace(content="No tools needed", tool_calls=None)
+        mock_response = SimpleNamespace(message=mock_message, finish_reason="stop")
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == "No tools needed"
+        assert result.tool_calls == []
+        assert result.finish_reason == "stop"
+
+    def test_parse_response_missing_attributes(self):
+        """Test parsing response with missing attributes."""
+        handler = OpenAILLMHandler()
+        
+        # Mock response with missing attributes
+        mock_message = SimpleNamespace()  # No content or tool_calls
+        mock_response = SimpleNamespace(message=mock_message)  # No finish_reason
+        
+        result = handler.parse_response(mock_response)
+        
+        assert result.content == ""
+        assert result.tool_calls == []
+        assert result.finish_reason == ""
+
+    def test_create_tool_message(self):
+        """Test creating tool message."""
+        handler = OpenAILLMHandler()
+        
+        tool_call = ToolCall(
+            id="call_123",
+            name="get_weather",
+            arguments={"location": "NYC"},
+            index=0
+        )
+        result = {"temperature": "72F", "condition": "sunny"}
+        
+        message = handler.create_tool_message(tool_call, result)
+        
+        expected = {
+            "role": "tool",
+            "content": [
+                {
+                    "function_response": {
+                        "name": "get_weather",
+                        "response": {"result": result},
+                        "call_id": "call_123",
+                    }
+                }
+            ],
+        }
+        
+        assert message == expected
+
+    def test_create_tool_message_string_result(self):
+        """Test creating tool message with string result."""
+        handler = OpenAILLMHandler()
+        
+        tool_call = ToolCall(id="call_456", name="get_time", arguments={})
+        result = "2023-12-01 10:30:00"
+        
+        message = handler.create_tool_message(tool_call, result)
+        
+        assert message["role"] == "tool"
+        assert message["content"][0]["function_response"]["response"]["result"] == result
+        assert message["content"][0]["function_response"]["call_id"] == "call_456"
+
+    def test_iterate_stream(self):
+        """Test stream iteration."""
+        handler = OpenAILLMHandler()
+        
+        # Mock streaming response
+        mock_chunks = ["chunk1", "chunk2", "chunk3"]
+        
+        result = list(handler._iterate_stream(mock_chunks))
+        
+        assert result == mock_chunks
+
+    def test_iterate_stream_empty(self):
+        """Test stream iteration with empty response."""
+        handler = OpenAILLMHandler()
+        
+        result = list(handler._iterate_stream([]))
+        
+        assert result == []
+
+    def test_parse_response_tool_call_missing_attributes(self):
+        """Test parsing tool calls with missing attributes."""
+        handler = OpenAILLMHandler()
+        
+        # Mock tool call with missing attributes
+        mock_function = SimpleNamespace()  # No name or arguments
+        mock_tool_call = SimpleNamespace(function=mock_function)  # No id or index
+        
+        mock_message = SimpleNamespace(content="", tool_calls=[mock_tool_call])
+        mock_response = SimpleNamespace(message=mock_message, finish_reason="tool_calls")
+        
+        result = handler.parse_response(mock_response)
+        
+        assert len(result.tool_calls) == 1
+        assert result.tool_calls[0].id == ""
+        assert result.tool_calls[0].name == ""
+        assert result.tool_calls[0].arguments == ""
+        assert result.tool_calls[0].index is None

tests/llm/test_anthropic.py

@@ -1,68 +0,0 @@
-import unittest
-from unittest.mock import patch, Mock
-from application.llm.anthropic import AnthropicLLM
-
-class TestAnthropicLLM(unittest.TestCase):
-
-    def setUp(self):
-        self.api_key = "TEST_API_KEY"
-        self.llm = AnthropicLLM(api_key=self.api_key)
-
-    @patch("application.llm.anthropic.settings")
-    def test_init_default_api_key(self, mock_settings):
-        mock_settings.ANTHROPIC_API_KEY = "DEFAULT_API_KEY"
-        llm = AnthropicLLM()
-        self.assertEqual(llm.api_key, "DEFAULT_API_KEY")
-
-    def test_gen(self):
-        messages = [
-            {"content": "context"},
-            {"content": "question"}
-        ]
-        mock_response = Mock()
-        mock_response.completion = "test completion"
-
-        with patch("application.cache.get_redis_instance") as mock_make_redis:
-            mock_redis_instance = mock_make_redis.return_value
-            mock_redis_instance.get.return_value = None
-            mock_redis_instance.set = Mock()
-
-            with patch.object(self.llm.anthropic.completions, "create", return_value=mock_response) as mock_create:
-                response = self.llm.gen("test_model", messages)
-                self.assertEqual(response, "test completion")
-
-                prompt_expected = "### Context \n context \n ### Question \n question"
-                mock_create.assert_called_with(
-                    model="test_model",
-                    max_tokens_to_sample=300,
-                    stream=False,
-                    prompt=f"{self.llm.HUMAN_PROMPT} {prompt_expected}{self.llm.AI_PROMPT}"
-                )
-            mock_redis_instance.set.assert_called_once()
-
-    def test_gen_stream(self):
-        messages = [
-            {"content": "context"},
-            {"content": "question"}
-        ]
-        mock_responses = [Mock(completion="response_1"), Mock(completion="response_2")]
-        mock_tools = Mock()
-
-        with patch("application.cache.get_redis_instance") as mock_make_redis:
-            mock_redis_instance = mock_make_redis.return_value
-            mock_redis_instance.get.return_value = None
-            mock_redis_instance.set = Mock()
-
-            with patch.object(self.llm.anthropic.completions, "create", return_value=iter(mock_responses)) as mock_create:
-                responses = list(self.llm.gen_stream("test_model", messages, tools=mock_tools))
-                self.assertListEqual(responses, ["response_1", "response_2"])
-
-                prompt_expected = "### Context \n context \n ### Question \n question"
-                mock_create.assert_called_with(
-                    model="test_model",
-                    prompt=f"{self.llm.HUMAN_PROMPT} {prompt_expected}{self.llm.AI_PROMPT}",
-                    max_tokens_to_sample=300,
-                    stream=True
-                )
-if __name__ == "__main__":
-    unittest.main()

tests/llm/test_anthropic_llm.py

@@ -0,0 +1,65 @@
+import sys
+import types
+import pytest
+
+class _FakeCompletion:
+    def __init__(self, text):
+        self.completion = text
+
+class _FakeCompletions:
+    def __init__(self):
+        self.last_kwargs = None
+        self._stream = [_FakeCompletion("s1"), _FakeCompletion("s2")]
+
+    def create(self, **kwargs):
+        self.last_kwargs = kwargs
+        if kwargs.get("stream"):
+            return self._stream
+        return _FakeCompletion("final")
+
+class _FakeAnthropic:
+    def __init__(self, api_key=None):
+        self.api_key = api_key
+        self.completions = _FakeCompletions()
+
+
+@pytest.fixture(autouse=True)
+def patch_anthropic(monkeypatch):
+    fake = types.ModuleType("anthropic")
+    fake.Anthropic = _FakeAnthropic
+    fake.HUMAN_PROMPT = "<HUMAN>"
+    fake.AI_PROMPT = "<AI>"
+    sys.modules["anthropic"] = fake
+    yield
+    sys.modules.pop("anthropic", None)
+
+
+def test_anthropic_raw_gen_builds_prompt_and_returns_completion():
+    from application.llm.anthropic import AnthropicLLM
+
+    llm = AnthropicLLM(api_key="k")
+    msgs = [
+        {"content": "ctx"},
+        {"content": "q"},
+    ]
+    out = llm._raw_gen(llm, model="claude-2", messages=msgs, stream=False, max_tokens=55)
+    assert out == "final"
+    last = llm.anthropic.completions.last_kwargs
+    assert last["model"] == "claude-2"
+    assert last["max_tokens_to_sample"] == 55
+    assert last["prompt"].startswith("<HUMAN>") and last["prompt"].endswith("<AI>")
+    assert "### Context" in last["prompt"] and "### Question" in last["prompt"]
+
+
+def test_anthropic_raw_gen_stream_yields_chunks():
+    from application.llm.anthropic import AnthropicLLM
+
+    llm = AnthropicLLM(api_key="k")
+    msgs = [
+        {"content": "ctx"},
+        {"content": "q"},
+    ]
+    gen = llm._raw_gen_stream(llm, model="claude", messages=msgs, stream=True, max_tokens=10)
+    chunks = list(gen)
+    assert chunks == ["s1", "s2"]
+

tests/llm/test_google_llm.py

@@ -0,0 +1,151 @@
+import types
+import pytest
+
+from application.llm.google_ai import GoogleLLM
+
+class _FakePart:
+    def __init__(self, text=None, function_call=None, file_data=None):
+        self.text = text
+        self.function_call = function_call
+        self.file_data = file_data
+
+    @staticmethod
+    def from_text(text):
+        return _FakePart(text=text)
+
+    @staticmethod
+    def from_function_call(name, args):
+        return _FakePart(function_call=types.SimpleNamespace(name=name, args=args))
+
+    @staticmethod
+    def from_function_response(name, response):
+        # not used in assertions but present for completeness
+        return _FakePart(function_call=None, text=str(response))
+
+    @staticmethod
+    def from_uri(file_uri, mime_type):
+        # mimic presence of file data for streaming detection
+        return _FakePart(file_data=types.SimpleNamespace(file_uri=file_uri, mime_type=mime_type))
+
+
+class _FakeContent:
+    def __init__(self, role, parts):
+        self.role = role
+        self.parts = parts
+
+
+class FakeTypesModule:
+    Part = _FakePart
+    Content = _FakeContent
+
+    class GenerateContentConfig:
+        def __init__(self):
+            self.system_instruction = None
+            self.tools = None
+            self.response_schema = None
+            self.response_mime_type = None
+
+
+class FakeModels:
+    def __init__(self):
+        self.last_args = None
+        self.last_kwargs = None
+
+    class _Resp:
+        def __init__(self, text=None, candidates=None):
+            self.text = text
+            self.candidates = candidates or []
+
+    def generate_content(self, *args, **kwargs):
+        self.last_args, self.last_kwargs = args, kwargs
+        return FakeModels._Resp(text="ok")
+
+    def generate_content_stream(self, *args, **kwargs):
+        self.last_args, self.last_kwargs = args, kwargs
+        # Simulate stream of text parts
+        part1 = types.SimpleNamespace(text="a", candidates=None)
+        part2 = types.SimpleNamespace(text="b", candidates=None)
+        return [part1, part2]
+
+
+class FakeClient:
+    def __init__(self, *_, **__):
+        self.models = FakeModels()
+
+
+@pytest.fixture(autouse=True)
+def patch_google_modules(monkeypatch):
+    # Patch the types module used by GoogleLLM
+    import application.llm.google_ai as gmod
+    monkeypatch.setattr(gmod, "types", FakeTypesModule)
+    monkeypatch.setattr(gmod.genai, "Client", FakeClient)
+
+
+def test_clean_messages_google_basic():
+    llm = GoogleLLM(api_key="key")
+    msgs = [
+        {"role": "assistant", "content": "hi"},
+        {"role": "user", "content": [
+            {"text": "hello"},
+            {"files": [{"file_uri": "gs://x", "mime_type": "image/png"}]},
+            {"function_call": {"name": "fn", "args": {"a": 1}}},
+        ]},
+    ]
+    cleaned = llm._clean_messages_google(msgs)
+
+    assert all(hasattr(c, "role") and hasattr(c, "parts") for c in cleaned)
+    assert any(c.role == "model" for c in cleaned)
+    assert any(hasattr(p, "text") for c in cleaned for p in c.parts)
+
+
+def test_raw_gen_calls_google_client_and_returns_text():
+    llm = GoogleLLM(api_key="key")
+    msgs = [{"role": "user", "content": "hello"}]
+    out = llm._raw_gen(llm, model="gemini-2.0", messages=msgs, stream=False)
+    assert out == "ok"
+
+
+def test_raw_gen_stream_yields_chunks():
+    llm = GoogleLLM(api_key="key")
+    msgs = [{"role": "user", "content": "hello"}]
+    gen = llm._raw_gen_stream(llm, model="gemini", messages=msgs, stream=True)
+    assert list(gen) == ["a", "b"]
+
+
+def test_prepare_structured_output_format_type_mapping():
+    llm = GoogleLLM(api_key="key")
+    schema = {
+        "type": "object",
+        "properties": {
+            "a": {"type": "string"},
+            "b": {"type": "array", "items": {"type": "integer"}},
+        },
+        "required": ["a"],
+    }
+    out = llm.prepare_structured_output_format(schema)
+    assert out["type"] == "OBJECT"
+    assert out["properties"]["a"]["type"] == "STRING"
+    assert out["properties"]["b"]["type"] == "ARRAY"
+
+
+def test_prepare_messages_with_attachments_appends_files(monkeypatch):
+    llm = GoogleLLM(api_key="key")
+    llm.storage = types.SimpleNamespace(
+        file_exists=lambda path: True,
+        process_file=lambda path, processor_func, **kwargs: "gs://file_uri"
+    )
+    monkeypatch.setattr(llm, "_upload_file_to_google", lambda att: "gs://file_uri")
+
+    messages = [{"role": "user", "content": "Hi"}]
+    attachments = [
+        {"path": "/tmp/img.png", "mime_type": "image/png"},
+        {"path": "/tmp/doc.pdf", "mime_type": "application/pdf"},
+    ]
+
+    out = llm.prepare_messages_with_attachments(messages, attachments)
+    user_msg = next(m for m in out if m["role"] == "user")
+    assert isinstance(user_msg["content"], list)
+    files_entry = next((p for p in user_msg["content"] if isinstance(p, dict) and "files" in p), None)
+    assert files_entry is not None
+    assert isinstance(files_entry["files"], list) and len(files_entry["files"]) == 2
+