diff --git a/deployment/optional/docker-compose.optional.ollama-cpu.yaml b/deployment/optional/docker-compose.optional.ollama-cpu.yaml new file mode 100644 index 00000000..d7127314 --- /dev/null +++ b/deployment/optional/docker-compose.optional.ollama-cpu.yaml @@ -0,0 +1,11 @@ +version: "3.8" +services: + ollama: + image: ollama/ollama + ports: + - "11434:11434" + volumes: + - ollama_data:/root/.ollama + +volumes: + ollama_data: \ No newline at end of file diff --git a/deployment/optional/docker-compose.optional.ollama-gpu.yaml b/deployment/optional/docker-compose.optional.ollama-gpu.yaml new file mode 100644 index 00000000..17d79100 --- /dev/null +++ b/deployment/optional/docker-compose.optional.ollama-gpu.yaml @@ -0,0 +1,16 @@ +version: "3.8" +services: + ollama: + image: ollama/ollama + ports: + - "11434:11434" + volumes: + - ollama_data:/root/.ollama + deploy: + resources: + reservations: + devices: + - capabilities: [gpu] + +volumes: + ollama_data: \ No newline at end of file diff --git a/docs/pages/API/API-docs.md b/docs/pages/API/API-docs.md deleted file mode 100644 index a85ed6f8..00000000 --- a/docs/pages/API/API-docs.md +++ /dev/null @@ -1,350 +0,0 @@ -# API Endpoints Documentation - -*Currently, the application provides the following main API endpoints:* - - -### 1. /api/answer -**Description:** - -This endpoint is used to request answers to user-provided questions. - -**Request:** - -**Method**: `POST` - -**Headers**: Content-Type should be set to `application/json; charset=utf-8` - -**Request Body**: JSON object with the following fields: -* `question` — The user's question. -* `history` — (Optional) Previous conversation history. -* `api_key`— Your API key. -* `embeddings_key` — Your embeddings key. -* `active_docs` — The location of active documentation. - -Here is a JavaScript Fetch Request example: -```js -// answer (POST http://127.0.0.1:5000/api/answer) -fetch("http://127.0.0.1:5000/api/answer", { - "method": "POST", - "headers": { - "Content-Type": "application/json; charset=utf-8" - }, - "body": JSON.stringify({"question":"Hi","history":null,"api_key":"OPENAI_API_KEY","embeddings_key":"OPENAI_API_KEY", - "active_docs": "javascript/.project/ES2015/openai_text-embedding-ada-002/"}) -}) -.then((res) => res.text()) -.then(console.log.bind(console)) -``` - -**Response** - -In response, you will get a JSON document containing the `answer`, `query` and `result`: -```json -{ - "answer": "Hi there! How can I help you?\n", - "query": "Hi", - "result": "Hi there! How can I help you?\nSOURCES:" -} -``` - -### 2. /api/docs_check - -**Description:** - -This endpoint will make sure documentation is loaded on the server (just run it every time user is switching between libraries (documentations)). - -**Request:** - -**Method**: `POST` - -**Headers**: Content-Type should be set to `application/json; charset=utf-8` - -**Request Body**: JSON object with the field: -* `docs` — The location of the documentation: -```js -// docs_check (POST http://127.0.0.1:5000/api/docs_check) -fetch("http://127.0.0.1:5000/api/docs_check", { - "method": "POST", - "headers": { - "Content-Type": "application/json; charset=utf-8" - }, - "body": JSON.stringify({"docs":"javascript/.project/ES2015/openai_text-embedding-ada-002/"}) -}) -.then((res) => res.text()) -.then(console.log.bind(console)) -``` - -**Response:** - -In response, you will get a JSON document like this one indicating whether the documentation exists or not: -```json -{ - "status": "exists" -} -``` - - -### 3. /api/combine -**Description:** - -This endpoint provides information about available vectors and their locations with a simple GET request. - -**Request:** - -**Method**: `GET` - -**Response:** - -Response will include: -* `date` -* `description` -* `docLink` -* `fullName` -* `language` -* `location` (local or docshub) -* `model` -* `name` -* `version` - -Example of JSON in Docshub and local: - -image - -### 4. /api/upload -**Description:** - -This endpoint is used to upload a file that needs to be trained, response is JSON with task ID, which can be used to check on task's progress. - -**Request:** - -**Method**: `POST` - -**Request Body**: A multipart/form-data form with file upload and additional fields, including `user` and `name`. - -HTML example: - -```html -
- - - - - -
-``` - -**Response:** - -JSON response with a status and a task ID that can be used to check the task's progress. - - -### 5. /api/task_status -**Description:** - -This endpoint is used to get the status of a task (`task_id`) from `/api/upload` - -**Request:** - -**Method**: `GET` - -**Query Parameter**: `task_id` (task ID to check) - -**Sample JavaScript Fetch Request:** -```js -// Task status (Get http://127.0.0.1:5000/api/task_status) -fetch("http://localhost:5001/api/task_status?task_id=YOUR_TASK_ID", { - "method": "GET", - "headers": { - "Content-Type": "application/json; charset=utf-8" - }, -}) -.then((res) => res.text()) -.then(console.log.bind(console)) -``` - -**Response:** - -There are two types of responses: - -1. While the task is still running, the 'current' value will show progress from 0 to 100. - ```json - { - "result": { - "current": 1 - }, - "status": "PROGRESS" - } - ``` - -2. When task is completed: - ```json - { - "result": { - "directory": "temp", - "filename": "install.rst", - "formats": [ - ".rst", - ".md", - ".pdf" - ], - "name_job": "somename", - "user": "local" - }, - "status": "SUCCESS" - } - ``` - -### 6. /api/delete_old -**Description:** - -This endpoint is used to delete old Vector Stores. - -**Request:** - -**Method**: `GET` - -**Query Parameter**: `task_id` - -**Sample JavaScript Fetch Request:** -```js -// delete_old (GET http://127.0.0.1:5000/api/delete_old) -fetch("http://localhost:5001/api/delete_old?task_id=YOUR_TASK_ID", { - "method": "GET", - "headers": { - "Content-Type": "application/json; charset=utf-8" - }, -}) -.then((res) => res.text()) -.then(console.log.bind(console)) - -``` -**Response:** - -JSON response indicating the status of the operation: - -```json -{ "status": "ok" } -``` - -### 7. /api/get_api_keys -**Description:** - -The endpoint retrieves a list of API keys for the user. - -**Request:** - -**Method**: `GET` - -**Sample JavaScript Fetch Request:** -```js -// get_api_keys (GET http://127.0.0.1:5000/api/get_api_keys) -fetch("http://localhost:5001/api/get_api_keys", { - "method": "GET", - "headers": { - "Content-Type": "application/json; charset=utf-8" - }, -}) -.then((res) => res.text()) -.then(console.log.bind(console)) - -``` -**Response:** - -JSON response with a list of created API keys: - -```json -[ - { - "id": "string", - "name": "string", - "key": "string", - "source": "string" - }, - ... - ] -``` - -### 8. /api/create_api_key - -**Description:** - -Create a new API key for the user. - -**Request:** - -**Method**: `POST` - -**Headers**: Content-Type should be set to `application/json; charset=utf-8` - -**Request Body**: JSON object with the following fields: -* `name` — A name for the API key. -* `source` — The source documents that will be used. -* `prompt_id` — The prompt ID. -* `chunks` — The number of chunks used to process an answer. - -Here is a JavaScript Fetch Request example: -```js -// create_api_key (POST http://127.0.0.1:5000/api/create_api_key) -fetch("http://127.0.0.1:5000/api/create_api_key", { - "method": "POST", - "headers": { - "Content-Type": "application/json; charset=utf-8" - }, - "body": JSON.stringify({"name":"Example Key Name", - "source":"Example Source", - "prompt_id":"creative", - "chunks":"2"}) -}) -.then((res) => res.json()) -.then(console.log.bind(console)) -``` - -**Response** - -In response, you will get a JSON document containing the `id` and `key`: -```json -{ - "id": "string", - "key": "string" -} -``` - -### 9. /api/delete_api_key - -**Description:** - -Delete an API key for the user. - -**Request:** - -**Method**: `POST` - -**Headers**: Content-Type should be set to `application/json; charset=utf-8` - -**Request Body**: JSON object with the field: -* `id` — The unique identifier of the API key to be deleted. - -Here is a JavaScript Fetch Request example: -```js -// delete_api_key (POST http://127.0.0.1:5000/api/delete_api_key) -fetch("http://127.0.0.1:5000/api/delete_api_key", { - "method": "POST", - "headers": { - "Content-Type": "application/json; charset=utf-8" - }, - "body": JSON.stringify({"id":"API_KEY_ID"}) -}) -.then((res) => res.json()) -.then(console.log.bind(console)) -``` - -**Response:** - -In response, you will get a JSON document indicating the status of the operation: -```json -{ - "status": "ok" -} -``` \ No newline at end of file diff --git a/docs/pages/API/_meta.json b/docs/pages/API/_meta.json deleted file mode 100644 index 4873d38c..00000000 --- a/docs/pages/API/_meta.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "API-docs": { - "title": "🗂️️ API-docs", - "href": "/API/API-docs" - }, - "api-key-guide": { - "title": "🔐 API Keys guide", - "href": "/API/api-key-guide" - } -} \ No newline at end of file diff --git a/docs/pages/Deploying/Development-Environment.md b/docs/pages/Deploying/Development-Environment.md deleted file mode 100644 index 6a7b010c..00000000 --- a/docs/pages/Deploying/Development-Environment.md +++ /dev/null @@ -1,78 +0,0 @@ -## Development Environments - -### Spin up Mongo and Redis - -For development, only two containers are used from [docker-compose.yaml](https://github.com/arc53/DocsGPT/blob/main/deployment/docker-compose.yaml) (by deleting all services except for Redis and Mongo). -See file [docker-compose-dev.yaml](https://github.com/arc53/DocsGPT/blob/main/deployment/docker-compose-dev.yaml). - -Run - -``` -docker compose -f deployment/docker-compose-dev.yaml build -docker compose -f deployment/docker-compose-dev.yaml up -d -``` - -### Run the Backend - -> [!Note] -> Make sure you have Python 3.12 installed. - -1. Export required environment variables or prepare a `.env` file in the project folder: - - Copy [.env-template](https://github.com/arc53/DocsGPT/blob/main/application/.env-template) and create `.env`. - -(check out [`application/core/settings.py`](application/core/settings.py) if you want to see more config options.) - -2. (optional) Create a Python virtual environment: - You can follow the [Python official documentation](https://docs.python.org/3/tutorial/venv.html) for virtual environments. - -a) On Mac OS and Linux - -```commandline -python -m venv venv -. venv/bin/activate -``` - -b) On Windows - -```commandline -python -m venv venv - venv/Scripts/activate -``` - -3. Download embedding model and save it in the `model/` folder: -You can use the script below, or download it manually from [here](https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip), unzip it and save it in the `model/` folder. - -```commandline -wget https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip -unzip mpnet-base-v2.zip -d model -rm mpnet-base-v2.zip -``` - -4. Install dependencies for the backend: - -```commandline -pip install -r application/requirements.txt -``` - -5. Run the app using `flask --app application/app.py run --host=0.0.0.0 --port=7091`. -6. Start worker with `celery -A application.app.celery worker -l INFO`. - -> [!Note] -> You can also launch the in a debugger mode in vscode by accessing SHIFT + CMD + D or SHIFT + Windows + D on windows and selecting Flask or Celery. - - -### Start Frontend - -> [!Note] -> Make sure you have Node version 16 or higher. - -1. Navigate to the [/frontend](https://github.com/arc53/DocsGPT/tree/main/frontend) folder. -2. Install the required packages `husky` and `vite` (ignore if already installed). - -```commandline -npm install husky -g -npm install vite -g -``` - -3. Install dependencies by running `npm install --include=dev`. -4. Run the app using `npm run dev`. \ No newline at end of file diff --git a/docs/pages/Deploying/Development-Environment.mdx b/docs/pages/Deploying/Development-Environment.mdx new file mode 100644 index 00000000..2852be19 --- /dev/null +++ b/docs/pages/Deploying/Development-Environment.mdx @@ -0,0 +1,163 @@ +--- +title: Setting Up a Development Environment +description: Guide to setting up a development environment for DocsGPT, including backend and frontend setup. +--- + +# Setting Up a Development Environment + +This guide will walk you through setting up a development environment for DocsGPT. This setup allows you to modify and test the application's backend and frontend components. + +## 1. Spin Up MongoDB and Redis + +For development purposes, you can quickly start MongoDB and Redis containers, which are the primary database and caching systems used by DocsGPT. We provide a dedicated Docker Compose file, `docker-compose-dev.yaml`, located in the `deployment` directory, that includes only these essential services. + +You can find the `docker-compose-dev.yaml` file [here](https://github.com/arc53/DocsGPT/blob/main/deployment/docker-compose-dev.yaml). + +**Steps to start MongoDB and Redis:** + +1. Navigate to the root directory of your DocsGPT repository in your terminal. + +2. Run the following commands to build and start the containers defined in `docker-compose-dev.yaml`: + + ```bash + docker compose -f deployment/docker-compose-dev.yaml build + docker compose -f deployment/docker-compose-dev.yaml up -d + ``` + + These commands will start MongoDB and Redis in detached mode, running in the background. + +## 2. Run the Backend + +To run the DocsGPT backend locally, you'll need to set up a Python environment and install the necessary dependencies. + +**Prerequisites:** + +* **Python 3.12:** Ensure you have Python 3.12 installed on your system. You can check your Python version by running `python --version` or `python3 --version` in your terminal. + +**Steps to run the backend:** + +1. **Configure Environment Variables:** + + DocsGPT backend settings are configured using environment variables. You can set these either in a `.env` file or directly in the `settings.py` file. For a comprehensive overview of all settings, please refer to the [DocsGPT Settings Guide](/Deploying/DocsGPT-Settings). + + * **Option 1: Using a `.env` file (Recommended):** + * If you haven't already, create a file named `.env` in the **root directory** of your DocsGPT project. + * Modify the `.env` file to adjust settings as needed. You can find a comprehensive list of configurable options in [`application/core/settings.py`](application/core/settings.py). + + * **Option 2: Exporting Environment Variables:** + * Alternatively, you can export environment variables directly in your terminal. However, using a `.env` file is generally more organized for development. + +2. **Create a Python Virtual Environment (Optional but Recommended):** + + Using a virtual environment isolates project dependencies and avoids conflicts with system-wide Python packages. + + * **macOS and Linux:** + + ```bash + python -m venv venv + . venv/bin/activate + ``` + + * **Windows:** + + ```bash + python -m venv venv + venv/Scripts/activate + ``` + +3. **Download Embedding Model:** + + The backend requires an embedding model. Download the `mpnet-base-v2` model and place it in the `model/` directory within the project root. You can use the following script: + + ```bash + wget https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip + unzip mpnet-base-v2.zip -d model + rm mpnet-base-v2.zip + ``` + + Alternatively, you can manually download the zip file from [here](https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip), unzip it, and place the extracted folder in `model/`. + +4. **Install Backend Dependencies:** + + Navigate to the root of your DocsGPT repository and install the required Python packages: + + ```bash + pip install -r application/requirements.txt + ``` + +5. **Run the Flask App:** + + Start the Flask backend application: + + ```bash + flask --app application/app.py run --host=0.0.0.0 --port=7091 + ``` + + This command will launch the backend server, making it accessible on `http://localhost:7091`. + +6. **Start the Celery Worker:** + + Open a new terminal window (and activate your virtual environment if you used one). Start the Celery worker to handle background tasks: + + ```bash + celery -A application.app.celery worker -l INFO + ``` + + This command will start the Celery worker, which processes tasks such as document parsing and vector embedding. + +**Running in Debugger (VSCode):** + +For easier debugging, you can launch the Flask app and Celery worker directly from VSCode's debugger. + +* Press Shift + Cmd + D (macOS) or Shift + Windows + D (Windows) to open the Run and Debug view. +* You should see configurations named "Flask" and "Celery". Select the desired configuration and click the "Start Debugging" button (green play icon). + +## 3. Start the Frontend + +To run the DocsGPT frontend locally, you'll need Node.js and npm (Node Package Manager). + +**Prerequisites:** + +* **Node.js version 16 or higher:** Ensure you have Node.js version 16 or greater installed. You can check your Node.js version by running `node -v` in your terminal. npm is usually bundled with Node.js. + +**Steps to start the frontend:** + +1. **Navigate to the Frontend Directory:** + + In your terminal, change the current directory to the `frontend` folder within your DocsGPT repository: + + ```bash + cd frontend + ``` + +2. **Install Global Packages (If Needed):** + + If you don't have `husky` and `vite` installed globally, you can install them: + + ```bash + npm install husky -g + npm install vite -g + ``` + You can skip this step if you already have these packages installed or prefer to use local installations (though global installation simplifies running the commands in this guide). + +3. **Install Frontend Dependencies:** + + Install the project's frontend dependencies using npm: + + ```bash + npm install --include=dev + ``` + + This command reads the `package.json` file in the `frontend` directory and installs all listed dependencies, including development dependencies. + +4. **Run the Frontend App:** + + Start the frontend development server: + + ```bash + npm run dev + ``` + + This command will start the Vite development server. The frontend application will typically be accessible at [http://localhost:5173/](http://localhost:5173/). The terminal will display the exact URL where the frontend is running. + +With both the backend and frontend running, you should now have a fully functional DocsGPT development environment. You can access the application in your browser at [http://localhost:5173/](http://localhost:5173/) and start developing! \ No newline at end of file diff --git a/docs/pages/Deploying/Docker-Deploying.mdx b/docs/pages/Deploying/Docker-Deploying.mdx new file mode 100644 index 00000000..559fa4e3 --- /dev/null +++ b/docs/pages/Deploying/Docker-Deploying.mdx @@ -0,0 +1,135 @@ +--- +title: Docker Deployment of DocsGPT +description: Deploy DocsGPT using Docker and Docker Compose for easy setup and management. +--- + +# Docker Deployment of DocsGPT + +Docker is the recommended method for deploying DocsGPT, providing a consistent and isolated environment for the application to run. This guide will walk you through deploying DocsGPT using Docker and Docker Compose. + +## Prerequisites + +* **Docker Engine:** You need to have Docker Engine installed on your system. + * **macOS:** [Docker Desktop for Mac](https://docs.docker.com/desktop/install/mac-install/) + * **Linux:** [Docker Engine Installation Guide](https://docs.docker.com/engine/install/) (follow instructions for your specific distribution) + * **Windows:** [Docker Desktop for Windows](https://docs.docker.com/desktop/install/windows-install/) (requires WSL 2 backend, see notes below) +* **Docker Compose:** Docker Compose is usually included with Docker Desktop. If you are using Docker Engine separately, ensure you have Docker Compose V2 installed. + +**Important Note for Windows Users:** Docker Desktop on Windows generally requires the WSL 2 backend to function correctly, especially when using features like host networking which are utilized in DocsGPT's Docker Compose setup. Ensure WSL 2 is enabled and configured in Docker Desktop settings. + +## Quickest Setup: Using DocsGPT Public API + +The fastest way to try out DocsGPT is by using the public API endpoint. This requires minimal configuration and no local LLM setup. + +1. **Clone the DocsGPT Repository (if you haven't already):** + + ```bash + git clone https://github.com/arc53/DocsGPT.git + cd DocsGPT + ``` + +2. **Create a `.env` file:** + + In the root directory of your DocsGPT repository, create a file named `.env`. + +3. **Add Public API Configuration to `.env`:** + + Open the `.env` file and add the following lines: + + ``` + LLM_NAME=docsgpt + VITE_API_STREAMING=true + ``` + + This minimal configuration tells DocsGPT to use the public API. For more advanced settings and other LLM options, refer to the [DocsGPT Settings Guide](/Deploying/DocsGPT-Settings). + +4. **Launch DocsGPT with Docker Compose:** + + Navigate to the root directory of the DocsGPT repository in your terminal and run: + + ```bash + docker compose -f deployment/docker-compose.yaml up -d + ``` + + The `-d` flag runs Docker Compose in detached mode (in the background). + +5. **Access DocsGPT in your browser:** + + Once the containers are running, open your web browser and go to [http://localhost:5173/](http://localhost:5173/). + +6. **Stopping DocsGPT:** + + To stop the application, navigate to the same directory in your terminal and run: + + ```bash + docker compose -f deployment/docker-compose.yaml down + ``` + +## Optional Ollama Setup (Local Models) + +DocsGPT provides optional Docker Compose files to easily integrate with [Ollama](https://ollama.com/) for running local models. These files add an official Ollama container to your Docker Compose setup. These files are located in the `deployment/optional/` directory. + +There are two Ollama optional files: + +* **`docker-compose.optional.ollama-cpu.yaml`**: For running Ollama on CPU. +* **`docker-compose.optional.ollama-gpu.yaml`**: For running Ollama on GPU (requires Docker to be configured for GPU usage). + +### Launching with Ollama and Pulling a Model + +1. **Clone the DocsGPT Repository and Create `.env` (as described above).** + +2. **Launch DocsGPT with Ollama Docker Compose:** + + Choose the appropriate Ollama Compose file (CPU or GPU) and launch DocsGPT: + + **CPU:** + ```bash + docker compose -f deployment/docker-compose.yaml -f deployment/optional/docker-compose.optional.ollama-cpu.yaml up -d + ``` + **GPU:** + ```bash + docker compose -f deployment/docker-compose.yaml -f deployment/optional/docker-compose.optional.ollama-gpu.yaml up -d + ``` + +3. **Pull the Ollama Model:** + + **Crucially, after launching with Ollama, you need to pull the desired model into the Ollama container.** Find the `MODEL_NAME` you configured in your `.env` file (e.g., `llama3.2:1b`). Then execute the following command to pull the model *inside* the running Ollama container: + + ```bash + docker compose -f deployment/docker-compose.yaml -f deployment/optional/docker-compose.optional.ollama-cpu.yaml exec -it ollama ollama pull + ``` + or (for GPU): + ```bash + docker compose -f deployment/docker-compose.yaml -f deployment/optional/docker-compose.optional.ollama-gpu.yaml exec -it ollama ollama pull + ``` + Replace `` with the actual model name from your `.env` file. + +4. **Access DocsGPT in your browser:** + + Once the model is pulled and containers are running, open your web browser and go to [http://localhost:5173/](http://localhost:5173/). + +5. **Stopping Ollama Setup:** + + To stop a DocsGPT setup launched with Ollama optional files, use `docker compose down` and include all the compose files used during the `up` command: + + ```bash + docker compose -f deployment/docker-compose.yaml -f deployment/optional/docker-compose.optional.ollama-cpu.yaml down + ``` + or + + ```bash + docker compose -f deployment/docker-compose.yaml -f deployment/optional/docker-compose.optional.ollama-gpu.yaml down + ``` + +**Important for GPU Usage:** + +* **NVIDIA Container Toolkit (for NVIDIA GPUs):** If you are using NVIDIA GPUs, you need to have the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) installed and configured on your system for Docker to access your GPU. +* **Docker GPU Configuration:** Ensure Docker is configured to utilize your GPU. Refer to the [Ollama Docker Hub page](https://hub.docker.com/r/ollama/ollama) and Docker documentation for GPU setup instructions specific to your GPU type (NVIDIA, AMD, Intel). + +## Restarting After Configuration Changes + +Whenever you modify the `.env` file or any Docker Compose files, you need to restart the Docker containers for the changes to be applied. Use the same `docker compose down` and `docker compose up -d` commands you used to launch DocsGPT, ensuring you include all relevant `-f` flags for optional files if you are using them. + +## Further Configuration + +This guide covers the basic Docker deployment of DocsGPT. For detailed information on configuring various aspects of DocsGPT, such as LLM providers, models, vector stores, and more, please refer to the comprehensive [DocsGPT Settings Guide](/Deploying/DocsGPT-Settings). \ No newline at end of file diff --git a/docs/pages/Deploying/DocsGPT-Settings.md b/docs/pages/Deploying/DocsGPT-Settings.md new file mode 100644 index 00000000..ce1e46ba --- /dev/null +++ b/docs/pages/Deploying/DocsGPT-Settings.md @@ -0,0 +1,107 @@ +--- +title: DocsGPT Settings +description: Configure your DocsGPT application by understanding the basic settings. +--- + +# DocsGPT Settings + +DocsGPT is highly configurable, allowing you to tailor it to your specific needs and preferences. You can control various aspects of the application, from choosing the Large Language Model (LLM) provider to selecting embedding models and vector stores. + +This document will guide you through the basic settings you can configure in DocsGPT. These settings determine how DocsGPT interacts with LLMs and processes your data. + +## Configuration Methods + +There are two primary ways to configure DocsGPT settings: + +### 1. Configuration via `.env` file (Recommended) + +The easiest and recommended way to configure basic settings is by using a `.env` file. This file should be located in the **root directory** of your DocsGPT project (the same directory where `setup.sh` is located). + +**Example `.env` file structure:** + +``` +LLM_NAME=openai +API_KEY=YOUR_OPENAI_API_KEY +MODEL_NAME=gpt-4o +``` + +### 2. Configuration via `settings.py` file (Advanced) + +For more advanced configurations or if you prefer to manage settings directly in code, you can modify the `settings.py` file. This file is located in the `application/core` directory of your DocsGPT project. + +While modifying `settings.py` offers more flexibility, it's generally recommended to use the `.env` file for basic settings and reserve `settings.py` for more complex adjustments or when you need to configure settings programmatically. + +**Location of `settings.py`:** `application/core/settings.py` + +## Basic Settings Explained + +Here are some of the most fundamental settings you'll likely want to configure: + +- **`LLM_NAME`**: This setting determines which Large Language Model (LLM) provider DocsGPT will use. It tells DocsGPT which API to interact with. + + - **Common values:** + - `docsgpt`: Use the DocsGPT Public API Endpoint (simple and free, as offered in `setup.sh` option 1). + - `openai`: Use OpenAI's API (requires an API key). + - `google`: Use Google's Vertex AI or Gemini models. + - `anthropic`: Use Anthropic's Claude models. + - `groq`: Use Groq's models. + - `huggingface`: Use HuggingFace Inference API. + - `azure_openai`: Use Azure OpenAI Service. + - `openai` (when using local inference engines like Ollama, Llama.cpp, TGI, etc.): This signals DocsGPT to use an OpenAI-compatible API format, even if the actual LLM is running locally. + +- **`MODEL_NAME`**: Specifies the specific model to use from the chosen LLM provider. The available models depend on the `LLM_NAME` you've selected. + + - **Examples:** + - For `LLM_NAME=openai`: `gpt-4o` + - For `LLM_NAME=google`: `gemini-2.0-flash` + - For local models (e.g., Ollama): `llama3.2:1b` (or any model name available in your setup). + +- **`EMBEDDINGS_NAME`**: This setting defines which embedding model DocsGPT will use to generate vector embeddings for your documents. Embeddings are numerical representations of text that allow DocsGPT to understand the semantic meaning of your documents for efficient search and retrieval. + + - **Default value:** `huggingface_sentence-transformers/all-mpnet-base-v2` (a good general-purpose embedding model). + - **Other options:** You can explore other embedding models from Hugging Face Sentence Transformers or other providers if needed. + +- **`API_KEY`**: Required for most cloud-based LLM providers. This is your authentication key to access the LLM provider's API. You'll need to obtain this key from your chosen provider's platform. + +- **`OPENAI_BASE_URL`**: Specifically used when `LLM_NAME` is set to `openai` but you are connecting to a local inference engine (like Ollama, Llama.cpp, etc.) that exposes an OpenAI-compatible API. This setting tells DocsGPT where to find your local LLM server. + +## Configuration Examples + +Let's look at some concrete examples of how to configure these settings in your `.env` file. + +### Example for Cloud API Provider (OpenAI) + +To use OpenAI's `gpt-4o` model, you would configure your `.env` file like this: + +``` +LLM_NAME=openai +API_KEY=YOUR_OPENAI_API_KEY # Replace with your actual OpenAI API key +MODEL_NAME=gpt-4o +``` + +Make sure to replace `YOUR_OPENAI_API_KEY` with your actual OpenAI API key. + +### Example for Local Deployment + +To use a local Ollama server with the `llama3.2:1b` model, you would configure your `.env` file like this: + +``` +LLM_NAME=openai # Using OpenAI compatible API format for local models +API_KEY=None # API Key is not needed for local Ollama +MODEL_NAME=llama3.2:1b +OPENAI_BASE_URL=http://host.docker.internal:11434/v1 # Default Ollama API URL within Docker +EMBEDDINGS_NAME=huggingface_sentence-transformers/all-mpnet-base-v2 # You can also run embeddings locally if needed +``` + +In this case, even though you are using Ollama locally, `LLM_NAME` is set to `openai` because Ollama (and many other local inference engines) are designed to be API-compatible with OpenAI. `OPENAI_BASE_URL` points DocsGPT to the local Ollama server. + +## Exploring More Settings + +These are just the basic settings to get you started. The `settings.py` file contains many more advanced options that you can explore to further customize DocsGPT, such as: + +- Vector store configuration (`VECTOR_STORE`, Qdrant, Milvus, LanceDB settings) +- Retriever settings (`RETRIEVERS_ENABLED`) +- Cache settings (`CACHE_REDIS_URL`) +- And many more! + +For a complete list of available settings and their descriptions, refer to the `settings.py` file in `application/core`. Remember to restart your Docker containers after making changes to your `.env` file or `settings.py` for the changes to take effect. \ No newline at end of file diff --git a/docs/pages/Deploying/Kubernetes-Deploying.md b/docs/pages/Deploying/Kubernetes-Deploying.mdx similarity index 92% rename from docs/pages/Deploying/Kubernetes-Deploying.md rename to docs/pages/Deploying/Kubernetes-Deploying.mdx index 7e8768cf..8f1c8f7a 100644 --- a/docs/pages/Deploying/Kubernetes-Deploying.md +++ b/docs/pages/Deploying/Kubernetes-Deploying.mdx @@ -1,4 +1,10 @@ -# Self-hosting DocsGPT on Kubernetes +--- +title: Deploying DocsGPT on Kubernetes +description: Learn how to self-host DocsGPT on a Kubernetes cluster for scalable and robust deployments. +--- + +# Self-hosting DocsGPT + on Kubernetes This guide will walk you through deploying DocsGPT on Kubernetes. diff --git a/docs/pages/Deploying/Railway-Deploying.md b/docs/pages/Deploying/Railway-Deploying.md deleted file mode 100644 index 770da2f4..00000000 --- a/docs/pages/Deploying/Railway-Deploying.md +++ /dev/null @@ -1,254 +0,0 @@ - -# Self-hosting DocsGPT on Railway - - - -Here's a step-by-step guide on how to host DocsGPT on Railway App. - - - -At first Clone and set up the project locally to run , test and Modify. - - - -### 1. Clone and GitHub SetUp - -a. Open Terminal (Windows Shell or Git bash(recommended)). - - - -b. Type `git clone https://github.com/arc53/DocsGPT.git` - - - -#### Download the package information - - - -Once it has finished cloning the repository, it is time to download the package information from all sources. To do so, simply enter the following command: - - - -`sudo apt update` - - - -#### Install Docker and Docker Compose - - - -DocsGPT backend and worker use Python, Frontend is written on React and the whole application is containerized using Docker. To install Docker and Docker Compose, enter the following commands: - - - -`sudo apt install docker.io` - - - -And now install docker-compose: - - - -`sudo apt install docker-compose` - - - -#### Access the DocsGPT Folder - - - -Enter the following command to access the folder in which the DocsGPT docker-compose file is present. - - - -`cd DocsGPT/` - - - -#### Prepare the Environment - - - -Inside the DocsGPT folder create a `.env` file and copy the contents of `.env_sample` into it. - - - -`nano .env` - - - -Make sure your `.env` file looks like this: - - - -``` - -OPENAI_API_KEY=(Your OpenAI API key) - -VITE_API_STREAMING=true - -SELF_HOSTED_MODEL=false - -``` - - - -To save the file, press CTRL+X, then Y, and then ENTER. - - - -Next, set the correct IP for the Backend by opening the docker-compose.yaml file: - - - -`nano deployment/docker-compose.yaml` - - - -And Change line 7 to: `VITE_API_HOST=http://localhost:7091` - -to this `VITE_API_HOST=http://:7091` - - - -This will allow the frontend to connect to the backend. - - - -#### Running the Application - - - -You're almost there! Now that all the necessary bits and pieces have been installed, it is time to run the application. To do so, use the following command: - - - -`sudo docker compose -f deployment/docker-compose.yaml up -d` - - - -Launching it for the first time will take a few minutes to download all the necessary dependencies and build. - - - -Once this is done you can go ahead and close the terminal window. - - - -### 2. Pushing it to your own Repository - - - -a. Create a Repository on your GitHub. - - - -b. Open Terminal in the same directory of the Cloned project. - - - -c. Type `git init` - - - -d. `git add .` - - - -e. `git commit -m "first-commit"` - - - -f. `git remote add origin ` - - - -g. `git push git push --set-upstream origin master` - -Your local files will now be pushed to your GitHub Account. :) - - -### 3. Create a Railway Account: - - - -If you haven't already, create or log in to your railway account do it by visiting [Railway](https://railway.app/) - - - -Signup via **GitHub** [Recommended]. - - - -### 4. Start New Project: - - - -a. Open Railway app and Click on "Start New Project." - - - -b. Choose any from the list of options available (Recommended "**Deploy from GitHub Repo**") - - - -c. Choose the required Repository from your GitHub. - - - -d. Configure and allow access to modify your GitHub content from the pop-up window. - - - -e. Agree to all the terms and conditions. - - - -PS: It may take a few minutes for the account setup to complete. - - - -#### You will get A free trial of $5 (use it for trial and then purchase if satisfied and needed) - - - -### 5. Connecting to Your newly Railway app with GitHub - - - -a. Choose DocsGPT repo from the list of your GitHub repository that you want to deploy now. - - - -b. Click on Deploy now. - - - -![Three Tabs will be there](/Railway-selection.png) - - - -c. Select Variables Tab. - - - -d. Upload the env file here that you used for local setup. - - - -e. Go to Settings Tab now. - - - -f. Go to "Networking" and click on Generate Domain Name, to get the URL of your hosted project. - - - -g. You can update the Root directory, build command, installation command as per need. - -*[However recommended not the disturb these options and leave them as default if not that needed.]* - - - - -Your own DocsGPT is now available at the Generated domain URl. :) diff --git a/docs/pages/Deploying/_meta.json b/docs/pages/Deploying/_meta.json index d01e1f67..e3ee7485 100644 --- a/docs/pages/Deploying/_meta.json +++ b/docs/pages/Deploying/_meta.json @@ -1,22 +1,22 @@ { - "Hosting-the-app": { - "title": "☁️ Hosting DocsGPT", - "href": "/Deploying/Hosting-the-app" + "DocsGPT-Settings": { + "title": "⚙️ App Configuration", + "href": "/Deploying/DocsGPT-Settings" }, - "Quickstart": { - "title": "⚡️Quickstart", - "href": "/Deploying/Quickstart" + "Docker-Deploying": { + "title": "🛳️ Docker Setup", + "href": "/Deploying/Docker-Deploying" }, "Development-Environment": { "title": "🛠️Development Environment", "href": "/Deploying/Development-Environment" }, - "Railway-Deploying": { - "title": "🚂Deploying on Railway", - "href": "/Deploying/Railway-Deploying" - }, "Kubernetes-Deploying": { - "title": "☸️Deploying on Kubernetes", + "title": "☸️ Deploying on Kubernetes", "href": "/Deploying/Kubernetes-Deploying" + }, + "Hosting-the-app": { + "title": "☁️ Hosting DocsGPT", + "href": "/Deploying/Hosting-the-app" } } \ No newline at end of file diff --git a/docs/pages/Extensions/_meta.json b/docs/pages/Extensions/_meta.json index 270367de..6a28a60e 100644 --- a/docs/pages/Extensions/_meta.json +++ b/docs/pages/Extensions/_meta.json @@ -1,14 +1,22 @@ { - "Chatwoot-extension": { - "title": "💬️ Chatwoot Extension", - "href": "/Extensions/Chatwoot-extension" + "api-key-guide": { + "title": "🔑 Getting API key", + "href": "/Extensions/api-key-guide" }, "react-widget": { - "title": "🏗️ Widget setup", - "href": "/Extensions/react-widget" + "title": "💬️ Chat Widget", + "href": "/Extensions/chat-widget" + }, + "search-widget": { + "title": "🔎 Search Widget", + "href": "/Extensions/search-widget" }, "Chrome-extension": { "title": "🌐 Chrome Extension", "href": "/Extensions/Chrome-extension" + }, + "Chatwoot-extension": { + "title": "🗣️ Chatwoot Extension", + "href": "/Extensions/Chatwoot-extension" } } \ No newline at end of file diff --git a/docs/pages/API/api-key-guide.md b/docs/pages/Extensions/api-key-guide.md similarity index 100% rename from docs/pages/API/api-key-guide.md rename to docs/pages/Extensions/api-key-guide.md diff --git a/docs/pages/Extensions/search-widget.md b/docs/pages/Extensions/search-widget.md new file mode 100644 index 00000000..e69de29b diff --git a/docs/pages/Models/_meta.json b/docs/pages/Models/_meta.json new file mode 100644 index 00000000..d1256cd1 --- /dev/null +++ b/docs/pages/Models/_meta.json @@ -0,0 +1,14 @@ +{ + "cloud-providers": { + "title": "☁️ Cloud Providers", + "href": "/Models/cloud-providers" + }, + "local-inference": { + "title": "🖥️ Local Inference", + "href": "/Models/local-inference" + }, + "embeddings": { + "title": "📝 Embeddings", + "href": "/Models/embeddings" + } +} \ No newline at end of file diff --git a/docs/pages/Models/cloud-providers.md b/docs/pages/Models/cloud-providers.md new file mode 100644 index 00000000..8097c777 --- /dev/null +++ b/docs/pages/Models/cloud-providers.md @@ -0,0 +1,79 @@ +--- +title: Connecting DocsGPT to LLM Providers +description: Explore the different Large Language Model (LLM) providers you can connect to DocsGPT, from cloud APIs to local inference engines. +--- + +# Connecting DocsGPT to LLM Providers + +DocsGPT is designed to be flexible and work with a variety of Large Language Model (LLM) providers. Whether you prefer the simplicity of a public API, the power of cloud-based models, or the control of local inference engines, DocsGPT can be configured to meet your needs. + +This guide will introduce you to the LLM providers that DocsGPT natively supports and explain how to connect to them. + +## Supported LLM Providers + +DocsGPT offers built-in support for the following LLM providers, selectable during the `setup.sh` script execution: + +**Cloud API Providers:** + +* **DocsGPT Public API** +* **OpenAI** +* **Google (Vertex AI, Gemini)** +* **Anthropic (Claude)** +* **Groq** +* **HuggingFace Inference API** +* **Azure OpenAI** + +## Configuration via `.env` file + +Connecting DocsGPT to an LLM provider is primarily configured through environment variables set in the `.env` file located in the root directory of your DocsGPT project. + +**Basic Configuration Parameters:** + +* **`LLM_NAME`**: This setting is crucial and specifies the provider you want to use. The values correspond to the provider names listed above (e.g., `docsgpt`, `openai`, `google`, `ollama`, etc.). +* **`MODEL_NAME`**: Determines the specific model to be used from the chosen provider (e.g., `gpt-4o`, `gemini-2.0-flash`, `llama3.2:1b`). Refer to the provider's documentation for available model names. +* **`API_KEY`**: Required for most cloud API providers. Obtain this key from your provider's platform and set it in the `.env` file. +* **`OPENAI_BASE_URL`**: Specifically used when connecting to a local inference engine that is OpenAI API compatible. This setting points DocsGPT to the address of your local server. + +## Configuration Examples + +Here are examples of `.env` configurations for different LLM providers. + +**Example for OpenAI:** + +To use OpenAI's `gpt-4o` model, your `.env` file would look like this: + +``` +LLM_NAME=openai +API_KEY=YOUR_OPENAI_API_KEY # Replace with your actual OpenAI API key +MODEL_NAME=gpt-4o +``` + +**Example for Local Ollama:** + +To connect to a local Ollama instance running `llama3.2:1b`, configure your `.env` as follows: + +``` +LLM_NAME=openai # Using OpenAI compatible API format for local models +API_KEY=None # API Key is not needed for local Ollama +MODEL_NAME=llama3.2:1b +OPENAI_BASE_URL=http://host.docker.internal:11434/v1 # Default Ollama API URL within Docker +``` + +**Example for OpenAI-Compatible API (DeepSeek):** + +Many LLM providers offer APIs that are compatible with the OpenAI API format. DeepSeek is one such example. To connect to DeepSeek, you would still use `LLM_NAME=openai` and point `OPENAI_BASE_URL` to the DeepSeek API endpoint. + +``` +LLM_NAME=openai +API_KEY=YOUR_DEEPSEEK_API_KEY # Your DeepSeek API key +MODEL_NAME=deepseek-chat # Or your desired DeepSeek model name +OPENAI_BASE_URL=https://api.deepseek.com/v1 # DeepSeek API base URL +``` + +**Important Note:** When using OpenAI-compatible APIs, you might need to adjust other settings as well, depending on the specific API's requirements. Always consult the provider's API documentation and the [DocsGPT Settings Guide](/Deploying/DocsGPT-Settings) for detailed configuration options. + +## Exploring More Providers and Advanced Settings + +The providers listed above are those with direct support in `setup.sh`. However, DocsGPT's flexible design allows you to connect to virtually any LLM provider that offers an API, especially those compatible with the OpenAI API standard. + +For a comprehensive list of all configurable settings, including advanced options for each provider and details on how to connect to other LLMs, please refer to the [DocsGPT Settings Guide](/Deploying/DocsGPT-Settings). This guide provides in-depth information on customizing your DocsGPT setup to work with a wide range of LLM providers and tailor the application to your specific needs. \ No newline at end of file diff --git a/docs/pages/Models/embeddings.md b/docs/pages/Models/embeddings.md new file mode 100644 index 00000000..e69de29b diff --git a/docs/pages/Models/local-inference.md b/docs/pages/Models/local-inference.md new file mode 100644 index 00000000..e69de29b diff --git a/docs/pages/Tools/_meta.json b/docs/pages/Tools/_meta.json new file mode 100644 index 00000000..e69de29b diff --git a/docs/pages/Tools/creating.md b/docs/pages/Tools/creating.md new file mode 100644 index 00000000..e69de29b diff --git a/docs/pages/Tools/main-rename.md b/docs/pages/Tools/main-rename.md new file mode 100644 index 00000000..e69de29b diff --git a/docs/pages/Tools/requests.md b/docs/pages/Tools/requests.md new file mode 100644 index 00000000..e69de29b diff --git a/docs/pages/_meta.json b/docs/pages/_meta.json new file mode 100644 index 00000000..aa2f4367 --- /dev/null +++ b/docs/pages/_meta.json @@ -0,0 +1,16 @@ + +{ + "index": "Home", + "quickstart": "Quickstart", + "Deploying": "Deploying", + "Models": "Models", + "Tools": "Tools", + "Extensions": "Extensions", + "https://gptcloud.arc53.com/": { + "title": "API", + "href": "https://gptcloud.arc53.com/", + "newWindow": true + }, + "Guides": "Guides", + "changelog": "Changelog" +} \ No newline at end of file diff --git a/docs/pages/Deploying/Quickstart.md b/docs/pages/changelog.md similarity index 98% rename from docs/pages/Deploying/Quickstart.md rename to docs/pages/changelog.md index cebf6bd7..7479c1cb 100644 --- a/docs/pages/Deploying/Quickstart.md +++ b/docs/pages/changelog.md @@ -1,3 +1,6 @@ +--- +title: 'Changelog' +--- ## Launching Web App **Note**: Make sure you have Docker installed diff --git a/docs/pages/index.mdx b/docs/pages/index.mdx index 423a0a01..4c617870 100644 --- a/docs/pages/index.mdx +++ b/docs/pages/index.mdx @@ -4,22 +4,19 @@ title: 'Home' import { Cards, Card } from 'nextra/components' import Image from 'next/image' import deployingGuides from './Deploying/_meta.json'; -import developingGuides from './API/_meta.json'; import extensionGuides from './Extensions/_meta.json'; import mainGuides from './Guides/_meta.json'; - - export const allGuides = { ...deployingGuides, - ...developingGuides, + ...extensionGuides, ...mainGuides, }; -### **DocsGPT 🦖** +## **DocsGPT 🦖** DocsGPT 🦖 is an innovative open-source tool designed to simplify the retrieval of information from project documentation using advanced GPT models 🤖. Eliminate lengthy manual searches 🔍 and enhance your documentation experience with DocsGPT, and consider contributing to its AI-powered future 🚀. @@ -33,6 +30,14 @@ DocsGPT 🦖 is an innovative open-source tool designed to simplify the retrieva Try it yourself: [https://www.docsgpt.cloud/](https://www.docsgpt.cloud/) +### Features: +- **Chat** - Ask any question on your mind and get instant answers +- **Bots** - Easily connect DocsGPT's *brain* to widgets and chatbots +- **Tools** - Build complex workflows with model performing all actions +- **RAG** - Connect any document to give model reliable knowledge +- **Open source** - Customize anything with clear codebase +- **Platform Agnostic** - Connect any LLM that suits your needs + ( diff --git a/docs/pages/quickstart.mdx b/docs/pages/quickstart.mdx new file mode 100644 index 00000000..cef1cd68 --- /dev/null +++ b/docs/pages/quickstart.mdx @@ -0,0 +1,84 @@ +--- +title: Quickstart - Launching DocsGPT Web App +description: Get started with DocsGPT quickly by launching the web application using the setup script. +--- + +# Quickstart + +**Prerequisites:** + +* **Docker:** Ensure you have Docker installed and running on your system. + +## Launching DocsGPT (macOS and Linux) + +The easiest way to launch DocsGPT is using the provided `setup.sh` script. This script automates the configuration process and offers several setup options. + +**Steps:** + +1. **Download the DocsGPT Repository:** + + First, you need to download the DocsGPT repository to your local machine. You can do this using Git: + + ```bash + git clone https://github.com/arc53/DocsGPT.git + cd DocsGPT + ``` + +2. **Run the `setup.sh` script:** + + Navigate to the DocsGPT directory in your terminal and execute the `setup.sh` script: + + ```bash + ./setup.sh + ``` + +3. **Follow the interactive setup:** + + The `setup.sh` script will guide you through an interactive menu with the following options: + + ``` + Welcome to DocsGPT Setup! + How would you like to proceed? + 1) Use DocsGPT Public API Endpoint (simple and free) + 2) Serve Local (with Ollama) + 3) Connect Local Inference Engine + 4) Connect Cloud API Provider + Choose option (1-4): + ``` + + Let's break down each option: + + * **1) Use DocsGPT Public API Endpoint (simple and free):** This is the simplest option to get started. It utilizes the DocsGPT public API, requiring no API keys or local model downloads. Choose this for a quick and easy setup. + + * **2) Serve Local (with Ollama):** This option allows you to run a Large Language Model locally using [Ollama](https://ollama.com/). You'll be prompted to choose between CPU or GPU for Ollama and select a model to download. This is a good option for local processing and experimentation. + + * **3) Connect Local Inference Engine:** If you are already running a local inference engine like Llama.cpp, Text Generation Inference (TGI), vLLM, or others, choose this option. You'll be asked to select your engine and provide the necessary connection details. This is for users with existing local LLM infrastructure. + + * **4) Connect Cloud API Provider:** This option lets you connect DocsGPT to a commercial Cloud API provider such as OpenAI, Google (Vertex AI/Gemini), Anthropic (Claude), Groq, HuggingFace Inference API, or Azure OpenAI. You will need an API key from your chosen provider. Select this if you prefer to use a powerful cloud-based LLM. + + After selecting an option and providing any required information (like API keys or model names), the script will configure your `.env` file and start DocsGPT using Docker Compose. + +4. **Access DocsGPT in your browser:** + + Once the setup is complete and Docker containers are running, navigate to [http://localhost:5173/](http://localhost:5173/) in your web browser to access the DocsGPT web application. + +5. **Stopping DocsGPT:** + + To stop DocsGPT, simply open a new terminal in the `DocsGPT` directory and run: + + ```bash + docker compose -f deployment/docker-compose.yaml down + ``` + (or the specific `docker compose` command shown at the end of the `setup.sh` execution, which may include optional compose files depending on your choices). + +## Launching DocsGPT (Windows) + +For Windows users, we recommend following the Docker deployment guide for detailed instructions. Please refer to the [Docker Deployment documentation](/Deploying/Docker-Deploying) for step-by-step instructions on setting up DocsGPT on Windows using Docker. + +**Important for Windows:** Ensure Docker Desktop is installed and running correctly on your Windows system before proceeding. + +## Advanced Configuration + +For more advanced customization of DocsGPT settings, such as configuring vector stores, embedding models, and other parameters, please refer to the [DocsGPT Settings documentation](/Deploying/DocsGPT-Settings). This guide explains how to modify the `.env` file or `settings.py` for deeper configuration. + +Enjoy using DocsGPT! \ No newline at end of file diff --git a/setup.sh b/setup.sh index 3b2c7855..7775e24e 100755 --- a/setup.sh +++ b/setup.sh @@ -1,14 +1,79 @@ #!/bin/bash -# Function to prompt the user for their choice -prompt_user() { - echo "Do you want to:" - echo "1. Use DocsGPT public API (simple and free)" - echo "2. Download the language model locally (12GB)" - echo "3. Use the OpenAI API (requires an API key)" - read -p "Enter your choice (1, 2 or 3): " choice +# Color codes +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +DEFAULT_FG='\033[39m' +RED='\033[0;31m' +NC='\033[0m' +BOLD='\033[1m' + +# Base Compose file (relative to script location) +COMPOSE_FILE="$(dirname "$(readlink -f "$0")")/deployment/docker-compose.yaml" +ENV_FILE="$(dirname "$(readlink -f "$0")")/.env" + +# Animation function +animate_dino() { + tput civis # Hide cursor + local dino_lines=( + " ######### " + " ############# " + " ##################" + " ####################" + " ######################" + " ####################### ######" + " ############################### " + " ################################## " + " ################ ############ " + " ################## ########## " + " ##################### ######## " + " ###################### ###### ### " + " ############ ########## #### ## " + " ############# ######### ##### " + " ############## ######### " + " ############## ########## " + "############ ####### " + " ###### ###### #### " + " ################ " + " ################# " + ) + + # Static DocsGPT text + local static_text=( + " ____ ____ ____ _____ " + " | _ \\ ___ ___ ___ / ___| _ \\_ _|" + " | | | |/ _ \\ / __/ __| | _| |_) || | " + " | |_| | (_) | (__\\__ \\ |_| | __/ | | " + " |____/ \\___/ \\___|___/\\____|_| |_| " + " " + ) + + # Print static text + clear + for line in "${static_text[@]}"; do + echo "$line" + done + + tput sc + + # Build-up animation + for i in "${!dino_lines[@]}"; do + tput rc + for ((j=0; j<=i; j++)); do + echo "${dino_lines[$j]}" + done + sleep 0.05 + done + + sleep 0.5 + + tput rc + tput ed + + tput cnorm } +# Check and start Docker function check_and_start_docker() { # Check if Docker is running if ! docker info > /dev/null 2>&1; then @@ -31,110 +96,394 @@ check_and_start_docker() { # Wait for Docker to be fully operational with animated dots echo -n "Waiting for Docker to start" while ! docker system info > /dev/null 2>&1; do - for _ in {1..3}; do + for i in {1..3}; do echo -n "." sleep 1 done - echo -ne "\rWaiting for Docker to start " # Reset to overwrite previous dots + echo -ne "\rWaiting for Docker to start " done echo -e "\nDocker has started!" fi } -# Function to handle the choice to download the model locally -download_locally() { - echo "LLM_NAME=llama.cpp" > .env - echo "VITE_API_STREAMING=true" >> .env - echo "EMBEDDINGS_NAME=huggingface_sentence-transformers/all-mpnet-base-v2" >> .env - echo "The .env file has been created with LLM_NAME set to llama.cpp." - - # Creating the directory if it does not exist - mkdir -p models - - # Downloading the model to the specific directory - echo "Downloading the model..." - # check if docsgpt-7b-f16.gguf does not exist - if [ ! -f models/docsgpt-7b-f16.gguf ]; then - echo "Downloading the model..." - wget -P models https://d3dg1063dc54p9.cloudfront.net/models/docsgpt-7b-f16.gguf - echo "Model downloaded to models directory." - else - echo "Model already exists." - fi - - # Call the function to check and start Docker if needed - check_and_start_docker - - docker compose -f deployment/docker-compose-local.yaml build && docker compose -f deployment/docker-compose-local.yaml up -d - #python -m venv venv - #source venv/bin/activate - pip install -r application/requirements.txt - pip install llama-cpp-python - pip install sentence-transformers - export LLM_NAME=llama.cpp - export EMBEDDINGS_NAME=huggingface_sentence-transformers/all-mpnet-base-v2 - export FLASK_APP=application/app.py - export FLASK_DEBUG=true - export CELERY_BROKER_URL=redis://localhost:6379/0 - export CELERY_RESULT_BACKEND=redis://localhost:6379/1 - echo "The application is now running on http://localhost:5173" - echo "You can stop the application by running the following command:" - echo "Ctrl + C and then" - echo "Then pkill -f 'flask run' and then" - echo "docker compose down" - flask run --host=0.0.0.0 --port=7091 & - celery -A application.app.celery worker -l INFO +# Function to prompt the user for the main menu choice +prompt_main_menu() { + echo -e "\n${DEFAULT_FG}${BOLD}Welcome to DocsGPT Setup!${NC}" + echo -e "${DEFAULT_FG}How would you like to proceed?${NC}" + echo -e "${YELLOW}1) Use DocsGPT Public API Endpoint (simple and free)${NC}" + echo -e "${YELLOW}2) Serve Local (with Ollama)${NC}" + echo -e "${YELLOW}3) Connect Local Inference Engine${NC}" + echo -e "${YELLOW}4) Connect Cloud API Provider${NC}" + echo + read -p "$(echo -e "${DEFAULT_FG}Choose option (1-4): ${NC}")" main_choice } -# Function to handle the choice to use the OpenAI API -use_openai() { - read -p "Please enter your OpenAI API key: " api_key - echo "API_KEY=$api_key" > .env - echo "LLM_NAME=openai" >> .env - echo "VITE_API_STREAMING=true" >> .env - echo "The .env file has been created with API_KEY set to your provided key." - - # Call the function to check and start Docker if needed - check_and_start_docker - - docker compose -f deployment/docker-compose.yaml build && docker compose -f deployment/docker-compose.yaml up -d - - echo "The application will run on http://localhost:5173" - echo "You can stop the application by running the following command:" - echo "docker compose down" +# Function to prompt for Local Inference Engine options +prompt_local_inference_engine_options() { + clear + echo -e "\n${DEFAULT_FG}${BOLD}Connect Local Inference Engine${NC}" + echo -e "${DEFAULT_FG}Choose your local inference engine:${NC}" + echo -e "${YELLOW}1) LLaMa.cpp${NC}" + echo -e "${YELLOW}2) Ollama${NC}" + echo -e "${YELLOW}3) Text Generation Inference (TGI)${NC}" + echo -e "${YELLOW}4) SGLang${NC}" + echo -e "${YELLOW}5) vLLM${NC}" + echo -e "${YELLOW}6) Aphrodite${NC}" + echo -e "${YELLOW}7) FriendliAI${NC}" + echo -e "${YELLOW}8) LMDeploy${NC}" + echo -e "${YELLOW}b) Back to Main Menu${NC}" + echo + read -p "$(echo -e "${DEFAULT_FG}Choose option (1-8, or b): ${NC}")" engine_choice } -use_docsgpt() { +# Function to prompt for Cloud API Provider options +prompt_cloud_api_provider_options() { + clear + echo -e "\n${DEFAULT_FG}${BOLD}Connect Cloud API Provider${NC}" + echo -e "${DEFAULT_FG}Choose your Cloud API Provider:${NC}" + echo -e "${YELLOW}1) OpenAI${NC}" + echo -e "${YELLOW}2) Google (Vertex AI, Gemini)${NC}" + echo -e "${YELLOW}3) Anthropic (Claude)${NC}" + echo -e "${YELLOW}4) Groq${NC}" + echo -e "${YELLOW}5) HuggingFace Inference API${NC}" + echo -e "${YELLOW}6) Azure OpenAI${NC}" + echo -e "${YELLOW}b) Back to Main Menu${NC}" + echo + read -p "$(echo -e "${DEFAULT_FG}Choose option (1-6, or b): ${NC}")" provider_choice +} + +# Function to prompt for Ollama CPU/GPU options +prompt_ollama_options() { + clear + echo -e "\n${DEFAULT_FG}${BOLD}Serve Local with Ollama${NC}" + echo -e "${DEFAULT_FG}Choose how to serve Ollama:${NC}" + echo -e "${YELLOW}1) CPU${NC}" + echo -e "${YELLOW}2) GPU${NC}" + echo -e "${YELLOW}b) Back to Main Menu${NC}" + echo + read -p "$(echo -e "${DEFAULT_FG}Choose option (1-2, or b): ${NC}")" ollama_choice +} + +# 1) Use DocsGPT Public API Endpoint (simple and free) +use_docs_public_api_endpoint() { + echo -e "\n${NC}Setting up DocsGPT Public API Endpoint...${NC}" echo "LLM_NAME=docsgpt" > .env echo "VITE_API_STREAMING=true" >> .env - echo "The .env file has been created with API_KEY set to your provided key." + echo -e "${GREEN}.env file configured for DocsGPT Public API.${NC}" - # Call the function to check and start Docker if needed check_and_start_docker - docker compose -f deployment/docker-compose.yaml build && docker compose -f deployment/docker-compose.yaml up -d + echo -e "\n${NC}Starting Docker Compose...${NC}" + docker compose --env-file "${ENV_FILE}" -f "${COMPOSE_FILE}" build && docker compose --env-file "${ENV_FILE}" -f "${COMPOSE_FILE}" up -d + docker_compose_status=$? # Capture exit status of docker compose - echo "The application will run on http://localhost:5173" - echo "You can stop the application by running the following command:" - echo "docker compose down" + echo "Docker Compose Exit Status: $docker_compose_status" + + if [ "$docker_compose_status" -ne 0 ]; then + echo -e "\n${RED}${BOLD}Error starting Docker Compose. Please ensure Docker Compose is installed and in your PATH.${NC}" + echo -e "${RED}Refer to Docker documentation for installation instructions: https://docs.docker.com/compose/install/${NC}" + exit 1 # Indicate failure and EXIT SCRIPT + fi + + echo -e "\n${GREEN}DocsGPT is now running on http://localhost:5173${NC}" + echo -e "${YELLOW}You can stop the application by running: docker compose -f \"${COMPOSE_FILE}\" down${NC}" } -# Prompt the user for their choice -prompt_user +# 2) Serve Local (with Ollama) +serve_local_ollama() { + local ollama_choice model_name + local docker_compose_file_suffix + local model_name_prompt + local default_model="llama3.2:1b" -# Handle the user's choice -case $choice in - 1) - use_docsgpt - ;; - 2) - download_locally - ;; - 3) - use_openai - ;; - *) - echo "Invalid choice. Please choose either 1 or 2." - ;; -esac + get_model_name_ollama() { + read -p "$(echo -e "${DEFAULT_FG}Enter Ollama Model Name (leave empty for default: ${default_model} (1.3GB)): ${NC}")" model_name_input + if [ -z "$model_name_input" ]; then + model_name="$default_model" # Set default model if input is empty + else + model_name="$model_name_input" # Use user-provided model name + fi + } + + + while true; do + clear + prompt_ollama_options + case "$ollama_choice" in + 1) # CPU + docker_compose_file_suffix="cpu" + get_model_name_ollama + break ;; + 2) # GPU + echo -e "\n${YELLOW}For this option to work correctly you need to have a supported GPU and configure Docker to utilize it.${NC}" + echo -e "${YELLOW}Refer to: https://hub.docker.com/r/ollama/ollama for more information.${NC}" + read -p "$(echo -e "${DEFAULT_FG}Continue with GPU setup? (y/b): ${NC}")" confirm_gpu + case "$confirm_gpu" in + y|Y) + docker_compose_file_suffix="gpu" + get_model_name_ollama + break ;; + b|B) clear; return ;; # Back to Main Menu + *) echo -e "\n${RED}Invalid choice. Please choose y or b.${NC}" ; sleep 1 ;; + esac + ;; + b|B) clear; return ;; # Back to Main Menu + *) echo -e "\n${RED}Invalid choice. Please choose 1-2, or b.${NC}" ; sleep 1 ;; + esac + done + + + echo -e "\n${NC}Configuring for Ollama ($(echo "$docker_compose_file_suffix" | tr '[:lower:]' '[:upper:]'))...${NC}" # Using tr for uppercase - more compatible + echo "API_KEY=xxxx" > .env # Placeholder API Key + echo "LLM_NAME=openai" >> .env + echo "MODEL_NAME=$model_name" >> .env + echo "VITE_API_STREAMING=true" >> .env + echo "OPENAI_BASE_URL=http://host.docker.internal:11434/v1" >> .env + echo "EMBEDDINGS_NAME=huggingface_sentence-transformers/all-mpnet-base-v2" >> .env + echo -e "${GREEN}.env file configured for Ollama ($(echo "$docker_compose_file_suffix" | tr '[:lower:]' '[:upper:]')${NC}${GREEN}).${NC}" + echo -e "${YELLOW}Note: MODEL_NAME is set to '${BOLD}$model_name${NC}${YELLOW}'. You can change it later in the .env file.${NC}" + + + check_and_start_docker + local compose_files=( + -f "${COMPOSE_FILE}" + -f "$(dirname "${COMPOSE_FILE}")/optional/docker-compose.optional.ollama-${docker_compose_file_suffix}.yaml" + ) + + echo -e "\n${NC}Starting Docker Compose with Ollama (${docker_compose_file_suffix})...${NC}" + docker compose --env-file "${ENV_FILE}" "${compose_files[@]}" build + docker compose --env-file "${ENV_FILE}" "${compose_files[@]}" up -d + docker_compose_status=$? + + echo "Docker Compose Exit Status: $docker_compose_status" # Debug output + + if [ "$docker_compose_status" -ne 0 ]; then + echo -e "\n${RED}${BOLD}Error starting Docker Compose. Please ensure Docker Compose is installed and in your PATH.${NC}" + echo -e "${RED}Refer to Docker documentation for installation instructions: https://docs.docker.com/compose/install/${NC}" + exit 1 # Indicate failure and EXIT SCRIPT + fi + + echo "Waiting for Ollama container to be ready..." + OLLAMA_READY=false + while ! $OLLAMA_READY; do + CONTAINER_STATUS=$(docker compose "${compose_files[@]}" ps --services --filter "status=running" --format '{{.Service}}') + if [[ "$CONTAINER_STATUS" == *"ollama"* ]]; then # Check if 'ollama' service is in running services + OLLAMA_READY=true + echo "Ollama container is running." + else + echo "Ollama container not yet ready, waiting..." + sleep 5 + fi + done + + echo "Pulling $model_name model for Ollama..." + docker compose --env-file "${ENV_FILE}" "${compose_files[@]}" exec -it ollama ollama pull "$model_name" + + + echo -e "\n${GREEN}DocsGPT is now running with Ollama (${docker_compose_file_suffix}) on http://localhost:5173${NC}" + printf -v compose_files_escaped "%q " "${compose_files[@]}" + echo -e "${YELLOW}You can stop the application by running: docker compose ${compose_files_escaped}down${NC}" +} + +# 3) Connect Local Inference Engine +connect_local_inference_engine() { + local engine_choice + local model_name_prompt model_name openai_base_url + + get_model_name() { + read -p "$(echo -e "${DEFAULT_FG}Enter Model Name (leave empty to set later as None): ${NC}")" model_name + if [ -z "$model_name" ]; then + model_name="None" + fi + } + + while true; do + clear + prompt_local_inference_engine_options + case "$engine_choice" in + 1) # LLaMa.cpp + engine_name="LLaMa.cpp" + openai_base_url="http://localhost:8000/v1" + get_model_name + break ;; + 2) # Ollama + engine_name="Ollama" + openai_base_url="http://localhost:11434/v1" + get_model_name + break ;; + 3) # TGI + engine_name="TGI" + openai_base_url="http://localhost:8080/v1" + get_model_name + break ;; + 4) # SGLang + engine_name="SGLang" + openai_base_url="http://localhost:30000/v1" + get_model_name + break ;; + 5) # vLLM + engine_name="vLLM" + openai_base_url="http://localhost:8000/v1" + get_model_name + break ;; + 6) # Aphrodite + engine_name="Aphrodite" + openai_base_url="http://localhost:2242/v1" + get_model_name + break ;; + 7) # FriendliAI + engine_name="FriendliAI" + openai_base_url="http://localhost:8997/v1" + get_model_name + break ;; + 8) # LMDeploy + engine_name="LMDeploy" + openai_base_url="http://localhost:23333/v1" + get_model_name + break ;; + b|B) clear; return ;; # Back to Main Menu + *) echo -e "\n${RED}Invalid choice. Please choose 1-8, or b.${NC}" ; sleep 1 ;; + esac + done + + echo -e "\n${NC}Configuring for Local Inference Engine: ${BOLD}${engine_name}...${NC}" + echo "API_KEY=None" > .env + echo "LLM_NAME=openai" >> .env + echo "MODEL_NAME=$model_name" >> .env + echo "VITE_API_STREAMING=true" >> .env + echo "OPENAI_BASE_URL=$openai_base_url" >> .env + echo "EMBEDDINGS_NAME=huggingface_sentence-transformers/all-mpnet-base-v2" >> .env + echo -e "${GREEN}.env file configured for ${BOLD}${engine_name}${NC}${GREEN} with OpenAI API format.${NC}" + echo -e "${YELLOW}Note: MODEL_NAME is set to '${BOLD}$model_name${NC}${YELLOW}'. You can change it later in the .env file.${NC}" + + check_and_start_docker + + echo -e "\n${NC}Starting Docker Compose...${NC}" + docker compose --env-file "${ENV_FILE}" -f "${COMPOSE_FILE}" build && docker compose -f "${COMPOSE_FILE}" up -d + docker_compose_status=$? + + echo "Docker Compose Exit Status: $docker_compose_status" # Debug output + + if [ "$docker_compose_status" -ne 0 ]; then + echo -e "\n${RED}${BOLD}Error starting Docker Compose. Please ensure Docker Compose is installed and in your PATH.${NC}" + echo -e "${RED}Refer to Docker documentation for installation instructions: https://docs.docker.com/compose/install/${NC}" + exit 1 # Indicate failure and EXIT SCRIPT + fi + + echo -e "\n${GREEN}DocsGPT is now configured to connect to ${BOLD}${engine_name}${NC}${GREEN} at ${BOLD}$openai_base_url${NC}" + echo -e "${YELLOW}Ensure your ${BOLD}${engine_name} inference server is running at that address${NC}" + echo -e "\n${GREEN}DocsGPT is running at http://localhost:5173${NC}" + echo -e "${YELLOW}You can stop the application by running: docker compose -f \"${COMPOSE_FILE}\" down${NC}" +} + + +# 4) Connect Cloud API Provider +connect_cloud_api_provider() { + local provider_choice api_key llm_name + local setup_result # Variable to store the return status + + get_api_key() { + echo -e "${YELLOW}Your API key will be stored locally in the .env file and will not be sent anywhere else${NC}" + read -p "$(echo -e "${DEFAULT_FG}Please enter your API key: ${NC}")" api_key + } + + while true; do + clear + prompt_cloud_api_provider_options + case "$provider_choice" in + 1) # OpenAI + provider_name="OpenAI" + llm_name="openai" + model_name="gpt-4o" + get_api_key + break ;; + 2) # Google + provider_name="Google (Vertex AI, Gemini)" + llm_name="google" + model_name="gemini-2.0-flash" + get_api_key + break ;; + 3) # Anthropic + provider_name="Anthropic (Claude)" + llm_name="anthropic" + model_name="claude-3-5-sonnet-latest" + get_api_key + break ;; + 4) # Groq + provider_name="Groq" + llm_name="groq" + model_name="llama-3.1-8b-instant" + get_api_key + break ;; + 5) # HuggingFace Inference API + provider_name="HuggingFace Inference API" + llm_name="huggingface" + model_name="meta-llama/Llama-3.1-8B-Instruct" + get_api_key + break ;; + 6) # Azure OpenAI + provider_name="Azure OpenAI" + llm_name="azure_openai" + model_name="gpt-4o" + get_api_key + break ;; + b|B) clear; return ;; # Clear screen and Back to Main Menu + *) echo -e "\n${RED}Invalid choice. Please choose 1-6, or b.${NC}" ; sleep 1 ;; + esac + done + + echo -e "\n${NC}Configuring for Cloud API Provider: ${BOLD}${provider_name}...${NC}" + echo "API_KEY=$api_key" > .env + echo "LLM_NAME=$llm_name" >> .env + echo "MODEL_NAME=$model_name" >> .env + echo "VITE_API_STREAMING=true" >> .env + echo -e "${GREEN}.env file configured for ${BOLD}${provider_name}${NC}${GREEN}.${NC}" + + check_and_start_docker + + echo -e "\n${NC}Starting Docker Compose...${NC}" + docker compose --env-file "${ENV_FILE}" -f "${COMPOSE_FILE}" build && docker compose -f "${COMPOSE_FILE}" up -d + docker_compose_status=$? + + echo "Docker Compose Exit Status: $docker_compose_status" # Debug output + + if [ "$docker_compose_status" -ne 0 ]; then + echo -e "\n${RED}${BOLD}Error starting Docker Compose. Please ensure Docker Compose is installed and in your PATH.${NC}" + echo -e "${RED}Refer to Docker documentation for installation instructions: https://docs.docker.com/compose/install/${NC}" + exit 1 # Indicate failure and EXIT SCRIPT + fi + + echo -e "\n${GREEN}DocsGPT is now configured to use ${BOLD}${provider_name}${NC}${GREEN} on http://localhost:5173${NC}" + echo -e "${YELLOW}You can stop the application by running: docker compose -f \"${COMPOSE_FILE}\" down${NC}" +} + + +# Main script execution +animate_dino + +while true; do # Main menu loop + clear # Clear screen before showing main menu again + prompt_main_menu + + case $main_choice in + 1) # Use DocsGPT Public API Endpoint + use_docs_public_api_endpoint + ;; + 2) # Serve Local (with Ollama) + serve_local_ollama + ;; + 3) # Connect Local Inference Engine + connect_local_inference_engine + ;; + 4) # Connect Cloud API Provider + connect_cloud_api_provider + ;; + *) + echo -e "\n${RED}Invalid choice. Please choose 1-4.${NC}" ; sleep 1 ;; + esac +done + +echo -e "\n${GREEN}${BOLD}DocsGPT Setup Complete.${NC}" + +exit 0 \ No newline at end of file