mirror of
https://github.com/arc53/DocsGPT.git
synced 2025-11-29 08:33:20 +00:00
Merge pull request #1604 from arc53/update-docs-readme
Update docs readme
This commit is contained in:
Alex
@@ -27,6 +27,7 @@ Before creating issues, please check out how the latest version of our app looks
|
||||
|
||||
### 👨💻 If you're interested in contributing code, here are some important things to know:
|
||||
|
||||
For instructions on setting up a development environment, please refer to our [Development Deployment Guide](https://docs.docsgpt.cloud/Deploying/Development-Environment).
|
||||
|
||||
Tech Stack Overview:
|
||||
|
||||
@@ -36,15 +37,14 @@ Tech Stack Overview:
|
||||
|
||||
### 🌐 If you are looking to contribute to frontend (⚛️React, Vite):
|
||||
|
||||
- The current frontend is being migrated from [`/application`](https://github.com/arc53/DocsGPT/tree/main/application) to [`/frontend`](https://github.com/arc53/DocsGPT/tree/main/frontend) with a new design, so please contribute to the new one.
|
||||
- Check out this [milestone](https://github.com/arc53/DocsGPT/milestone/1) and its issues.
|
||||
|
||||
- The updated Figma design can be found [here](https://www.figma.com/file/OXLtrl1EAy885to6S69554/DocsGPT?node-id=0%3A1&t=hjWVuxRg9yi5YkJ9-1).
|
||||
|
||||
Please try to follow the guidelines.
|
||||
|
||||
### 🖥 If you are looking to contribute to Backend (🐍 Python):
|
||||
|
||||
- Review our issues and contribute to [`/application`](https://github.com/arc53/DocsGPT/tree/main/application) or [`/scripts`](https://github.com/arc53/DocsGPT/tree/main/scripts) (please disregard old [`ingest_rst.py`](https://github.com/arc53/DocsGPT/blob/main/scripts/old/ingest_rst.py) [`ingest_rst_sphinx.py`](https://github.com/arc53/DocsGPT/blob/main/scripts/old/ingest_rst_sphinx.py) files; these will be deprecated soon).
|
||||
- Review our issues and contribute to [`/application`](https://github.com/arc53/DocsGPT/tree/main/application)
|
||||
- All new code should be covered with unit tests ([pytest](https://github.com/pytest-dev/pytest)). Please find tests under [`/tests`](https://github.com/arc53/DocsGPT/tree/main/tests) folder.
|
||||
- Before submitting your Pull Request, ensure it can be queried after ingesting some test data.
|
||||
|
||||
|
||||
173
README.md
173
README.md
@@ -17,7 +17,13 @@
|
||||
<a href="https://github.com/arc53/DocsGPT/blob/main/LICENSE"></a>
|
||||
<a href="https://discord.gg/n5BX8dh8rU"></a>
|
||||
<a href="https://twitter.com/docsgptai"></a>
|
||||
|
||||
|
||||
<br>
|
||||
|
||||
[☁️ Cloud Version](https://app.docsgpt.cloud/) • [💬 Discord](https://discord.gg/n5BX8dh8rU) • [📖 Guides](https://docs.docsgpt.cloud/)
|
||||
<br>
|
||||
[👫 Contribute](https://github.com/arc53/DocsGPT/blob/main/CONTRIBUTING.md) • [🏠 Self-host](https://docs.docsgpt.cloud/Guides/How-to-use-different-LLM) • [⚡️ Quickstart](https://github.com/arc53/DocsGPT#quickstart)
|
||||
|
||||
</div>
|
||||
<div align="center">
|
||||
<img src="https://d3dg1063dc54p9.cloudfront.net/videos/demov4.gif" alt="video-example-of-docs-gpt" width="800" height="450">
|
||||
@@ -37,7 +43,16 @@
|
||||
|
||||
## Roadmap
|
||||
|
||||
You can find our roadmap [here](https://github.com/orgs/arc53/projects/2). Please don't hesitate to contribute or create issues, it helps us improve DocsGPT!
|
||||
- [x] Full GoogleAI compatibility (Jan 2025)
|
||||
- [x] Add tools (Jan 2025)
|
||||
- [ ] Anthropic Tool compatibility
|
||||
- [ ] Add triggerable actions / tools
|
||||
- [ ] Add oath2 authentication for tools and sources
|
||||
- [ ] Manually updating chunks in the app UI
|
||||
- [ ] Devcontainer for easy development
|
||||
- [ ] Chatbots menu re-design to handle tools, scheduling, and more
|
||||
|
||||
You can find our full roadmap [here](https://github.com/orgs/arc53/projects/2). Please don't hesitate to contribute or create issues, it helps us improve DocsGPT!
|
||||
|
||||
### Production Support / Help for Companies:
|
||||
|
||||
@@ -48,157 +63,77 @@ We're eager to provide personalized assistance when deploying your DocsGPT to a
|
||||
[Send Email :email:](mailto:support@docsgpt.cloud?subject=DocsGPT%20support%2Fsolutions)
|
||||
|
||||
|
||||
## Our Open-Source Models Optimized for DocsGPT:
|
||||
|
||||
| Name | Base Model | Requirements (or similar) |
|
||||
| --------------------------------------------------------------------- | ----------- | ------------------------- |
|
||||
| [Docsgpt-7b-mistral](https://huggingface.co/Arc53/docsgpt-7b-mistral) | Mistral-7b | 1xA10G gpu |
|
||||
| [Docsgpt-14b](https://huggingface.co/Arc53/docsgpt-14b) | llama-2-14b | 2xA10 gpu's |
|
||||
| [Docsgpt-40b-falcon](https://huggingface.co/Arc53/docsgpt-40b-falcon) | falcon-40b | 8xA10G gpu's |
|
||||
|
||||
If you don't have enough resources to run it, you can use bitsnbytes to quantize.
|
||||
|
||||
## End to End AI Framework for Information Retrieval
|
||||
|
||||
|
||||
|
||||
## Useful Links
|
||||
|
||||
- :mag: :fire: [Cloud Version](https://app.docsgpt.cloud/)
|
||||
|
||||
- :speech_balloon: :tada: [Join our Discord](https://discord.gg/n5BX8dh8rU)
|
||||
|
||||
- :books: :sunglasses: [Guides](https://docs.docsgpt.cloud/)
|
||||
|
||||
- :couple: [Interested in contributing?](https://github.com/arc53/DocsGPT/blob/main/CONTRIBUTING.md)
|
||||
|
||||
- :file_folder: :rocket: [How to use any other documentation](https://docs.docsgpt.cloud/Guides/How-to-train-on-other-documentation)
|
||||
|
||||
- :house: :closed_lock_with_key: [How to host it locally (so all data will stay on-premises)](https://docs.docsgpt.cloud/Guides/How-to-use-different-LLM)
|
||||
|
||||
## Project Structure
|
||||
|
||||
- Application - Flask app (main application).
|
||||
|
||||
- Extensions - Chrome extension.
|
||||
|
||||
- Scripts - Script that creates similarity search index for other libraries.
|
||||
|
||||
- Frontend - Frontend uses <a href="https://vitejs.dev/">Vite</a> and <a href="https://react.dev/">React</a>.
|
||||
|
||||
## QuickStart
|
||||
|
||||
> [!Note]
|
||||
> Make sure you have [Docker](https://docs.docker.com/engine/install/) installed
|
||||
|
||||
|
||||
1. Clone the repository and run the following command:
|
||||
```bash
|
||||
git clone https://github.com/arc53/DocsGPT.git
|
||||
cd DocsGPT
|
||||
```
|
||||
|
||||
On Mac OS or Linux, write:
|
||||
|
||||
`./setup.sh`
|
||||
|
||||
2. Run the following command:
|
||||
```bash
|
||||
./setup.sh
|
||||
```
|
||||
|
||||
It will install all the dependencies and allow you to download the local model, use OpenAI or use our LLM API.
|
||||
|
||||
Otherwise, refer to this Guide for Windows:
|
||||
|
||||
1. Download and open this repository with `git clone https://github.com/arc53/DocsGPT.git`
|
||||
2. Create a `.env` file in your root directory and set the env variables and `VITE_API_STREAMING` to true or false, depending on whether you want streaming answers or not.
|
||||
On windows:
|
||||
|
||||
2. Create a `.env` file in your root directory and set the env variables.
|
||||
It should look like this inside:
|
||||
|
||||
```
|
||||
LLM_NAME=[docsgpt or openai or others]
|
||||
VITE_API_STREAMING=true
|
||||
API_KEY=[if LLM_NAME is openai]
|
||||
```
|
||||
|
||||
See optional environment variables in the [/.env-template](https://github.com/arc53/DocsGPT/blob/main/.env-template) and [/application/.env_sample](https://github.com/arc53/DocsGPT/blob/main/application/.env_sample) files.
|
||||
See optional environment variables in the [/application/.env_sample](https://github.com/arc53/DocsGPT/blob/main/application/.env_sample) file.
|
||||
|
||||
3. Run [./run-with-docker-compose.sh](https://github.com/arc53/DocsGPT/blob/main/run-with-docker-compose.sh).
|
||||
3. Run the following command:
|
||||
|
||||
```bash
|
||||
docker-compose up
|
||||
```
|
||||
4. Navigate to http://localhost:5173/.
|
||||
|
||||
To stop, just run `Ctrl + C`.
|
||||
|
||||
## Development Environments
|
||||
|
||||
### Spin up Mongo and Redis
|
||||
|
||||
For development, only two containers are used from [docker-compose.yaml](https://github.com/arc53/DocsGPT/blob/main/docker-compose.yaml) (by deleting all services except for Redis and Mongo).
|
||||
See file [docker-compose-dev.yaml](./docker-compose-dev.yaml).
|
||||
|
||||
Run
|
||||
|
||||
```
|
||||
docker compose -f docker-compose-dev.yaml build
|
||||
docker compose -f docker-compose-dev.yaml up -d
|
||||
```
|
||||
|
||||
### Run the Backend
|
||||
|
||||
> [!Note]
|
||||
> Make sure you have Python 3.12 installed.
|
||||
|
||||
1. Export required environment variables or prepare a `.env` file in the project folder:
|
||||
- Copy [.env-template](https://github.com/arc53/DocsGPT/blob/main/application/.env-template) and create `.env`.
|
||||
|
||||
(check out [`application/core/settings.py`](application/core/settings.py) if you want to see more config options.)
|
||||
|
||||
2. (optional) Create a Python virtual environment:
|
||||
You can follow the [Python official documentation](https://docs.python.org/3/tutorial/venv.html) for virtual environments.
|
||||
|
||||
a) On Mac OS and Linux
|
||||
|
||||
```commandline
|
||||
python -m venv venv
|
||||
. venv/bin/activate
|
||||
```
|
||||
|
||||
b) On Windows
|
||||
|
||||
```commandline
|
||||
python -m venv venv
|
||||
venv/Scripts/activate
|
||||
```
|
||||
|
||||
3. Download embedding model and save it in the `model/` folder:
|
||||
You can use the script below, or download it manually from [here](https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip), unzip it and save it in the `model/` folder.
|
||||
|
||||
```commandline
|
||||
wget https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip
|
||||
unzip mpnet-base-v2.zip -d model
|
||||
rm mpnet-base-v2.zip
|
||||
```
|
||||
|
||||
4. Install dependencies for the backend:
|
||||
|
||||
```commandline
|
||||
pip install -r application/requirements.txt
|
||||
```
|
||||
|
||||
5. Run the app using `flask --app application/app.py run --host=0.0.0.0 --port=7091`.
|
||||
6. Start worker with `celery -A application.app.celery worker -l INFO`.
|
||||
|
||||
### Start Frontend
|
||||
|
||||
> [!Note]
|
||||
> Make sure you have Node version 16 or higher.
|
||||
|
||||
1. Navigate to the [/frontend](https://github.com/arc53/DocsGPT/tree/main/frontend) folder.
|
||||
2. Install the required packages `husky` and `vite` (ignore if already installed).
|
||||
|
||||
```commandline
|
||||
npm install husky -g
|
||||
npm install vite -g
|
||||
```
|
||||
|
||||
3. Install dependencies by running `npm install --include=dev`.
|
||||
4. Run the app using `npm run dev`.
|
||||
> For development environment setup instructions, please refer to the [Development Environment Guide](https://docs.docsgpt.cloud/Deploying/Development-Environment).
|
||||
|
||||
## Contributing
|
||||
|
||||
Please refer to the [CONTRIBUTING.md](CONTRIBUTING.md) file for information about how to get involved. We welcome issues, questions, and pull requests.
|
||||
|
||||
## Architecture
|
||||
|
||||
|
||||
|
||||
## Project Structure
|
||||
|
||||
- Application - Flask app (main application).
|
||||
|
||||
- Extensions - Extensions, like react widget or discord bot.
|
||||
|
||||
- Frontend - Frontend uses <a href="https://vitejs.dev/">Vite</a> and <a href="https://react.dev/">React</a>.
|
||||
|
||||
- Scripts - Miscellaneous scripts.
|
||||
|
||||
## Code Of Conduct
|
||||
|
||||
We as members, contributors, and leaders, pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation. Please refer to the [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) file for more information about contributing.
|
||||
|
||||
|
||||
## Many Thanks To Our Contributors⚡
|
||||
|
||||
<a href="https://github.com/arc53/DocsGPT/graphs/contributors" alt="View Contributors">
|
||||
|
||||
78
docs/pages/Deploying/Development-Environment.md
Normal file
78
docs/pages/Deploying/Development-Environment.md
Normal file
@@ -0,0 +1,78 @@
|
||||
## Development Environments
|
||||
|
||||
### Spin up Mongo and Redis
|
||||
|
||||
For development, only two containers are used from [docker-compose.yaml](https://github.com/arc53/DocsGPT/blob/main/docker-compose.yaml) (by deleting all services except for Redis and Mongo).
|
||||
See file [docker-compose-dev.yaml](https://github.com/arc53/DocsGPT/blob/main/docker-compose-dev.yaml).
|
||||
|
||||
Run
|
||||
|
||||
```
|
||||
docker compose -f docker-compose-dev.yaml build
|
||||
docker compose -f docker-compose-dev.yaml up -d
|
||||
```
|
||||
|
||||
### Run the Backend
|
||||
|
||||
> [!Note]
|
||||
> Make sure you have Python 3.12 installed.
|
||||
|
||||
1. Export required environment variables or prepare a `.env` file in the project folder:
|
||||
- Copy [.env-template](https://github.com/arc53/DocsGPT/blob/main/application/.env-template) and create `.env`.
|
||||
|
||||
(check out [`application/core/settings.py`](application/core/settings.py) if you want to see more config options.)
|
||||
|
||||
2. (optional) Create a Python virtual environment:
|
||||
You can follow the [Python official documentation](https://docs.python.org/3/tutorial/venv.html) for virtual environments.
|
||||
|
||||
a) On Mac OS and Linux
|
||||
|
||||
```commandline
|
||||
python -m venv venv
|
||||
. venv/bin/activate
|
||||
```
|
||||
|
||||
b) On Windows
|
||||
|
||||
```commandline
|
||||
python -m venv venv
|
||||
venv/Scripts/activate
|
||||
```
|
||||
|
||||
3. Download embedding model and save it in the `model/` folder:
|
||||
You can use the script below, or download it manually from [here](https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip), unzip it and save it in the `model/` folder.
|
||||
|
||||
```commandline
|
||||
wget https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip
|
||||
unzip mpnet-base-v2.zip -d model
|
||||
rm mpnet-base-v2.zip
|
||||
```
|
||||
|
||||
4. Install dependencies for the backend:
|
||||
|
||||
```commandline
|
||||
pip install -r application/requirements.txt
|
||||
```
|
||||
|
||||
5. Run the app using `flask --app application/app.py run --host=0.0.0.0 --port=7091`.
|
||||
6. Start worker with `celery -A application.app.celery worker -l INFO`.
|
||||
|
||||
> [!Note]
|
||||
> You can also launch the in a debugger mode in vscode by accessing SHIFT + CMD + D or SHIFT + Windows + D on windows and selecting Flask or Celery.
|
||||
|
||||
|
||||
### Start Frontend
|
||||
|
||||
> [!Note]
|
||||
> Make sure you have Node version 16 or higher.
|
||||
|
||||
1. Navigate to the [/frontend](https://github.com/arc53/DocsGPT/tree/main/frontend) folder.
|
||||
2. Install the required packages `husky` and `vite` (ignore if already installed).
|
||||
|
||||
```commandline
|
||||
npm install husky -g
|
||||
npm install vite -g
|
||||
```
|
||||
|
||||
3. Install dependencies by running `npm install --include=dev`.
|
||||
4. Run the app using `npm run dev`.
|
||||
@@ -15,11 +15,21 @@ If you prefer to follow manual steps, refer to this guide:
|
||||
1. Open and download this repository with
|
||||
```bash
|
||||
git clone https://github.com/arc53/DocsGPT.git
|
||||
cd DocsGPT
|
||||
```
|
||||
2. Create a `.env` file in your root directory and set your `API_KEY` with your [OpenAI API key](https://platform.openai.com/account/api-keys). (optional in case you want to use OpenAI)
|
||||
2. Create a `.env` file in your root directory and set the env variables.
|
||||
It should look like this inside:
|
||||
|
||||
```
|
||||
LLM_NAME=[docsgpt or openai or others]
|
||||
API_KEY=[if LLM_NAME is openai]
|
||||
```
|
||||
|
||||
See optional environment variables in the [/application/.env_sample](https://github.com/arc53/DocsGPT/blob/main/application/.env_sample) file.
|
||||
|
||||
3. Run the following commands:
|
||||
```bash
|
||||
docker-compose build && docker-compose up
|
||||
docker compose up
|
||||
```
|
||||
4. Navigate to http://localhost:5173/.
|
||||
|
||||
@@ -27,43 +37,28 @@ To stop, simply press **Ctrl + C**.
|
||||
|
||||
**For WINDOWS:**
|
||||
|
||||
To run the setup on Windows, you have two options: using the Windows Subsystem for Linux (WSL) or using Git Bash or Command Prompt.
|
||||
|
||||
**Option 1: Using Windows Subsystem for Linux (WSL):**
|
||||
|
||||
1. Install WSL if you haven't already. You can follow the official Microsoft documentation for installation: (https://learn.microsoft.com/en-us/windows/wsl/install).
|
||||
2. After setting up WSL, open the WSL terminal.
|
||||
3. Clone the repository and create the `.env` file:
|
||||
1. Open and download this repository with
|
||||
```bash
|
||||
git clone https://github.com/arc53/DocsGPT.git
|
||||
cd DocsGPT
|
||||
echo "API_KEY=Yourkey" > .env
|
||||
echo "VITE_API_STREAMING=true" >> .env
|
||||
```
|
||||
4. Run the following command to start the setup with Docker Compose:
|
||||
```bash
|
||||
./run-with-docker-compose.sh
|
||||
```
|
||||
6. Open your web browser and navigate to http://localhost:5173/.
|
||||
7. To stop the setup, just press **Ctrl + C** in the WSL terminal
|
||||
|
||||
**Option 2: Using Git Bash or Command Prompt (CMD):**
|
||||
2. Create a `.env` file in your root directory and set the env variables.
|
||||
It should look like this inside:
|
||||
|
||||
1. Install Git for Windows if you haven't already. Download it from the official website: (https://gitforwindows.org/).
|
||||
2. Open Git Bash or Command Prompt.
|
||||
3. Clone the repository and create the `.env` file:
|
||||
```bash
|
||||
git clone https://github.com/arc53/DocsGPT.git
|
||||
cd DocsGPT
|
||||
echo "API_KEY=Yourkey" > .env
|
||||
echo "VITE_API_STREAMING=true" >> .env
|
||||
```
|
||||
4. Run the following command to start the setup with Docker Compose:
|
||||
```bash
|
||||
./run-with-docker-compose.sh
|
||||
LLM_NAME=[docsgpt or openai or others]
|
||||
API_KEY=[if LLM_NAME is openai]
|
||||
```
|
||||
5. Open your web browser and navigate to http://localhost:5173/.
|
||||
6. To stop the setup, just press **Ctrl + C** in the Git Bash or Command Prompt terminal.
|
||||
|
||||
These steps should help you set up and run the project on Windows using either WSL or Git Bash/Command Prompt.
|
||||
See optional environment variables in the [/application/.env_sample](https://github.com/arc53/DocsGPT/blob/main/application/.env_sample) file.
|
||||
|
||||
3. Run the following command:
|
||||
|
||||
```bash
|
||||
docker-compose up
|
||||
```
|
||||
4. Navigate to http://localhost:5173/.
|
||||
5. To stop the setup, just press **Ctrl + C** in the WSL terminal
|
||||
|
||||
**Important:** Ensure that Docker is installed and properly configured on your Windows system for these steps to work.
|
||||
|
||||
@@ -7,6 +7,10 @@
|
||||
"title": "⚡️Quickstart",
|
||||
"href": "/Deploying/Quickstart"
|
||||
},
|
||||
"Development-Environment": {
|
||||
"title": "🛠️Development Environment",
|
||||
"href": "/Deploying/Development-Environment"
|
||||
},
|
||||
"Railway-Deploying": {
|
||||
"title": "🚂Deploying on Railway",
|
||||
"href": "/Deploying/Railway-Deploying"
|
||||
|
||||
Reference in New Issue
Block a user