mirror of
https://github.com/arc53/DocsGPT.git
synced 2025-11-30 17:13:15 +00:00
Merge branch 'main' of https://github.com/ManishMadan2882/docsgpt
This commit is contained in:
ManishMadan2882
@@ -27,6 +27,7 @@ Before creating issues, please check out how the latest version of our app looks
|
||||
|
||||
### 👨💻 If you're interested in contributing code, here are some important things to know:
|
||||
|
||||
For instructions on setting up a development environment, please refer to our [Development Deployment Guide](https://docs.docsgpt.cloud/Deploying/Development-Environment).
|
||||
|
||||
Tech Stack Overview:
|
||||
|
||||
@@ -36,15 +37,14 @@ Tech Stack Overview:
|
||||
|
||||
### 🌐 If you are looking to contribute to frontend (⚛️React, Vite):
|
||||
|
||||
- The current frontend is being migrated from [`/application`](https://github.com/arc53/DocsGPT/tree/main/application) to [`/frontend`](https://github.com/arc53/DocsGPT/tree/main/frontend) with a new design, so please contribute to the new one.
|
||||
- Check out this [milestone](https://github.com/arc53/DocsGPT/milestone/1) and its issues.
|
||||
|
||||
- The updated Figma design can be found [here](https://www.figma.com/file/OXLtrl1EAy885to6S69554/DocsGPT?node-id=0%3A1&t=hjWVuxRg9yi5YkJ9-1).
|
||||
|
||||
Please try to follow the guidelines.
|
||||
|
||||
### 🖥 If you are looking to contribute to Backend (🐍 Python):
|
||||
|
||||
- Review our issues and contribute to [`/application`](https://github.com/arc53/DocsGPT/tree/main/application) or [`/scripts`](https://github.com/arc53/DocsGPT/tree/main/scripts) (please disregard old [`ingest_rst.py`](https://github.com/arc53/DocsGPT/blob/main/scripts/old/ingest_rst.py) [`ingest_rst_sphinx.py`](https://github.com/arc53/DocsGPT/blob/main/scripts/old/ingest_rst_sphinx.py) files; these will be deprecated soon).
|
||||
- Review our issues and contribute to [`/application`](https://github.com/arc53/DocsGPT/tree/main/application)
|
||||
- All new code should be covered with unit tests ([pytest](https://github.com/pytest-dev/pytest)). Please find tests under [`/tests`](https://github.com/arc53/DocsGPT/tree/main/tests) folder.
|
||||
- Before submitting your Pull Request, ensure it can be queried after ingesting some test data.
|
||||
|
||||
|
||||
205
README.md
205
README.md
@@ -3,13 +3,11 @@
|
||||
</h1>
|
||||
|
||||
<p align="center">
|
||||
<strong>Open-Source Documentation Assistant</strong>
|
||||
<strong>Open-Source RAG Assistant</strong>
|
||||
</p>
|
||||
|
||||
<p align="left">
|
||||
<strong><a href="https://www.docsgpt.cloud/">DocsGPT</a></strong> is a cutting-edge open-source solution that streamlines the process of finding information in the project documentation. With its integration of the powerful <strong>GPT</strong> models, developers can easily ask questions about a project and receive accurate answers.
|
||||
|
||||
Say goodbye to time-consuming manual searches, and let <strong><a href="https://www.docsgpt.cloud/">DocsGPT</a></strong> help you quickly find the information you need. Try it out and see how it revolutionizes your project documentation experience. Contribute to its development and be a part of the future of AI-powered assistance.
|
||||
<strong><a href="https://www.docsgpt.cloud/">DocsGPT</a></strong> is an open-source genAI tool that helps users get reliable answers from any knowledge source, while avoiding hallucinations. It enables quick and reliable information retrieval, with tooling and agentic system capability built in.
|
||||
</p>
|
||||
|
||||
<div align="center">
|
||||
@@ -20,175 +18,122 @@ Say goodbye to time-consuming manual searches, and let <strong><a href="https://
|
||||
<a href="https://discord.gg/n5BX8dh8rU"></a>
|
||||
<a href="https://twitter.com/docsgptai"></a>
|
||||
|
||||
|
||||
<br>
|
||||
|
||||
[☁️ Cloud Version](https://app.docsgpt.cloud/) • [💬 Discord](https://discord.gg/n5BX8dh8rU) • [📖 Guides](https://docs.docsgpt.cloud/)
|
||||
<br>
|
||||
[👫 Contribute](https://github.com/arc53/DocsGPT/blob/main/CONTRIBUTING.md) • [🏠 Self-host](https://docs.docsgpt.cloud/Guides/How-to-use-different-LLM) • [⚡️ Quickstart](https://github.com/arc53/DocsGPT#quickstart)
|
||||
|
||||
</div>
|
||||
<div align="center">
|
||||
<img src="https://d3dg1063dc54p9.cloudfront.net/videos/demov4.gif" alt="video-example-of-docs-gpt" width="800" height="450">
|
||||
</div>
|
||||
<h3 align="left">
|
||||
<strong>Key Features:</strong>
|
||||
</h3>
|
||||
<ul align="left">
|
||||
<li><strong>🗂️ Wide Format Support:</strong> Reads PDF, DOCX, CSV, XLSX, EPUB, MD, RST, HTML, MDX, JSON, PPTX, and images.</li>
|
||||
<li><strong>🌐 Web & Data Integration:</strong> Ingests from URLs, sitemaps, Reddit, GitHub and web crawlers.</li>
|
||||
<li><strong>✅ Reliable Answers:</strong> Get accurate, hallucination-free responses with source citations viewable in a clean UI.</li>
|
||||
<li><strong>🔗 Actionable Tooling:</strong> Connect to APIs, tools, and other services to enable LLM actions.</li>
|
||||
<li><strong>🧩 Pre-built Integrations:</strong> Use readily available HTML/React chat widgets, search tools, Discord/Telegram bots, and more.</li>
|
||||
<li><strong>🔌 Flexible Deployment:</strong> Works with major LLMs (OpenAI, Google, Anthropic) and local models (Ollama, llama_cpp).</li>
|
||||
<li><strong>🏢 Secure & Scalable:</strong> Run privately and securely with Kubernetes support, designed for enterprise-grade reliability.</li>
|
||||
</ul>
|
||||
|
||||
## Roadmap
|
||||
|
||||
- [x] Full GoogleAI compatibility (Jan 2025)
|
||||
- [x] Add tools (Jan 2025)
|
||||
- [ ] Anthropic Tool compatibility
|
||||
- [ ] Add triggerable actions / tools (webhook)
|
||||
- [ ] Add OAuth 2.0 authentication for tools and sources
|
||||
- [ ] Manually updating chunks in the app UI
|
||||
- [ ] Devcontainer for easy development
|
||||
- [ ] Chatbots menu re-design to handle tools, scheduling, and more
|
||||
|
||||
You can find our full roadmap [here](https://github.com/orgs/arc53/projects/2). Please don't hesitate to contribute or create issues, it helps us improve DocsGPT!
|
||||
|
||||
### Production Support / Help for Companies:
|
||||
|
||||
We're eager to provide personalized assistance when deploying your DocsGPT to a live environment.
|
||||
|
||||
[Book a Meeting :wave:](https://cal.com/arc53/docsgpt-demo-b2b)
|
||||
[Get a Demo :wave:](https://www.docsgpt.cloud/contact)
|
||||
|
||||
[Send Email :email:](mailto:contact@arc53.com?subject=DocsGPT%20support%2Fsolutions)
|
||||
[Send Email :email:](mailto:support@docsgpt.cloud?subject=DocsGPT%20support%2Fsolutions)
|
||||
|
||||
|
||||
<img src="https://github.com/user-attachments/assets/9a1f21de-7a15-4e42-9424-70d22ba5a913" alt="video-example-of-docs-gpt" width="1000" height="500">
|
||||
|
||||
## Roadmap
|
||||
|
||||
You can find our roadmap [here](https://github.com/orgs/arc53/projects/2). Please don't hesitate to contribute or create issues, it helps us improve DocsGPT!
|
||||
|
||||
## Our Open-Source Models Optimized for DocsGPT:
|
||||
|
||||
| Name | Base Model | Requirements (or similar) |
|
||||
| --------------------------------------------------------------------- | ----------- | ------------------------- |
|
||||
| [Docsgpt-7b-mistral](https://huggingface.co/Arc53/docsgpt-7b-mistral) | Mistral-7b | 1xA10G gpu |
|
||||
| [Docsgpt-14b](https://huggingface.co/Arc53/docsgpt-14b) | llama-2-14b | 2xA10 gpu's |
|
||||
| [Docsgpt-40b-falcon](https://huggingface.co/Arc53/docsgpt-40b-falcon) | falcon-40b | 8xA10G gpu's |
|
||||
|
||||
If you don't have enough resources to run it, you can use bitsnbytes to quantize.
|
||||
|
||||
## End to End AI Framework for Information Retrieval
|
||||
|
||||
|
||||
|
||||
## Useful Links
|
||||
|
||||
- :mag: :fire: [Cloud Version](https://app.docsgpt.cloud/)
|
||||
|
||||
- :speech_balloon: :tada: [Join our Discord](https://discord.gg/n5BX8dh8rU)
|
||||
|
||||
- :books: :sunglasses: [Guides](https://docs.docsgpt.cloud/)
|
||||
|
||||
- :couple: [Interested in contributing?](https://github.com/arc53/DocsGPT/blob/main/CONTRIBUTING.md)
|
||||
|
||||
- :file_folder: :rocket: [How to use any other documentation](https://docs.docsgpt.cloud/Guides/How-to-train-on-other-documentation)
|
||||
|
||||
- :house: :closed_lock_with_key: [How to host it locally (so all data will stay on-premises)](https://docs.docsgpt.cloud/Guides/How-to-use-different-LLM)
|
||||
|
||||
## Project Structure
|
||||
|
||||
- Application - Flask app (main application).
|
||||
|
||||
- Extensions - Chrome extension.
|
||||
|
||||
- Scripts - Script that creates similarity search index for other libraries.
|
||||
|
||||
- Frontend - Frontend uses <a href="https://vitejs.dev/">Vite</a> and <a href="https://react.dev/">React</a>.
|
||||
|
||||
## QuickStart
|
||||
|
||||
> [!Note]
|
||||
> Make sure you have [Docker](https://docs.docker.com/engine/install/) installed
|
||||
|
||||
|
||||
1. Clone the repository and run the following command:
|
||||
```bash
|
||||
git clone https://github.com/arc53/DocsGPT.git
|
||||
cd DocsGPT
|
||||
```
|
||||
|
||||
On Mac OS or Linux, write:
|
||||
|
||||
`./setup.sh`
|
||||
|
||||
2. Run the following command:
|
||||
```bash
|
||||
./setup.sh
|
||||
```
|
||||
|
||||
It will install all the dependencies and allow you to download the local model, use OpenAI or use our LLM API.
|
||||
|
||||
Otherwise, refer to this Guide for Windows:
|
||||
|
||||
1. Download and open this repository with `git clone https://github.com/arc53/DocsGPT.git`
|
||||
2. Create a `.env` file in your root directory and set the env variables and `VITE_API_STREAMING` to true or false, depending on whether you want streaming answers or not.
|
||||
On windows:
|
||||
|
||||
2. Create a `.env` file in your root directory and set the env variables.
|
||||
It should look like this inside:
|
||||
|
||||
```
|
||||
LLM_NAME=[docsgpt or openai or others]
|
||||
VITE_API_STREAMING=true
|
||||
API_KEY=[if LLM_NAME is openai]
|
||||
```
|
||||
|
||||
See optional environment variables in the [/.env-template](https://github.com/arc53/DocsGPT/blob/main/.env-template) and [/application/.env_sample](https://github.com/arc53/DocsGPT/blob/main/application/.env_sample) files.
|
||||
See optional environment variables in the [/application/.env_sample](https://github.com/arc53/DocsGPT/blob/main/application/.env_sample) file.
|
||||
|
||||
3. Run [./run-with-docker-compose.sh](https://github.com/arc53/DocsGPT/blob/main/run-with-docker-compose.sh).
|
||||
3. Run the following command:
|
||||
|
||||
```bash
|
||||
docker-compose up
|
||||
```
|
||||
4. Navigate to http://localhost:5173/.
|
||||
|
||||
To stop, just run `Ctrl + C`.
|
||||
|
||||
## Development Environments
|
||||
|
||||
### Spin up Mongo and Redis
|
||||
|
||||
For development, only two containers are used from [docker-compose.yaml](https://github.com/arc53/DocsGPT/blob/main/docker-compose.yaml) (by deleting all services except for Redis and Mongo).
|
||||
See file [docker-compose-dev.yaml](./docker-compose-dev.yaml).
|
||||
|
||||
Run
|
||||
|
||||
```
|
||||
docker compose -f docker-compose-dev.yaml build
|
||||
docker compose -f docker-compose-dev.yaml up -d
|
||||
```
|
||||
|
||||
### Run the Backend
|
||||
|
||||
> [!Note]
|
||||
> Make sure you have Python 3.12 installed.
|
||||
|
||||
1. Export required environment variables or prepare a `.env` file in the project folder:
|
||||
- Copy [.env-template](https://github.com/arc53/DocsGPT/blob/main/application/.env-template) and create `.env`.
|
||||
|
||||
(check out [`application/core/settings.py`](application/core/settings.py) if you want to see more config options.)
|
||||
|
||||
2. (optional) Create a Python virtual environment:
|
||||
You can follow the [Python official documentation](https://docs.python.org/3/tutorial/venv.html) for virtual environments.
|
||||
|
||||
a) On Mac OS and Linux
|
||||
|
||||
```commandline
|
||||
python -m venv venv
|
||||
. venv/bin/activate
|
||||
```
|
||||
|
||||
b) On Windows
|
||||
|
||||
```commandline
|
||||
python -m venv venv
|
||||
venv/Scripts/activate
|
||||
```
|
||||
|
||||
3. Download embedding model and save it in the `model/` folder:
|
||||
You can use the script below, or download it manually from [here](https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip), unzip it and save it in the `model/` folder.
|
||||
|
||||
```commandline
|
||||
wget https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip
|
||||
unzip mpnet-base-v2.zip -d model
|
||||
rm mpnet-base-v2.zip
|
||||
```
|
||||
|
||||
4. Install dependencies for the backend:
|
||||
|
||||
```commandline
|
||||
pip install -r application/requirements.txt
|
||||
```
|
||||
|
||||
5. Run the app using `flask --app application/app.py run --host=0.0.0.0 --port=7091`.
|
||||
6. Start worker with `celery -A application.app.celery worker -l INFO`.
|
||||
|
||||
### Start Frontend
|
||||
|
||||
> [!Note]
|
||||
> Make sure you have Node version 16 or higher.
|
||||
|
||||
1. Navigate to the [/frontend](https://github.com/arc53/DocsGPT/tree/main/frontend) folder.
|
||||
2. Install the required packages `husky` and `vite` (ignore if already installed).
|
||||
|
||||
```commandline
|
||||
npm install husky -g
|
||||
npm install vite -g
|
||||
```
|
||||
|
||||
3. Install dependencies by running `npm install --include=dev`.
|
||||
4. Run the app using `npm run dev`.
|
||||
> For development environment setup instructions, please refer to the [Development Environment Guide](https://docs.docsgpt.cloud/Deploying/Development-Environment).
|
||||
|
||||
## Contributing
|
||||
|
||||
Please refer to the [CONTRIBUTING.md](CONTRIBUTING.md) file for information about how to get involved. We welcome issues, questions, and pull requests.
|
||||
|
||||
## Architecture
|
||||
|
||||
|
||||
|
||||
## Project Structure
|
||||
|
||||
- Application - Flask app (main application).
|
||||
|
||||
- Extensions - Extensions, like react widget or discord bot.
|
||||
|
||||
- Frontend - Frontend uses <a href="https://vitejs.dev/">Vite</a> and <a href="https://react.dev/">React</a>.
|
||||
|
||||
- Scripts - Miscellaneous scripts.
|
||||
|
||||
## Code Of Conduct
|
||||
|
||||
We as members, contributors, and leaders, pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation. Please refer to the [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) file for more information about contributing.
|
||||
|
||||
|
||||
## Many Thanks To Our Contributors⚡
|
||||
|
||||
<a href="https://github.com/arc53/DocsGPT/graphs/contributors" alt="View Contributors">
|
||||
|
||||
@@ -60,6 +60,7 @@ prance==23.6.21.0
|
||||
primp==0.10.0
|
||||
prompt-toolkit==3.0.48
|
||||
protobuf==5.29.3
|
||||
psycopg2-binary==2.9.10
|
||||
py==1.11.0
|
||||
pydantic==2.10.4
|
||||
pydantic-core==2.27.2
|
||||
|
||||
163
application/tools/implementations/postgres.py
Normal file
163
application/tools/implementations/postgres.py
Normal file
@@ -0,0 +1,163 @@
|
||||
import psycopg2
|
||||
from application.tools.base import Tool
|
||||
|
||||
class PostgresTool(Tool):
|
||||
"""
|
||||
PostgreSQL Database Tool
|
||||
A tool for connecting to a PostgreSQL database using a connection string,
|
||||
executing SQL queries, and retrieving schema information.
|
||||
"""
|
||||
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
self.connection_string = config.get("token", "")
|
||||
|
||||
def execute_action(self, action_name, **kwargs):
|
||||
actions = {
|
||||
"postgres_execute_sql": self._execute_sql,
|
||||
"postgres_get_schema": self._get_schema,
|
||||
}
|
||||
|
||||
if action_name in actions:
|
||||
return actions[action_name](**kwargs)
|
||||
else:
|
||||
raise ValueError(f"Unknown action: {action_name}")
|
||||
|
||||
def _execute_sql(self, sql_query):
|
||||
"""
|
||||
Executes an SQL query against the PostgreSQL database using a connection string.
|
||||
"""
|
||||
conn = None # Initialize conn to None for error handling
|
||||
try:
|
||||
conn = psycopg2.connect(self.connection_string)
|
||||
cur = conn.cursor()
|
||||
cur.execute(sql_query)
|
||||
conn.commit()
|
||||
|
||||
if sql_query.strip().lower().startswith("select"):
|
||||
column_names = [desc[0] for desc in cur.description] if cur.description else []
|
||||
results = []
|
||||
rows = cur.fetchall()
|
||||
for row in rows:
|
||||
results.append(dict(zip(column_names, row)))
|
||||
response_data = {"data": results, "column_names": column_names}
|
||||
else:
|
||||
row_count = cur.rowcount
|
||||
response_data = {"message": f"Query executed successfully, {row_count} rows affected."}
|
||||
|
||||
cur.close()
|
||||
return {
|
||||
"status_code": 200,
|
||||
"message": "SQL query executed successfully.",
|
||||
"response_data": response_data,
|
||||
}
|
||||
|
||||
except psycopg2.Error as e:
|
||||
error_message = f"Database error: {e}"
|
||||
print(f"Database error: {e}")
|
||||
return {
|
||||
"status_code": 500,
|
||||
"message": "Failed to execute SQL query.",
|
||||
"error": error_message,
|
||||
}
|
||||
finally:
|
||||
if conn: # Ensure connection is closed even if errors occur
|
||||
conn.close()
|
||||
|
||||
def _get_schema(self, db_name):
|
||||
"""
|
||||
Retrieves the schema of the PostgreSQL database using a connection string.
|
||||
"""
|
||||
conn = None # Initialize conn to None for error handling
|
||||
try:
|
||||
conn = psycopg2.connect(self.connection_string)
|
||||
cur = conn.cursor()
|
||||
|
||||
cur.execute("""
|
||||
SELECT
|
||||
table_name,
|
||||
column_name,
|
||||
data_type,
|
||||
column_default,
|
||||
is_nullable
|
||||
FROM
|
||||
information_schema.columns
|
||||
WHERE
|
||||
table_schema = 'public'
|
||||
ORDER BY
|
||||
table_name,
|
||||
ordinal_position;
|
||||
""")
|
||||
|
||||
schema_data = {}
|
||||
for row in cur.fetchall():
|
||||
table_name, column_name, data_type, column_default, is_nullable = row
|
||||
if table_name not in schema_data:
|
||||
schema_data[table_name] = []
|
||||
schema_data[table_name].append({
|
||||
"column_name": column_name,
|
||||
"data_type": data_type,
|
||||
"column_default": column_default,
|
||||
"is_nullable": is_nullable
|
||||
})
|
||||
|
||||
cur.close()
|
||||
return {
|
||||
"status_code": 200,
|
||||
"message": "Database schema retrieved successfully.",
|
||||
"schema": schema_data,
|
||||
}
|
||||
|
||||
except psycopg2.Error as e:
|
||||
error_message = f"Database error: {e}"
|
||||
print(f"Database error: {e}")
|
||||
return {
|
||||
"status_code": 500,
|
||||
"message": "Failed to retrieve database schema.",
|
||||
"error": error_message,
|
||||
}
|
||||
finally:
|
||||
if conn: # Ensure connection is closed even if errors occur
|
||||
conn.close()
|
||||
|
||||
def get_actions_metadata(self):
|
||||
return [
|
||||
{
|
||||
"name": "postgres_execute_sql",
|
||||
"description": "Execute an SQL query against the PostgreSQL database and return the results. Use this tool to interact with the database, e.g., retrieve specific data or perform updates. Only SELECT queries will return data, other queries will return execution status.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"sql_query": {
|
||||
"type": "string",
|
||||
"description": "The SQL query to execute.",
|
||||
},
|
||||
},
|
||||
"required": ["sql_query"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "postgres_get_schema",
|
||||
"description": "Retrieve the schema of the PostgreSQL database, including tables and their columns. Use this to understand the database structure before executing queries. db_name is 'default' if not provided.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"db_name": {
|
||||
"type": "string",
|
||||
"description": "The name of the database to retrieve the schema for.",
|
||||
},
|
||||
},
|
||||
"required": ["db_name"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
},
|
||||
]
|
||||
|
||||
def get_config_requirements(self):
|
||||
return {
|
||||
"token": {
|
||||
"type": "string",
|
||||
"description": "PostgreSQL database connection string (e.g., 'postgresql://user:password@host:port/dbname')",
|
||||
},
|
||||
}
|
||||
78
docs/pages/Deploying/Development-Environment.md
Normal file
78
docs/pages/Deploying/Development-Environment.md
Normal file
@@ -0,0 +1,78 @@
|
||||
## Development Environments
|
||||
|
||||
### Spin up Mongo and Redis
|
||||
|
||||
For development, only two containers are used from [docker-compose.yaml](https://github.com/arc53/DocsGPT/blob/main/docker-compose.yaml) (by deleting all services except for Redis and Mongo).
|
||||
See file [docker-compose-dev.yaml](https://github.com/arc53/DocsGPT/blob/main/docker-compose-dev.yaml).
|
||||
|
||||
Run
|
||||
|
||||
```
|
||||
docker compose -f docker-compose-dev.yaml build
|
||||
docker compose -f docker-compose-dev.yaml up -d
|
||||
```
|
||||
|
||||
### Run the Backend
|
||||
|
||||
> [!Note]
|
||||
> Make sure you have Python 3.12 installed.
|
||||
|
||||
1. Export required environment variables or prepare a `.env` file in the project folder:
|
||||
- Copy [.env-template](https://github.com/arc53/DocsGPT/blob/main/application/.env-template) and create `.env`.
|
||||
|
||||
(check out [`application/core/settings.py`](application/core/settings.py) if you want to see more config options.)
|
||||
|
||||
2. (optional) Create a Python virtual environment:
|
||||
You can follow the [Python official documentation](https://docs.python.org/3/tutorial/venv.html) for virtual environments.
|
||||
|
||||
a) On Mac OS and Linux
|
||||
|
||||
```commandline
|
||||
python -m venv venv
|
||||
. venv/bin/activate
|
||||
```
|
||||
|
||||
b) On Windows
|
||||
|
||||
```commandline
|
||||
python -m venv venv
|
||||
venv/Scripts/activate
|
||||
```
|
||||
|
||||
3. Download embedding model and save it in the `model/` folder:
|
||||
You can use the script below, or download it manually from [here](https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip), unzip it and save it in the `model/` folder.
|
||||
|
||||
```commandline
|
||||
wget https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip
|
||||
unzip mpnet-base-v2.zip -d model
|
||||
rm mpnet-base-v2.zip
|
||||
```
|
||||
|
||||
4. Install dependencies for the backend:
|
||||
|
||||
```commandline
|
||||
pip install -r application/requirements.txt
|
||||
```
|
||||
|
||||
5. Run the app using `flask --app application/app.py run --host=0.0.0.0 --port=7091`.
|
||||
6. Start worker with `celery -A application.app.celery worker -l INFO`.
|
||||
|
||||
> [!Note]
|
||||
> You can also launch the in a debugger mode in vscode by accessing SHIFT + CMD + D or SHIFT + Windows + D on windows and selecting Flask or Celery.
|
||||
|
||||
|
||||
### Start Frontend
|
||||
|
||||
> [!Note]
|
||||
> Make sure you have Node version 16 or higher.
|
||||
|
||||
1. Navigate to the [/frontend](https://github.com/arc53/DocsGPT/tree/main/frontend) folder.
|
||||
2. Install the required packages `husky` and `vite` (ignore if already installed).
|
||||
|
||||
```commandline
|
||||
npm install husky -g
|
||||
npm install vite -g
|
||||
```
|
||||
|
||||
3. Install dependencies by running `npm install --include=dev`.
|
||||
4. Run the app using `npm run dev`.
|
||||
@@ -15,11 +15,21 @@ If you prefer to follow manual steps, refer to this guide:
|
||||
1. Open and download this repository with
|
||||
```bash
|
||||
git clone https://github.com/arc53/DocsGPT.git
|
||||
cd DocsGPT
|
||||
```
|
||||
2. Create a `.env` file in your root directory and set your `API_KEY` with your [OpenAI API key](https://platform.openai.com/account/api-keys). (optional in case you want to use OpenAI)
|
||||
2. Create a `.env` file in your root directory and set the env variables.
|
||||
It should look like this inside:
|
||||
|
||||
```
|
||||
LLM_NAME=[docsgpt or openai or others]
|
||||
API_KEY=[if LLM_NAME is openai]
|
||||
```
|
||||
|
||||
See optional environment variables in the [/application/.env_sample](https://github.com/arc53/DocsGPT/blob/main/application/.env_sample) file.
|
||||
|
||||
3. Run the following commands:
|
||||
```bash
|
||||
docker-compose build && docker-compose up
|
||||
docker compose up
|
||||
```
|
||||
4. Navigate to http://localhost:5173/.
|
||||
|
||||
@@ -27,43 +37,28 @@ To stop, simply press **Ctrl + C**.
|
||||
|
||||
**For WINDOWS:**
|
||||
|
||||
To run the setup on Windows, you have two options: using the Windows Subsystem for Linux (WSL) or using Git Bash or Command Prompt.
|
||||
|
||||
**Option 1: Using Windows Subsystem for Linux (WSL):**
|
||||
|
||||
1. Install WSL if you haven't already. You can follow the official Microsoft documentation for installation: (https://learn.microsoft.com/en-us/windows/wsl/install).
|
||||
2. After setting up WSL, open the WSL terminal.
|
||||
3. Clone the repository and create the `.env` file:
|
||||
1. Open and download this repository with
|
||||
```bash
|
||||
git clone https://github.com/arc53/DocsGPT.git
|
||||
cd DocsGPT
|
||||
echo "API_KEY=Yourkey" > .env
|
||||
echo "VITE_API_STREAMING=true" >> .env
|
||||
```
|
||||
4. Run the following command to start the setup with Docker Compose:
|
||||
```bash
|
||||
./run-with-docker-compose.sh
|
||||
```
|
||||
6. Open your web browser and navigate to http://localhost:5173/.
|
||||
7. To stop the setup, just press **Ctrl + C** in the WSL terminal
|
||||
|
||||
**Option 2: Using Git Bash or Command Prompt (CMD):**
|
||||
2. Create a `.env` file in your root directory and set the env variables.
|
||||
It should look like this inside:
|
||||
|
||||
1. Install Git for Windows if you haven't already. Download it from the official website: (https://gitforwindows.org/).
|
||||
2. Open Git Bash or Command Prompt.
|
||||
3. Clone the repository and create the `.env` file:
|
||||
```bash
|
||||
git clone https://github.com/arc53/DocsGPT.git
|
||||
cd DocsGPT
|
||||
echo "API_KEY=Yourkey" > .env
|
||||
echo "VITE_API_STREAMING=true" >> .env
|
||||
```
|
||||
4. Run the following command to start the setup with Docker Compose:
|
||||
```bash
|
||||
./run-with-docker-compose.sh
|
||||
LLM_NAME=[docsgpt or openai or others]
|
||||
API_KEY=[if LLM_NAME is openai]
|
||||
```
|
||||
5. Open your web browser and navigate to http://localhost:5173/.
|
||||
6. To stop the setup, just press **Ctrl + C** in the Git Bash or Command Prompt terminal.
|
||||
|
||||
These steps should help you set up and run the project on Windows using either WSL or Git Bash/Command Prompt.
|
||||
See optional environment variables in the [/application/.env_sample](https://github.com/arc53/DocsGPT/blob/main/application/.env_sample) file.
|
||||
|
||||
3. Run the following command:
|
||||
|
||||
```bash
|
||||
docker-compose up
|
||||
```
|
||||
4. Navigate to http://localhost:5173/.
|
||||
5. To stop the setup, just press **Ctrl + C** in the WSL terminal
|
||||
|
||||
**Important:** Ensure that Docker is installed and properly configured on your Windows system for these steps to work.
|
||||
|
||||
@@ -7,6 +7,10 @@
|
||||
"title": "⚡️Quickstart",
|
||||
"href": "/Deploying/Quickstart"
|
||||
},
|
||||
"Development-Environment": {
|
||||
"title": "🛠️Development Environment",
|
||||
"href": "/Deploying/Development-Environment"
|
||||
},
|
||||
"Railway-Deploying": {
|
||||
"title": "🚂Deploying on Railway",
|
||||
"href": "/Deploying/Railway-Deploying"
|
||||
|
||||
@@ -25,7 +25,10 @@ DocsGPT 🦖 is an innovative open-source tool designed to simplify the retrieva
|
||||
|
||||
|
||||
|
||||
<Image src="/homevideo.gif" alt="homedemo" width={800} height={500}/>
|
||||
<video controls width={1920} height={1080} muted autoPlay loop playsInline>
|
||||
<source src="https://d3dg1063dc54p9.cloudfront.net/videos/demov4.mp4" type="video/mp4" />
|
||||
Your browser does not support the video tag.
|
||||
</video>
|
||||
|
||||
|
||||
Try it yourself: [https://www.docsgpt.cloud/](https://www.docsgpt.cloud/)
|
||||
|
||||
29
frontend/public/toolIcons/tool_postgres.svg
Normal file
29
frontend/public/toolIcons/tool_postgres.svg
Normal file
@@ -0,0 +1,29 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools -->
|
||||
<svg width="800px" height="800px" viewBox="-8.78 0 70 70" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns="http://www.w3.org/2000/svg">
|
||||
<metadata>
|
||||
<rdf:RDF>
|
||||
<cc:Work>
|
||||
<dc:subject>
|
||||
Data
|
||||
</dc:subject>
|
||||
<dc:identifier>
|
||||
sql-database-generic
|
||||
</dc:identifier>
|
||||
<dc:title>
|
||||
SQL Database (Generic)
|
||||
</dc:title>
|
||||
<dc:format>
|
||||
image/svg+xml
|
||||
</dc:format>
|
||||
<dc:publisher>
|
||||
Amido Limited
|
||||
</dc:publisher>
|
||||
<dc:creator>
|
||||
Richard Slater
|
||||
</dc:creator>
|
||||
<dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
|
||||
</cc:Work>
|
||||
</rdf:RDF>
|
||||
</metadata>
|
||||
<path d="m 852.97077,1013.9363 c -6.55238,-0.4723 -13.02857,-2.1216 -17.00034,-4.3296 -2.26232,-1.2576 -3.98589,-2.8032 -4.66223,-4.1807 l -0.4024,-0.8196 0,-25.70807 0,-25.7081 0.31843,-0.6465 c 1.42297,-2.889 5.96432,-5.4935 12.30378,-7.0562 2.15195,-0.5305 5.2586,-1.0588 7.79304,-1.3252 2.58797,-0.2721 9.44765,-0.2307 12.02919,0.073 6.86123,0.8061 12.69967,2.6108 16.29768,5.0377 1.38756,0.9359 2.81137,2.4334 3.29371,3.4642 l 0.41358,0.8838 -0.0354,25.6303 -0.0354,25.63047 -0.33195,0.6744 c -0.18257,0.3709 -0.73406,1.1007 -1.22553,1.6216 -2.99181,3.1715 -9.40919,5.5176 -17.8267,6.5172 -1.71567,0.2038 -9.16916,0.3686 -10.92937,0.2417 z m 12.07501,-22.02839 c -0.0252,-0.0657 -1.00472,-0.93831 -2.17671,-1.93922 -1.17199,-1.00091 -2.18138,-1.86687 -2.24309,-1.92436 -0.0617,-0.0575 0.15481,-0.26106 0.48117,-0.45237 0.32635,-0.19131 0.95163,-0.7235 1.3895,-1.18265 1.2805,-1.34272 1.88466,-3.00131 1.88466,-5.17388 0,-2.1388 -0.65162,-3.8645 -1.95671,-5.1818 -1.31533,-1.3278 -2.82554,-1.8983 -5.02486,-1.8983 -3.39007,0 -5.99368,1.9781 -6.82468,5.1851 -0.28586,1.1031 -0.28432,3.33211 0.003,4.31023 0.74941,2.55136 2.79044,4.40434 5.33062,4.83946 0.8596,0.14724 0.97605,0.21071 1.5621,0.85144 0.34829,0.38078 1.06301,1.14085 1.58827,1.68904 l 0.95501,0.9967 2.53878,0 c 1.39633,0 2.51816,-0.0537 2.49296,-0.11939 z m -8.70653,-7.10848 c -0.61119,-0.31868 -0.84225,-0.56599 -1.19079,-1.27453 -0.26919,-0.54724 -0.31522,-0.85851 -0.31824,-2.15197 -0.003,-1.3143 0.0388,-1.5983 0.31987,-2.169 0.45985,-0.9339 1.09355,-1.376 2.07384,-1.4469 1.36454,-0.099 2.15217,0.5707 2.56498,2.1801 0.50612,1.97321 -0.0504,4.07107 -1.26471,4.76729 -0.63707,0.36527 -1.58737,0.40659 -2.18495,0.095 z m -11.25315,3.66269 c 2.66179,-0.5048 4.1728,-2.0528 4.1728,-4.27495 0,-1.97137 -0.97548,-3.12004 -3.6716,-4.32364 -1.54338,-0.689 -2.10241,-1.1215 -2.10241,-1.6268 0,-0.4188 0.53052,-0.8777 1.14813,-0.993 0.60302,-0.1126 2.20237,0.1652 3.14683,0.5467 l 0.79167,0.3198 0,-1.7524 0,-1.7525 -0.85923,-0.1906 c -0.53103,-0.1178 -1.64689,-0.1885 -2.92137,-0.1849 -1.80528,0 -2.15881,0.044 -2.83818,0.3138 -1.98445,0.7878 -2.92613,2.1298 -2.91107,4.1485 0.0141,1.8898 1.01108,3.06864 3.49227,4.12912 1.46399,0.62572 2.05076,1.10218 2.05076,1.66522 0,1.1965 -1.99362,1.34375 -4.10437,0.30315 -0.57805,-0.28498 -1.09739,-0.54137 -1.1541,-0.56976 -0.0567,-0.0284 -0.10311,0.79023 -0.10311,1.81917 0,1.86239 0.002,1.87137 0.33919,1.99974 1.26979,0.48278 4.07626,0.69787 5.52379,0.42335 z m 30.4308,-1.72766 0,-1.58098 -2.40584,0 -2.40583,0 0,-5.43035 0,-5.4303 -2.13089,0 -2.13088,0 0,7.0113 0,7.01131 4.53672,0 4.53672,0 0,-1.58098 z m -14.84745,-27.70503 c 4.23447,-0.2937 7.4086,-0.8482 10.20178,-1.7821 2.78264,-0.9304 4.42643,-2.0562 4.79413,-3.2834 0.14166,-0.4729 0.13146,-0.6523 -0.0665,-1.1708 -0.88775,-2.3245 -5.84694,-4.1104 -13.42493,-4.8345 -3.24154,-0.3098 -9.13671,-0.2094 -12.22745,0.2081 -4.71604,0.6372 -8.54333,1.8208 -10.2451,3.1683 -3.44251,2.726 0.19793,5.7242 8.66397,7.1354 3.67084,0.6119 8.42674,0.828 12.30414,0.559 z" fill="#00bcf2" transform="translate(-830.906 -943.981)"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 4.1 KiB |
Reference in New Issue
Block a user