mirror of
https://github.com/arc53/DocsGPT.git
synced 2025-12-01 01:23:14 +00:00
Merge branch 'main' into faiss_delete_index
This commit is contained in:
BIN
Assets/DocsGPT tee-back.jpeg
Normal file
BIN
Assets/DocsGPT tee-back.jpeg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 48 KiB |
BIN
Assets/DocsGPT tee-front.jpeg
Normal file
BIN
Assets/DocsGPT tee-front.jpeg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 21 KiB |
@@ -19,19 +19,25 @@ Thank you for choosing to contribute to DocsGPT! We are all very grateful!
|
||||
|
||||
We value contributions in the form of discussions or suggestions. We recommend taking a look at existing issues and our [roadmap](https://github.com/orgs/arc53/projects/2).
|
||||
|
||||
If you're interested in contributing code, here are some important things to know:
|
||||
Before creating issues, please check out how the latest version of our app looks and works by launching it via [Quickstart](https://github.com/arc53/DocsGPT#quickstart) the version on our live demo is slightly modified with login. Your issues should relate to the version that you can launch via [Quickstart](https://github.com/arc53/DocsGPT#quickstart).
|
||||
|
||||
We have a frontend built with React (Vite) and a backend in Python.
|
||||
### 👨💻 If you're interested in contributing code, here are some important things to know:
|
||||
|
||||
### If you are looking to contribute to frontend (⚛️React, Vite):
|
||||
Tech Stack Overview:
|
||||
|
||||
- 🌐 Frontend: Built with React (Vite) ⚛️,
|
||||
|
||||
- 🖥 Backend: Developed in Python 🐍
|
||||
|
||||
### 🌐 If you are looking to contribute to frontend (⚛️React, Vite):
|
||||
|
||||
- The current frontend is being migrated from [`/application`](https://github.com/arc53/DocsGPT/tree/main/application) to [`/frontend`](https://github.com/arc53/DocsGPT/tree/main/frontend) with a new design, so please contribute to the new one.
|
||||
- Check out this [milestone](https://github.com/arc53/DocsGPT/milestone/1) and its issues.
|
||||
- The Figma design can be found [here](https://www.figma.com/file/OXLtrl1EAy885to6S69554/DocsGPT?node-id=0%3A1&t=hjWVuxRg9yi5YkJ9-1).
|
||||
- The updated Figma design can be found [here](https://www.figma.com/file/OXLtrl1EAy885to6S69554/DocsGPT?node-id=0%3A1&t=hjWVuxRg9yi5YkJ9-1).
|
||||
|
||||
Please try to follow the guidelines.
|
||||
|
||||
### If you are looking to contribute to Backend (🐍 Python):
|
||||
### 🖥 If you are looking to contribute to Backend (🐍 Python):
|
||||
|
||||
- Review our issues and contribute to [`/application`](https://github.com/arc53/DocsGPT/tree/main/application) or [`/scripts`](https://github.com/arc53/DocsGPT/tree/main/scripts) (please disregard old [`ingest_rst.py`](https://github.com/arc53/DocsGPT/blob/main/scripts/old/ingest_rst.py) [`ingest_rst_sphinx.py`](https://github.com/arc53/DocsGPT/blob/main/scripts/old/ingest_rst_sphinx.py) files; they will be deprecated soon).
|
||||
- All new code should be covered with unit tests ([pytest](https://github.com/pytest-dev/pytest)). Please find tests under [`/tests`](https://github.com/arc53/DocsGPT/tree/main/tests) folder.
|
||||
@@ -44,11 +50,62 @@ To run unit tests from the root of the repository, execute:
|
||||
python -m pytest
|
||||
```
|
||||
|
||||
### Workflow 📈 :
|
||||
- Fork repository
|
||||
- Make the required changes on your forked version
|
||||
- Commit those changes and submit those as a pull request so that it reflects on thr main repository.
|
||||
## Workflow 📈
|
||||
|
||||
Here's a step-by-step guide on how to contribute to DocsGPT:
|
||||
|
||||
1. **Fork the Repository:**
|
||||
- Click the "Fork" button at the top-right of this repository to create your fork.
|
||||
|
||||
2. **Create and Switch to a New Branch:**
|
||||
- Create a new branch for your contribution using:
|
||||
```shell
|
||||
git checkout -b your-branch-name
|
||||
```
|
||||
|
||||
3. **Make Changes:**
|
||||
- Make the required changes in your branch.
|
||||
|
||||
4. **Add Changes to the Staging Area:**
|
||||
- Add your changes to the staging area using:
|
||||
```shell
|
||||
git add .
|
||||
```
|
||||
|
||||
5. **Commit Your Changes:**
|
||||
- Commit your changes with a descriptive commit message using:
|
||||
```shell
|
||||
git commit -m "Your descriptive commit message"
|
||||
```
|
||||
|
||||
6. **Push Your Changes to the Remote Repository:**
|
||||
- Push your branch with changes to your fork on GitHub using:
|
||||
```shell
|
||||
git push origin your-branch-name
|
||||
```
|
||||
|
||||
7. **Submit a Pull Request (PR):**
|
||||
- Create a Pull Request from your branch to the main repository. Make sure to include a detailed description of your changes and reference any related issues.
|
||||
|
||||
8. **Collaborate:**
|
||||
- Be responsive to comments and feedback on your PR.
|
||||
- Make necessary updates as suggested.
|
||||
- Once your PR is approved, it will be merged into the main repository.
|
||||
|
||||
9. **Testing:**
|
||||
- Before submitting a Pull Request, ensure your code passes all unit tests.
|
||||
- To run unit tests from the root of the repository, execute:
|
||||
```shell
|
||||
python -m pytest
|
||||
```
|
||||
|
||||
*Note: You should run the unit test only after making the changes to the backend code.*
|
||||
|
||||
10. **Questions and Collaboration:**
|
||||
- Feel free to join our Discord. We're very friendly and welcoming to new contributors, so don't hesitate to reach out.
|
||||
|
||||
Thank you for considering contributing to DocsGPT! 🙏
|
||||
|
||||
## Questions/collaboration
|
||||
Feel free to join our [Discord](https://discord.gg/n5BX8dh8rU). We're very friendly and welcoming to new contributors, so don't hesitate to reach out.
|
||||
# Thank you so much for considering contributing to DocsGPT!🙏
|
||||
# Thank you so much for considering to contribute DocsGPT!🙏
|
||||
|
||||
@@ -32,4 +32,10 @@ Once you have created your PR and our maintainers have merged it, please fill in
|
||||
|
||||
Feel free to join our Discord server. We're here to help newcomers, so don't hesitate to jump in! [Join us here](https://discord.gg/n5BX8dh8rU).
|
||||
|
||||
Thank you very much for considering contributing to DocsGPT during Hacktoberfest! 🙏 Your contributions could earn you a stylish new t-shirt as a token of our appreciation. 🎁 Join us, and let's code together! 🚀
|
||||
Thank you very much for considering contributing to DocsGPT during Hacktoberfest! 🙏 Your contributions (not just simple typo) could earn you a stylish new t-shirt as a token of our appreciation. 🎁 Join us, and let's code together! 🚀
|
||||
|
||||
Here is a preview of the shirts:
|
||||
<p align="center">
|
||||
<img src="Assets/DocsGPT tee-front.jpeg" width="350" />
|
||||
<img src="Assets/DocsGPT tee-back.jpeg" width="350" />
|
||||
</p>
|
||||
|
||||
10
README.md
10
README.md
@@ -24,7 +24,7 @@ Say goodbye to time-consuming manual searches, and let <strong><a href="https://
|
||||
### Production Support / Help for companies:
|
||||
|
||||
We're eager to provide personalized assistance when deploying your DocsGPT to a live environment.
|
||||
- [Book Demo 👋](https://cal.com/arc53/docsgpt-demo-b2b)
|
||||
- [Book Demo 👋](https://airtable.com/appdeaL0F1qV8Bl2C/shrrJF1Ll7btCJRbP)
|
||||
- [Send Email ✉️](mailto:contact@arc53.com?subject=DocsGPT%20support%2Fsolutions)
|
||||
|
||||
### [🎉 Join the Hacktoberfest with DocsGPT and Earn a Free T-shirt! 🎉](https://github.com/arc53/DocsGPT/blob/main/HACKTOBERFEST.md)
|
||||
@@ -75,7 +75,7 @@ If you don't have enough resources to run it, you can use bitsnbytes to quantize
|
||||
|
||||
- Extensions - Chrome extension.
|
||||
|
||||
- Scripts - Script that creates similarity search index and stores for other libraries.
|
||||
- Scripts - Script that creates similarity search index for other libraries.
|
||||
|
||||
- Frontend - Frontend uses Vite and React.
|
||||
|
||||
@@ -105,6 +105,10 @@ Otherwise, refer to this Guide:
|
||||
|
||||
To stop, just run `Ctrl + C`.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Development environments
|
||||
|
||||
### Spin up mongo and redis
|
||||
@@ -127,7 +131,7 @@ Make sure you have Python 3.10 or 3.11 installed.
|
||||
(check out [`application/core/settings.py`](application/core/settings.py) if you want to see more config options.)
|
||||
|
||||
2. (optional) Create a Python virtual environment:
|
||||
You can follow the [Python official documentation](https://docs.python.org/3/tutorial/venv.html) for virtual environments .
|
||||
You can follow the [Python official documentation](https://docs.python.org/3/tutorial/venv.html) for virtual environments.
|
||||
|
||||
a) On Mac OS and Linux
|
||||
```commandline
|
||||
|
||||
@@ -41,7 +41,7 @@ Jinja2==3.1.2
|
||||
jmespath==1.0.1
|
||||
joblib==1.2.0
|
||||
kombu==5.2.4
|
||||
langchain==0.0.308
|
||||
langchain==0.0.312
|
||||
loguru==0.6.0
|
||||
lxml==4.9.2
|
||||
MarkupSafe==2.1.2
|
||||
@@ -104,3 +104,4 @@ urllib3==1.26.17
|
||||
vine==5.0.0
|
||||
wcwidth==0.2.6
|
||||
yarl==1.8.2
|
||||
sentence-transformers==2.2.2
|
||||
@@ -1,5 +1,5 @@
|
||||
from application.vectorstore.base import BaseVectorStore
|
||||
from langchain.vectorstores import FAISS
|
||||
from application.vectorstore.base import BaseVectorStore
|
||||
from application.core.settings import settings
|
||||
|
||||
class FaissStore(BaseVectorStore):
|
||||
@@ -7,23 +7,40 @@ class FaissStore(BaseVectorStore):
|
||||
def __init__(self, path, embeddings_key, docs_init=None):
|
||||
super().__init__()
|
||||
self.path = path
|
||||
embeddings = self._get_embeddings(settings.EMBEDDINGS_NAME, embeddings_key)
|
||||
if docs_init:
|
||||
self.docsearch = FAISS.from_documents(
|
||||
docs_init, self._get_embeddings(settings.EMBEDDINGS_NAME, embeddings_key)
|
||||
docs_init, embeddings
|
||||
)
|
||||
else:
|
||||
self.docsearch = FAISS.load_local(
|
||||
self.path, self._get_embeddings(settings.EMBEDDINGS_NAME, settings.EMBEDDINGS_KEY)
|
||||
self.path, embeddings
|
||||
)
|
||||
self.assert_embedding_dimensions(embeddings)
|
||||
|
||||
def search(self, *args, **kwargs):
|
||||
return self.docsearch.similarity_search(*args, **kwargs)
|
||||
|
||||
def add_texts(self, *args, **kwargs):
|
||||
return self.docsearch.add_texts(*args, **kwargs)
|
||||
|
||||
|
||||
def save_local(self, *args, **kwargs):
|
||||
return self.docsearch.save_local(*args, **kwargs)
|
||||
|
||||
def delete_index(self, *args, **kwargs):
|
||||
return self.docsearch.delete(*args, **kwargs)
|
||||
|
||||
def assert_embedding_dimensions(self, embeddings):
|
||||
"""
|
||||
Check that the word embedding dimension of the docsearch index matches
|
||||
the dimension of the word embeddings used
|
||||
"""
|
||||
if settings.EMBEDDINGS_NAME == "huggingface_sentence-transformers/all-mpnet-base-v2":
|
||||
try:
|
||||
word_embedding_dimension = embeddings.client[1].word_embedding_dimension
|
||||
except AttributeError as e:
|
||||
raise AttributeError("word_embedding_dimension not found in embeddings.client[1]") from e
|
||||
docsearch_index_dimension = self.docsearch.index.d
|
||||
if word_embedding_dimension != docsearch_index_dimension:
|
||||
raise ValueError(f"word_embedding_dimension ({word_embedding_dimension}) " +
|
||||
f"!= docsearch_index_word_embedding_dimension ({docsearch_index_dimension})")
|
||||
@@ -6,37 +6,30 @@ Here's a step-by-step guide on how to setup an Amazon Lightsail instance to host
|
||||
|
||||
(If you know how to create a Lightsail instance, you can skip to the recommended configuration part by clicking here).
|
||||
|
||||
### 1. Create an account or login to https://lightsail.aws.amazon.com
|
||||
### 1. Create an AWS Account:
|
||||
If you haven't already, create or log in to your AWS account at https://lightsail.aws.amazon.com.
|
||||
|
||||
### 2. Click on "Create instance"
|
||||
### 2. Create an Instance:
|
||||
|
||||
### 3. Create your instance
|
||||
a. Click "Create Instance."
|
||||
|
||||
The first step is to select the "Instance location". In most cases, there's no need to switch locations as the default one will work well.
|
||||
b. Select the "Instance location." In most cases, the default location works fine.
|
||||
|
||||
After that, it is time to pick your Instance Image. We recommend using "Linux/Unix" as the image and "Ubuntu 20.04 LTS" as the Operating System.
|
||||
c. Choose "Linux/Unix" as the image and "Ubuntu 20.04 LTS" as the Operating System.
|
||||
|
||||
As for instance plan, it'll vary depending on your unique demands, but a "1 GB, 1vCPU, 40GB SSD and 2TB transfer" setup should cover most scenarios.
|
||||
d. Configure the instance plan based on your requirements. A "1 GB, 1vCPU, 40GB SSD, and 2TB transfer" setup is recommended for most scenarios.
|
||||
|
||||
Lastly, identify your instance by giving it a unique name and then hit "Create instance".
|
||||
e. Give your instance a unique name and click "Create Instance."
|
||||
|
||||
PS: Once you create your instance, it'll likely take a few minutes for the setup to be completed.
|
||||
PS: It may take a few minutes for the instance setup to complete.
|
||||
|
||||
#### The recommended configuration is as follows:
|
||||
### Connecting to Your newly created Instance
|
||||
|
||||
- Ubuntu 20.04 LTS
|
||||
- 1GB RAM
|
||||
- 1vCPU
|
||||
- 40GB SSD Hard Drive
|
||||
- 2TB transfer
|
||||
Your instance will be ready a few minutes after creation. To access it, open the instance and click "Connect using SSH."
|
||||
|
||||
### Connecting to your newly created instance
|
||||
#### Clone the DocsGPT Repository
|
||||
|
||||
Your instance will be ready for use a few minutes after being created. To access it, just open it up and click on "Connect using SSH".
|
||||
|
||||
#### Clone the repository
|
||||
|
||||
A terminal window will pop up, and the first step will be to clone the DocsGPT git repository:
|
||||
A terminal window will pop up, and the first step will be to clone the DocsGPT Git repository:
|
||||
|
||||
`git clone https://github.com/arc53/DocsGPT.git`
|
||||
|
||||
@@ -56,15 +49,15 @@ And now install docker-compose:
|
||||
|
||||
`sudo apt install docker-compose`
|
||||
|
||||
#### Access the DocsGPT folder
|
||||
#### Access the DocsGPT Folder
|
||||
|
||||
Enter the following command to access the folder in which DocsGPT docker-compose file is present.
|
||||
Enter the following command to access the folder in which the DocsGPT docker-compose file is present.
|
||||
|
||||
`cd DocsGPT/`
|
||||
|
||||
#### Prepare the environment
|
||||
#### Prepare the Environment
|
||||
|
||||
Inside the DocsGPT folder, create a `.env` file and copy the contents of `.env_sample` into it.
|
||||
Inside the DocsGPT folder create a `.env` file and copy the contents of `.env_sample` into it.
|
||||
|
||||
`nano .env`
|
||||
|
||||
@@ -78,16 +71,16 @@ SELF_HOSTED_MODEL=false
|
||||
|
||||
To save the file, press CTRL+X, then Y, and then ENTER.
|
||||
|
||||
Next, we need to set a correct IP for our Backend. To do so, open the docker-compose.yml file:
|
||||
Next, set the correct IP for the Backend by opening the docker-compose.yml file:
|
||||
|
||||
`nano docker-compose.yml`
|
||||
|
||||
And change this line 7 `VITE_API_HOST=http://localhost:7091`
|
||||
And Change line 7 to: `VITE_API_HOST=http://localhost:7091`
|
||||
to this `VITE_API_HOST=http://<your instance public IP>:7091`
|
||||
|
||||
This will allow the frontend to connect to the backend.
|
||||
|
||||
#### Running the app
|
||||
#### Running the Application
|
||||
|
||||
You're almost there! Now that all the necessary bits and pieces have been installed, it is time to run the application. To do so, use the following command:
|
||||
|
||||
@@ -95,18 +88,17 @@ You're almost there! Now that all the necessary bits and pieces have been instal
|
||||
|
||||
Launching it for the first time will take a few minutes to download all the necessary dependencies and build.
|
||||
|
||||
Once this is done, you can go ahead and close the terminal window.
|
||||
Once this is done you can go ahead and close the terminal window.
|
||||
|
||||
#### Enabling ports
|
||||
#### Enabling Ports
|
||||
|
||||
Before you are able to access your live instance, you must first enable the port that it is using.
|
||||
a. Before you are able to access your live instance, you must first enable the port that it is using.
|
||||
|
||||
Open your Lightsail instance and head to "Networking".
|
||||
b. Open your Lightsail instance and head to "Networking".
|
||||
|
||||
Then click on "Add rule" under "IPv4 Firewall", enter `5173` as your port, and hit "Create".
|
||||
c. Then click on "Add rule" under "IPv4 Firewall", enter `5173` as your port, and hit "Create".
|
||||
Repeat the process for port `7091`.
|
||||
|
||||
#### Access your instance
|
||||
|
||||
Your instance will now be available under your Public IP Address and port `5173`. Enjoy!
|
||||
|
||||
Your instance is now available at your Public IP Address on port 5173. Enjoy using DocsGPT!
|
||||
|
||||
@@ -1,24 +1,107 @@
|
||||
## Launching Web App
|
||||
Note: Make sure you have Docker installed
|
||||
**Note**: Make sure you have Docker installed
|
||||
|
||||
On macOS or Linux, just write:
|
||||
**On macOS or Linux:**
|
||||
Just run the following command::
|
||||
|
||||
`./setup.sh`
|
||||
|
||||
It will install all the dependencies and give you an option to download the local model or use OpenAI
|
||||
This command will install all the necessary dependencies and provide you with an option to download the local model or use OpenAI.
|
||||
|
||||
Otherwise, refer to this Guide:
|
||||
If you prefer to follow manual steps, refer to this guide:
|
||||
|
||||
1. Open and download this repository with `git clone https://github.com/arc53/DocsGPT.git`.
|
||||
1. Open and download this repository with
|
||||
`git clone https://github.com/arc53/DocsGPT.git`.
|
||||
2. Create a `.env` file in your root directory and set your `API_KEY` with your [OpenAI API key](https://platform.openai.com/account/api-keys).
|
||||
3. Run `docker-compose build && docker-compose up`.
|
||||
3. Run the following commands:
|
||||
`docker-compose build && docker-compose up`.
|
||||
4. Navigate to `http://localhost:5173/`.
|
||||
|
||||
To stop, just run `Ctrl + C`.
|
||||
To stop, simply press Ctrl + C.
|
||||
|
||||
**For WINDOWS:**
|
||||
|
||||
To run the setup on Windows, you have two options: using the Windows Subsystem for Linux (WSL) or using Git Bash or Command Prompt.
|
||||
|
||||
**Option 1: Using Windows Subsystem for Linux (WSL):**
|
||||
|
||||
1. Install WSL if you haven't already. You can follow the official Microsoft documentation for installation: (https://learn.microsoft.com/en-us/windows/wsl/install).
|
||||
2. After setting up WSL, open the WSL terminal.
|
||||
3. Clone the repository and create the `.env` file:
|
||||
```
|
||||
git clone https://github.com/arc53/DocsGPT.git
|
||||
cd DocsGPT
|
||||
echo "API_KEY=Yourkey" > .env
|
||||
echo "VITE_API_STREAMING=true" >> .env
|
||||
```
|
||||
4. Run the following command to start the setup with Docker Compose:
|
||||
`./run-with-docker-compose.sh`
|
||||
5. Open your web browser and navigate to (http://localhost:5173/).
|
||||
6. To stop the setup, just press `Ctrl + C` in the WSL terminal
|
||||
|
||||
**Option 2: Using Git Bash or Command Prompt (CMD):**
|
||||
|
||||
1. Install Git for Windows if you haven't already. Download it from the official website: (https://gitforwindows.org/).
|
||||
2. Open Git Bash or Command Prompt.
|
||||
3. Clone the repository and create the `.env` file:
|
||||
```
|
||||
git clone https://github.com/arc53/DocsGPT.git
|
||||
cd DocsGPT
|
||||
echo "API_KEY=Yourkey" > .env
|
||||
echo "VITE_API_STREAMING=true" >> .env
|
||||
```
|
||||
4.Run the following command to start the setup with Docker Compose:
|
||||
`./run-with-docker-compose.sh`
|
||||
5.Open your web browser and navigate to (http://localhost:5173/).
|
||||
6.To stop the setup, just press Ctrl + C in the Git Bash or Command Prompt terminal.
|
||||
|
||||
These steps should help you set up and run the project on Windows using either WSL or Git Bash/Command Prompt.
|
||||
**Important:** Ensure that Docker is installed and properly configured on your Windows system for these steps to work.
|
||||
|
||||
|
||||
For WINDOWS:
|
||||
|
||||
To run the given setup on Windows, you can use the Windows Subsystem for Linux (WSL) or a Git Bash terminal to execute similar commands. Here are the steps adapted for Windows:
|
||||
|
||||
Option 1: Using Windows Subsystem for Linux (WSL):
|
||||
|
||||
1. Install WSL if you haven't already. You can follow the official Microsoft documentation for installation: (https://learn.microsoft.com/en-us/windows/wsl/install).
|
||||
2. After setting up WSL, open the WSL terminal.
|
||||
3. Clone the repository and create the `.env` file:
|
||||
```
|
||||
git clone https://github.com/arc53/DocsGPT.git
|
||||
cd DocsGPT
|
||||
echo "API_KEY=Yourkey" > .env
|
||||
echo "VITE_API_STREAMING=true" >> .env
|
||||
```
|
||||
4. Run the following command to start the setup with Docker Compose:
|
||||
`./run-with-docker-compose.sh`
|
||||
5. Open your web browser and navigate to (http://localhost:5173/).
|
||||
6. To stop the setup, just press `Ctrl + C` in the WSL terminal
|
||||
|
||||
Option 2: Using Git Bash or Command Prompt (CMD):
|
||||
|
||||
1. Install Git for Windows if you haven't already. You can download it from the official website: (https://gitforwindows.org/).
|
||||
2. Open Git Bash or Command Prompt.
|
||||
3. Clone the repository and create the `.env` file:
|
||||
```
|
||||
git clone https://github.com/arc53/DocsGPT.git
|
||||
cd DocsGPT
|
||||
echo "API_KEY=Yourkey" > .env
|
||||
echo "VITE_API_STREAMING=true" >> .env
|
||||
```
|
||||
4.Run the following command to start the setup with Docker Compose:
|
||||
`./run-with-docker-compose.sh`
|
||||
5.Open your web browser and navigate to (http://localhost:5173/).
|
||||
6.To stop the setup, just press Ctrl + C in the Git Bash or Command Prompt terminal.
|
||||
|
||||
These steps should help you set up and run the project on Windows using either WSL or Git Bash/Command Prompt. Make sure you have Docker installed and properly configured on your Windows system for this to work.
|
||||
|
||||
|
||||
### Chrome Extension
|
||||
|
||||
To install the Chrome extension:
|
||||
#### Installing the Chrome extension:
|
||||
To enhance your DocsGPT experience, you can install the DocsGPT Chrome extension. Here's how:
|
||||
|
||||
1. In the DocsGPT GitHub repository, click on the "Code" button and select "Download ZIP".
|
||||
2. Unzip the downloaded file to a location you can easily access.
|
||||
|
||||
@@ -104,7 +104,9 @@ fetch("http://localhost:5001/api/task_status?task_id=b2d2a0f4-387c-44fd-a443-e4f
|
||||
|
||||
Responses:
|
||||
There are two types of responses:
|
||||
1. While task is still running, where "current" will show progress from 0 to 100:
|
||||
|
||||
1. While the task is still running, the 'current' value will show progress from 0 to 100.
|
||||
|
||||
```json
|
||||
{
|
||||
"result": {
|
||||
@@ -133,7 +135,8 @@ There are two types of responses:
|
||||
```
|
||||
|
||||
### /api/delete_old
|
||||
Deletes old Vector stores:
|
||||
Deletes old Vector Stores:
|
||||
|
||||
```js
|
||||
// Task status (GET http://127.0.0.1:5000/api/docs_check)
|
||||
fetch("http://localhost:5001/api/task_status?task_id=b2d2a0f4-387c-44fd-a443-e4fe2e7454d1", {
|
||||
@@ -144,7 +147,6 @@ fetch("http://localhost:5001/api/task_status?task_id=b2d2a0f4-387c-44fd-a443-e4f
|
||||
})
|
||||
.then((res) => res.text())
|
||||
.then(console.log.bind(console))
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
|
||||
@@ -17,9 +17,9 @@ If you want for bot to stop responding to questions for a specific user or sessi
|
||||
|
||||
|
||||
### Optional (extra validation)
|
||||
In `app.py` uncomment lines 12-13 and 71-75
|
||||
1. In `app.py` uncomment lines 12-13 and 71-75
|
||||
|
||||
in your `.env` file add:
|
||||
2. in your `.env` file add:
|
||||
|
||||
```
|
||||
account_id=(optional) 1
|
||||
|
||||
@@ -165,11 +165,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
|
||||
*/
|
||||
|
||||
useEffect(() => {
|
||||
if (isMobile) {
|
||||
setNavOpen(false);
|
||||
return;
|
||||
}
|
||||
setNavOpen(true);
|
||||
setNavOpen(!isMobile);
|
||||
}, [isMobile]);
|
||||
|
||||
return (
|
||||
@@ -194,7 +190,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
|
||||
ref={navRef}
|
||||
className={`${
|
||||
!navOpen && '-ml-96 md:-ml-[18rem]'
|
||||
} duration-20 fixed z-20 flex h-full w-72 flex-col border-r-2 bg-gray-50 transition-all`}
|
||||
} duration-20 fixed top-0 z-20 flex h-full w-72 flex-col border-r-2 bg-gray-50 transition-all`}
|
||||
>
|
||||
<div className={'visible h-16 w-full border-b-2 md:h-12'}>
|
||||
<button
|
||||
@@ -229,22 +225,18 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
|
||||
}
|
||||
>
|
||||
<img src={Message} className="ml-4 w-5"></img>
|
||||
<p className="my-auto text-eerie-black">New Chat</p>
|
||||
<p className="my-auto text-sm text-eerie-black">New Chat</p>
|
||||
</NavLink>
|
||||
<div className="conversations-container max-h-[25rem] overflow-y-auto">
|
||||
{conversations
|
||||
? conversations.map((conversation) => (
|
||||
<ConversationTile
|
||||
key={conversation.id}
|
||||
conversation={conversation}
|
||||
selectConversation={(id) => handleConversationClick(id)}
|
||||
onDeleteConversation={(id) => handleDeleteConversation(id)}
|
||||
onSave={(conversation) =>
|
||||
updateConversationName(conversation)
|
||||
}
|
||||
/>
|
||||
))
|
||||
: null}
|
||||
{conversations?.map((conversation) => (
|
||||
<ConversationTile
|
||||
key={conversation.id}
|
||||
conversation={conversation}
|
||||
selectConversation={(id) => handleConversationClick(id)}
|
||||
onDeleteConversation={(id) => handleDeleteConversation(id)}
|
||||
onSave={(conversation) => updateConversationName(conversation)}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
|
||||
<div className="flex-grow border-b-2 border-gray-100"></div>
|
||||
@@ -289,7 +281,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
|
||||
<p className="ml-5 flex-1 overflow-hidden overflow-ellipsis whitespace-nowrap py-3">
|
||||
{doc.name} {doc.version}
|
||||
</p>
|
||||
{doc.location === 'local' ? (
|
||||
{doc.location === 'local' && (
|
||||
<img
|
||||
src={Exit}
|
||||
alt="Exit"
|
||||
@@ -300,7 +292,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) {
|
||||
handleDeleteClick(index, doc);
|
||||
}}
|
||||
/>
|
||||
) : null}
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -39,11 +39,7 @@ export default function Conversation() {
|
||||
useEffect(() => {
|
||||
const observerCallback: IntersectionObserverCallback = (entries) => {
|
||||
entries.forEach((entry) => {
|
||||
if (entry.isIntersecting) {
|
||||
setHasScrolledToLast(true);
|
||||
} else {
|
||||
setHasScrolledToLast(false);
|
||||
}
|
||||
setHasScrolledToLast(entry.isIntersecting);
|
||||
});
|
||||
};
|
||||
|
||||
@@ -121,7 +117,7 @@ export default function Conversation() {
|
||||
|
||||
return (
|
||||
<div className="flex flex-col justify-center p-4 md:flex-row">
|
||||
{queries.length > 0 && !hasScrolledToLast ? (
|
||||
{queries.length > 0 && !hasScrolledToLast && (
|
||||
<button
|
||||
onClick={scrollIntoView}
|
||||
aria-label="scroll to bottom"
|
||||
@@ -133,7 +129,7 @@ export default function Conversation() {
|
||||
className="h4- w-4 opacity-50 md:h-5 md:w-5"
|
||||
/>
|
||||
</button>
|
||||
) : null}
|
||||
)}
|
||||
|
||||
{queries.length > 0 && (
|
||||
<div className="mt-20 flex flex-col transition-all md:w-3/4">
|
||||
|
||||
@@ -74,7 +74,7 @@ const ConversationBubble = forwardRef<
|
||||
<div className="flex self-start">
|
||||
<Avatar className="mt-2 text-2xl" avatar="🦖"></Avatar>
|
||||
<div
|
||||
className={`ml-2 mr-5 flex flex-col items-center rounded-3xl bg-gray-1000 p-3.5 ${
|
||||
className={`ml-2 mr-5 flex flex-col rounded-3xl bg-gray-1000 p-3.5 ${
|
||||
type === 'ERROR'
|
||||
? 'flex-row rounded-full border border-transparent bg-[#FFE7E7] p-2 py-5 text-sm font-normal text-red-3000 dark:border-red-2000 dark:text-white'
|
||||
: 'flex-col rounded-3xl'
|
||||
@@ -118,7 +118,7 @@ const ConversationBubble = forwardRef<
|
||||
<>
|
||||
<span className="mt-3 h-px w-full bg-[#DEDEDE]"></span>
|
||||
<div className="mt-3 flex w-full flex-row flex-wrap items-center justify-start gap-2">
|
||||
<div className="py-1 px-2 text-base font-semibold">
|
||||
<div className="py-1 text-base font-semibold">
|
||||
Sources:
|
||||
</div>
|
||||
<div className="flex flex-row flex-wrap items-center justify-start gap-2">
|
||||
|
||||
@@ -68,7 +68,7 @@ export default function ConversationTile({
|
||||
onClick={() => {
|
||||
selectConversation(conversation.id);
|
||||
}}
|
||||
className={`my-auto mx-4 mt-4 flex h-12 cursor-pointer items-center justify-between gap-4 rounded-3xl hover:bg-gray-100 ${
|
||||
className={`my-auto mx-4 mt-4 flex h-9 cursor-pointer items-center justify-between gap-4 rounded-3xl hover:bg-gray-100 ${
|
||||
conversationId === conversation.id ? 'bg-gray-100' : ''
|
||||
}`}
|
||||
>
|
||||
@@ -77,7 +77,7 @@ export default function ConversationTile({
|
||||
conversationId === conversation.id ? 'w-[75%]' : 'w-[95%]'
|
||||
} gap-4`}
|
||||
>
|
||||
<img src={Message} className="ml-2 w-5"></img>
|
||||
<img src={Message} className="ml-4 w-5"></img>
|
||||
{isEdit ? (
|
||||
<input
|
||||
autoFocus
|
||||
@@ -92,7 +92,7 @@ export default function ConversationTile({
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
{conversationId === conversation.id ? (
|
||||
{conversationId === conversation.id && (
|
||||
<div className="flex">
|
||||
<img
|
||||
src={isEdit ? CheckMark : Edit}
|
||||
@@ -122,7 +122,7 @@ export default function ConversationTile({
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
) : null}
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -55,9 +55,8 @@ export default function Upload({
|
||||
setProgress(undefined);
|
||||
setModalState('INACTIVE');
|
||||
}}
|
||||
className={`rounded-3xl bg-purple-30 px-4 py-2 text-sm font-medium text-white ${
|
||||
isCancellable ? '' : 'hidden'
|
||||
}`}
|
||||
className={`rounded-3xl bg-purple-30 px-4 py-2 text-sm font-medium text-white ${isCancellable ? '' : 'hidden'
|
||||
}`}
|
||||
>
|
||||
Finish
|
||||
</button>
|
||||
@@ -206,7 +205,9 @@ export default function Upload({
|
||||
<div className="flex flex-row-reverse">
|
||||
<button
|
||||
onClick={uploadFile}
|
||||
className="ml-6 rounded-3xl bg-purple-30 py-2 px-6 text-white"
|
||||
className={`ml-6 rounded-3xl bg-purple-30 text-white ${files.length > 0 ? '' : 'bg-opacity-75 text-opacity-80'
|
||||
} py-2 px-6`}
|
||||
disabled={files.length === 0} // Disable the button if no file is selected
|
||||
>
|
||||
Train
|
||||
</button>
|
||||
@@ -227,9 +228,8 @@ export default function Upload({
|
||||
|
||||
return (
|
||||
<article
|
||||
className={`${
|
||||
modalState === 'ACTIVE' ? 'visible' : 'hidden'
|
||||
} absolute z-30 h-screen w-screen bg-gray-alpha`}
|
||||
className={`${modalState === 'ACTIVE' ? 'visible' : 'hidden'
|
||||
} absolute z-30 h-screen w-screen bg-gray-alpha`}
|
||||
>
|
||||
<article className="mx-auto mt-24 flex w-[90vw] max-w-lg flex-col gap-4 rounded-lg bg-white p-6 shadow-lg">
|
||||
{view}
|
||||
|
||||
@@ -47,7 +47,7 @@ javalang==0.13.0
|
||||
Jinja2==3.1.2
|
||||
jmespath==1.0.1
|
||||
joblib==1.3.1
|
||||
langchain==0.0.308
|
||||
langchain==0.0.312
|
||||
lxml==4.9.3
|
||||
manifest-ml==0.1.8
|
||||
MarkupSafe==2.1.3
|
||||
|
||||
19
tests/test_vector_store.py
Normal file
19
tests/test_vector_store.py
Normal file
@@ -0,0 +1,19 @@
|
||||
"""
|
||||
Tests regarding the vector store class, including checking
|
||||
compatibility between different transformers and local vector
|
||||
stores (index.faiss)
|
||||
"""
|
||||
import pytest
|
||||
from application.vectorstore.faiss import FaissStore
|
||||
from application.core.settings import settings
|
||||
|
||||
def test_init_local_faiss_store_huggingface():
|
||||
"""
|
||||
Test that asserts that trying to initialize a FaissStore with
|
||||
the huggingface sentence transformer below together with the
|
||||
index.faiss file in the application/ folder results in a
|
||||
dimension mismatch error.
|
||||
"""
|
||||
settings.EMBEDDINGS_NAME = "huggingface_sentence-transformers/all-mpnet-base-v2"
|
||||
with pytest.raises(ValueError):
|
||||
FaissStore("application/", "", None)
|
||||
Reference in New Issue
Block a user