diff --git a/README.md b/README.md index c24a7483..8105e67e 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@

- DocsGPT is a cutting-edge open-source solution that streamlines the process of finding information in project documentation. With its integration of the powerful GPT models, developers can easily ask questions about a project and receive accurate answers. + DocsGPT is a cutting-edge open-source solution that streamlines the process of finding information in the project documentation. With its integration of the powerful GPT models, developers can easily ask questions about a project and receive accurate answers. Say goodbye to time-consuming manual searches, and let DocsGPT help you quickly find the information you need. Try it out and see how it revolutionizes your project documentation experience. Contribute to its development and be a part of the future of AI-powered assistance.

@@ -21,61 +21,56 @@ Say goodbye to time-consuming manual searches, and let - + + Contributors ## License + The source code license is [MIT](https://opensource.org/license/mit/), as described in the [LICENSE](LICENSE) file. Built with [🦜️🔗 LangChain](https://github.com/hwchase17/langchain) diff --git a/application/api/user/routes.py b/application/api/user/routes.py index fdff2e97..b80562b2 100644 --- a/application/api/user/routes.py +++ b/application/api/user/routes.py @@ -84,6 +84,19 @@ def api_feedback(): ) return {"status": http.client.responses.get(response.status_code, "ok")} +@user.route("/api/delete_by_ids", methods=["get"]) +def delete_by_ids(): + """Delete by ID. These are the IDs in the vectorstore""" + + ids = request.args.get("path") + if not ids: + return {"status": "error"} + + if settings.VECTOR_STORE == "faiss": + result = vectors_collection.delete_index(ids=ids) + if result: + return {"status": "ok"} + return {"status": "error"} @user.route("/api/delete_old", methods=["get"]) def delete_old(): diff --git a/application/vectorstore/faiss.py b/application/vectorstore/faiss.py index e8960fe0..3a0a7b82 100644 --- a/application/vectorstore/faiss.py +++ b/application/vectorstore/faiss.py @@ -27,6 +27,9 @@ class FaissStore(BaseVectorStore): def save_local(self, *args, **kwargs): return self.docsearch.save_local(*args, **kwargs) + def delete_index(self, *args, **kwargs): + return self.docsearch.delete(*args, **kwargs) + def assert_embedding_dimensions(self, embeddings): """ Check that the word embedding dimension of the docsearch index matches @@ -40,5 +43,4 @@ class FaissStore(BaseVectorStore): docsearch_index_dimension = self.docsearch.index.d if word_embedding_dimension != docsearch_index_dimension: raise ValueError(f"word_embedding_dimension ({word_embedding_dimension}) " + - f"!= docsearch_index_word_embedding_dimension ({docsearch_index_dimension})") - + f"!= docsearch_index_word_embedding_dimension ({docsearch_index_dimension})") \ No newline at end of file diff --git a/docs/README.md b/docs/README.md index 8958f6d0..4e41a0de 100644 --- a/docs/README.md +++ b/docs/README.md @@ -50,4 +50,4 @@ yarn dev - Now, you should be able to view the docs on your local environment by visiting `http://localhost:5000`. You can explore the different markdown files and make changes as you see fit. -- Footnotes: This guide assumes you have Node.js and npm installed. The guide involves running a local server using yarn, and viewing the documentation offline. If you encounter any issues, it may be worth verifying your Node.js and npm installations and whether you have installed yarn correctly. +- **Footnotes:** This guide assumes you have Node.js and npm installed. The guide involves running a local server using yarn, and viewing the documentation offline. If you encounter any issues, it may be worth verifying your Node.js and npm installations and whether you have installed yarn correctly. diff --git a/docs/pages/Deploying/Hosting-the-app.md b/docs/pages/Deploying/Hosting-the-app.md index 74799ea3..31c3f55a 100644 --- a/docs/pages/Deploying/Hosting-the-app.md +++ b/docs/pages/Deploying/Hosting-the-app.md @@ -102,3 +102,7 @@ Repeat the process for port `7091`. #### Access your instance Your instance is now available at your Public IP Address on port 5173. Enjoy using DocsGPT! + +## Other Deployment Options + +- [Deploy DocsGPT on Civo Compute Cloud](https://dev.to/rutamhere/deploying-docsgpt-on-civo-compute-c) diff --git a/docs/pages/Developing/API-docs.md b/docs/pages/Developing/API-docs.md index 2d832845..09e4f875 100644 --- a/docs/pages/Developing/API-docs.md +++ b/docs/pages/Developing/API-docs.md @@ -1,9 +1,25 @@ -Currently, the application provides the following main API endpoints: +# API Endpoints Documentation -### /api/answer -It's a POST request that sends a JSON in body with 4 values. It will receive an answer for a user provided question. -Here is a JavaScript fetch example: +*Currently, the application provides the following main API endpoints:* + +### 1. /api/answer +**Description:** + +This endpoint is used to request answers to user-provided questions. + +**Request:** + +Method: POST +Headers: Content-Type should be set to "application/json; charset=utf-8" +Request Body: JSON object with the following fields: +* **question:** The user's question +* **history:** (Optional) Previous conversation history +* **api_key:** Your API key +* **embeddings_key:** Your embeddings key +* **active_docs:** The location of active documentation + +Here is a JavaScript Fetch Request example: ```js // answer (POST http://127.0.0.1:5000/api/answer) fetch("http://127.0.0.1:5000/api/answer", { @@ -18,8 +34,9 @@ fetch("http://127.0.0.1:5000/api/answer", { .then(console.log.bind(console)) ``` -In response, you will get a JSON document like this one: +**Response** +In response, you will get a JSON document containing the answer,query and the result: ```json { "answer": " Hi there! How can I help you?\n", @@ -28,10 +45,17 @@ In response, you will get a JSON document like this one: } ``` -### /api/docs_check -It will make sure documentation is loaded on a server (just run it every time user is switching between libraries (documentations)). -It's a POST request that sends a JSON in a body with 1 value. Here is a JavaScript fetch example: +### 2. /api/docs_check +**Description:** + +This endpoint will make sure documentation is loaded on the server (just run it every time user is switching between libraries (documentations)). + +**Request:** + +Headers: Content-Type should be set to "application/json; charset=utf-8" +Request Body: JSON object with the field: +* **docs:** The location of the documentation ```js // answer (POST http://127.0.0.1:5000/api/docs_check) fetch("http://127.0.0.1:5000/api/docs_check", { @@ -45,7 +69,9 @@ fetch("http://127.0.0.1:5000/api/docs_check", { .then(console.log.bind(console)) ``` -In response, you will get a JSON document like this one: +**Response:** + +In response, you will get a JSON document like this one indicating whether the documentation exists or not.: ```json { "status": "exists" @@ -53,18 +79,36 @@ In response, you will get a JSON document like this one: ``` -### /api/combine -Provides JSON that tells UI which vectors are available and where they are located with a simple get request. +### 3. /api/combine +**Description:** + +This endpoint provides information about available vectors and their locations with a simple GET request. + +**Request:** + +Method: GET + +**Response:** Response will include: `date`, `description`, `docLink`, `fullName`, `language`, `location` (local or docshub), `model`, `name`, `version`. + Example of JSON in Docshub and local: + image -### /api/upload -Uploads file that needs to be trained, response is JSON with task ID, which can be used to check on task's progress +### 4. /api/upload +**Description:** + +This endpoint is used to upload a file that needs to be trained, response is JSON with task ID, which can be used to check on task's progress. + +**Request:** + +Method: POST +Request Body: A multipart/form-data form with file upload and additional fields, including "user" and "name." + HTML example: ```html @@ -79,20 +123,24 @@ HTML example: ``` -Response: -```json -{ - "status": "ok", - "task_id": "b2684988-9047-428b-bd47-08518679103c" -} +**Response:** -``` +JSON response with a status and a task ID that can be used to check the task's progress. -### /api/task_status -Gets task status (`task_id`) from `/api/upload`: + +### 5. /api/task_status +**Description:** + +This endpoint is used to get the status of a task (`task_id`) from `/api/upload` + +**Request:** +Method: GET +Query Parameter: task_id (task ID to check) + +**Sample JavaScript Fetch Request:** ```js // Task status (Get http://127.0.0.1:5000/api/task_status) -fetch("http://localhost:5001/api/task_status?task_id=b2d2a0f4-387c-44fd-a443-e4fe2e7454d1", { +fetch("http://localhost:5001/api/task_status?task_id=YOUR_TASK_ID", { "method": "GET", "headers": { "Content-Type": "application/json; charset=utf-8" @@ -102,7 +150,8 @@ fetch("http://localhost:5001/api/task_status?task_id=b2d2a0f4-387c-44fd-a443-e4f .then(console.log.bind(console)) ``` -Responses: +**Response:** + There are two types of responses: 1. While the task is still running, the 'current' value will show progress from 0 to 100. @@ -134,9 +183,14 @@ There are two types of responses: } ``` -### /api/delete_old -Deletes old Vector Stores: +### 6. /api/delete_old +**Description:** +This endpoint is used to delete old Vector Stores. + +**Request:** + +Method: GET ```js // Task status (GET http://127.0.0.1:5000/api/docs_check) fetch("http://localhost:5001/api/task_status?task_id=b2d2a0f4-387c-44fd-a443-e4fe2e7454d1", { @@ -148,8 +202,10 @@ fetch("http://localhost:5001/api/task_status?task_id=b2d2a0f4-387c-44fd-a443-e4f .then((res) => res.text()) .then(console.log.bind(console)) -Response: +``` +**Response:** +JSON response indicating the status of the operation. ```json { "status": "ok" } ``` diff --git a/docs/pages/Extensions/react-widget.md b/docs/pages/Extensions/react-widget.md index 1cc11321..a31306a2 100644 --- a/docs/pages/Extensions/react-widget.md +++ b/docs/pages/Extensions/react-widget.md @@ -14,9 +14,9 @@ import "docsgpt/dist/style.css"; Then you can use it like this: `` DocsGPTWidget takes 3 props: -- `apiHost` — URL of your DocsGPT API. -- `selectDocs` — documentation that you want to use for your widget (e.g. `default` or `local/docs1.zip`). -- `apiKey` — usually it's empty. +1. `apiHost` — URL of your DocsGPT API. +2. `selectDocs` — documentation that you want to use for your widget (e.g. `default` or `local/docs1.zip`). +3. `apiKey` — usually it's empty. ### How to use DocsGPTWidget with [Nextra](https://nextra.site/) (Next.js + MDX) Install your widget as described above and then go to your `pages/` folder and create a new file `_app.js` with the following content: diff --git a/docs/pages/Guides/Customising-prompts.md b/docs/pages/Guides/Customising-prompts.md index 19dcdefd..6cfbbff7 100644 --- a/docs/pages/Guides/Customising-prompts.md +++ b/docs/pages/Guides/Customising-prompts.md @@ -1,4 +1,27 @@ -## To customize a main prompt, navigate to `/application/prompt/combine_prompt.txt` +# Customizing the Main Prompt -You can try editing it to see how the model responses. +To customize the main prompt for DocsGPT, follow these steps: + +1. Navigate to `/application/prompt/combine_prompt.txt`. + +2. Edit the `combine_prompt.txt` file to modify the prompt text. You can experiment with different phrasings and structures to see how the model responds. + +## Example Prompt Modification + +**Original Prompt:** +```markdown +You are a DocsGPT, friendly and helpful AI assistant by Arc53 that provides help with documents. You give thorough answers with code examples if possible. +Use the following pieces of context to help answer the users question. If its not relevant to the question, provide friendly responses. +You have access to chat history, and can use it to help answer the question. +When using code examples, use the following format: + +(code) +{summaries} +``` + + + +## Conclusion + +Customizing the main prompt for DocsGPT allows you to tailor the AI's responses to your unique requirements. Whether you need in-depth explanations, code examples, or specific insights, you can achieve it by modifying the main prompt. Remember to experiment and fine-tune your prompts to get the best results. diff --git a/docs/pages/Guides/How-to-train-on-other-documentation.md b/docs/pages/Guides/How-to-train-on-other-documentation.md index 2e8e4afa..aa1ff41d 100644 --- a/docs/pages/Guides/How-to-train-on-other-documentation.md +++ b/docs/pages/Guides/How-to-train-on-other-documentation.md @@ -12,28 +12,28 @@ It currently uses OPEN_AI to create the vector store, so make sure your document You can usually find documentation on Github in `docs/` folder for most open-source projects. ### 1. Find documentation in .rst/.md and create a folder with it in your scripts directory -- Name it `inputs/` -- Put all your .rst/.md files in there -- The search is recursive, so you don't need to flatten them +- Name it `inputs/`. +- Put all your .rst/.md files in there. +- The search is recursive, so you don't need to flatten them. -If there are no .rst/.md files just convert whatever you find to .txt and feed it. (don't forget to change the extension in script) +If there are no .rst/.md files just convert whatever you find to .txt file and feed it. (don't forget to change the extension in script) ### 2. Create .env file in `scripts/` folder And write your OpenAI API key inside -`OPENAI_API_KEY=` +`OPENAI_API_KEY=`. ### 3. Run scripts/ingest.py `python ingest.py ingest` -It will tell you how much it will cost +It will tell you how much it will cost. ### 4. Move `index.faiss` and `index.pkl` generated in `scripts/output` to `application/` folder. ### 5. Run web app -Once you run it will use new context that is relevant to your documentation -Make sure you select default in the dropdown in the UI +Once you run it will use new context that is relevant to your documentation. +Make sure you select default in the dropdown in the UI. ## Customization You can learn more about options while running ingest.py by running: diff --git a/docs/pages/Guides/How-to-use-different-LLM.md b/docs/pages/Guides/How-to-use-different-LLM.md index 0eaf483e..8d7cccce 100644 --- a/docs/pages/Guides/How-to-use-different-LLM.md +++ b/docs/pages/Guides/How-to-use-different-LLM.md @@ -1,10 +1,10 @@ -Fortunately, there are many providers for LLM's and some of them can even be run locally +Fortunately, there are many providers for LLMs, and some of them can even be run locally. There are two models used in the app: 1. Embeddings. 2. Text generation. -By default, we use OpenAI's models but if you want to change it or even run it locally, it's very simple! +By default, we use OpenAI's models, but if you want to change it or even run it locally, it's very simple! ### Go to .env file or set environment variables: @@ -31,6 +31,6 @@ Alternatively, if you wish to run Llama locally, you can run `setup.sh` and choo That's it! ### Hosting everything locally and privately (for using our optimised open-source models) -If you are working with important data and don't want anything to leave your premises. +If you are working with critical data and don't want anything to leave your premises. -Make sure you set `SELF_HOSTED_MODEL` as true in your `.env` variable and for your `LLM_NAME` you can use anything that's on Hugging Face. +Make sure you set `SELF_HOSTED_MODEL` as true in your `.env` variable, and for your `LLM_NAME`, you can use anything that is on Hugging Face. diff --git a/frontend/src/Navigation.tsx b/frontend/src/Navigation.tsx index be308eab..79383bbf 100644 --- a/frontend/src/Navigation.tsx +++ b/frontend/src/Navigation.tsx @@ -255,7 +255,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) { src={Arrow2} alt="arrow" className={`${ - isDocsListOpen ? 'rotate-0' : 'rotate-180' + !isDocsListOpen ? 'rotate-0' : 'rotate-180' } ml-auto mr-3 w-3 transition-all`} /> @@ -362,7 +362,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) { -
+
@@ -205,9 +206,10 @@ export default function Upload({
@@ -228,8 +230,9 @@ export default function Upload({ return (
{view}