From 4b1dad96cd4a7d90af5d85fc4aae0ac0dcc53343 Mon Sep 17 00:00:00 2001 From: Ayan Joshi Date: Sun, 8 Oct 2023 12:42:56 +0530 Subject: [PATCH 01/15] Commit --- application/prompts/custom_prompt.txt | 34 +++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 application/prompts/custom_prompt.txt diff --git a/application/prompts/custom_prompt.txt b/application/prompts/custom_prompt.txt new file mode 100644 index 00000000..054da1e1 --- /dev/null +++ b/application/prompts/custom_prompt.txt @@ -0,0 +1,34 @@ +Welcome to the custom prompt file. You can use this file to create prompts for various tasks or experiments. Below are some examples of prompts you can use: + +1. *Summarization Task:* + + Prompt: Summarize the following article in 3-4 sentences. + Input: [Insert article here] + Output: Summarized article + +2. *Translation Task:* + + Prompt: Translate the English text into French. + Input: "Hello, how are you?" + Output: "Bonjour, comment ça va ?" + +3. *Coding Task:* + + Prompt: Write a Python function that calculates the factorial of a given number. + Input: (Provide any specific number) + Output: (Gives factorial of the number) + +4. *Question-Answering Task:* + + Prompt: Answer the following question based on the given passage. + Input: [Insert passage here] + Question: [Insert question here] + Output: [Provided answer of the question] + +5. *Story Generation Task:* + + Prompt: Generate a short story about a detective solving a mystery. + Input: [Provide any additional context or details] + Output: [Gives short summary of the story] + +Feel free to customize and add your own prompts to this file. Each section includes a prompt and optional input. Happy Coding ! \ No newline at end of file From c688656607a6be2cbdf57038a75bf533556fb4ab Mon Sep 17 00:00:00 2001 From: Ayan Joshi Date: Mon, 9 Oct 2023 14:27:59 +0530 Subject: [PATCH 02/15] Commit --- application/prompts/custom_prompt.txt | 34 -------------------- docs/pages/Guides/Customising-prompts.md | 40 ++++++++++++++++++++++-- 2 files changed, 38 insertions(+), 36 deletions(-) delete mode 100644 application/prompts/custom_prompt.txt diff --git a/application/prompts/custom_prompt.txt b/application/prompts/custom_prompt.txt deleted file mode 100644 index 054da1e1..00000000 --- a/application/prompts/custom_prompt.txt +++ /dev/null @@ -1,34 +0,0 @@ -Welcome to the custom prompt file. You can use this file to create prompts for various tasks or experiments. Below are some examples of prompts you can use: - -1. *Summarization Task:* - - Prompt: Summarize the following article in 3-4 sentences. - Input: [Insert article here] - Output: Summarized article - -2. *Translation Task:* - - Prompt: Translate the English text into French. - Input: "Hello, how are you?" - Output: "Bonjour, comment ça va ?" - -3. *Coding Task:* - - Prompt: Write a Python function that calculates the factorial of a given number. - Input: (Provide any specific number) - Output: (Gives factorial of the number) - -4. *Question-Answering Task:* - - Prompt: Answer the following question based on the given passage. - Input: [Insert passage here] - Question: [Insert question here] - Output: [Provided answer of the question] - -5. *Story Generation Task:* - - Prompt: Generate a short story about a detective solving a mystery. - Input: [Provide any additional context or details] - Output: [Gives short summary of the story] - -Feel free to customize and add your own prompts to this file. Each section includes a prompt and optional input. Happy Coding ! \ No newline at end of file diff --git a/docs/pages/Guides/Customising-prompts.md b/docs/pages/Guides/Customising-prompts.md index 1d3a7d4a..e2e26133 100644 --- a/docs/pages/Guides/Customising-prompts.md +++ b/docs/pages/Guides/Customising-prompts.md @@ -1,4 +1,40 @@ -## To customize a main prompt navigate to `/application/prompt/combine_prompt.txt` +# Customizing the Main Prompt -You can try editing it to see how the model responses. +To customize the main prompt for DocsGPT, follow these steps: + +1. Navigate to `/application/prompt/combine_prompt.txt`. + +2. Edit the `combine_prompt.txt` file to modify the prompt text. You can experiment with different phrasings and structures to see how the model responds. + +## Example Prompt Modification + +**Original Prompt:** +```markdown +QUESTION: How to merge tables in pandas? +========= +Content: pandas provides various facilities for easily combining together Series or DataFrame with various kinds of set logic for the indexes and relational algebra functionality in the case of join / merge-type operations. +Source: 28-pl +Content: pandas provides a single function, merge(), as the entry point for all standard database join operations between DataFrame or named Series objects: \n\npandas.merge(left, right, how='inner', on=None, left_on=None, right_on=None, left_index=False, right_index=False, sort=False, suffixes=('_x', '_y'), copy=True, indicator=False, validate=None) +Source: 30-pl +========= +FINAL ANSWER: To merge two tables in pandas, you can use the pd.merge() function. The basic syntax is: \n\npd.merge(left, right, on, how) \n\nwhere left and right are the two tables to merge, on is the column to merge on, and how is the type of merge to perform. \n\nFor example, to merge the two tables df1 and df2 on the column 'id', you can use: \n\npd.merge(df1, df2, on='id', how='inner') +SOURCES: 28-pl 30-pl + + +## Modified Prompt (for illustration): + +QUESTION: Explain the concept of pandas merge in Python. +========= +Content: The pandas merge function in Python allows you to combine DataFrame objects by performing SQL-style joins. It is a powerful tool for data manipulation and analysis. +Source: Official pandas documentation +Content: You can use the merge function with various options such as 'how,' 'on,' 'left_on,' and 'right_on' to control how the merging is done. +Source: Data science tutorial blog +========= +FINAL ANSWER: To perform a pandas merge in Python, use the `pd.merge()` function. This function takes two DataFrames as input and merges them based on the specified columns. For example, to merge DataFrames `df1` and `df2` on the 'key' column, you can use: `pd.merge(df1, df2, on='key', how='inner')`. +SOURCES: Official pandas documentation, Data science tutorial blog + + +## Conclusion + +Customizing the main prompt for DocsGPT allows you to tailor the AI's responses to your unique requirements. Whether you need in-depth explanations, code examples, or specific insights, you can achieve it by modifying the main prompt. Remember to experiment and fine-tune your prompts to get the best results. From d7b28a35867e5751aa4f735d5af0bbbf120dca2a Mon Sep 17 00:00:00 2001 From: Ayan Joshi Date: Tue, 10 Oct 2023 19:10:24 +0530 Subject: [PATCH 03/15] Update Customising-prompts.md --- docs/pages/Guides/Customising-prompts.md | 26 +++++++----------------- 1 file changed, 7 insertions(+), 19 deletions(-) diff --git a/docs/pages/Guides/Customising-prompts.md b/docs/pages/Guides/Customising-prompts.md index e2e26133..2426974c 100644 --- a/docs/pages/Guides/Customising-prompts.md +++ b/docs/pages/Guides/Customising-prompts.md @@ -10,28 +10,16 @@ To customize the main prompt for DocsGPT, follow these steps: **Original Prompt:** ```markdown -QUESTION: How to merge tables in pandas? -========= -Content: pandas provides various facilities for easily combining together Series or DataFrame with various kinds of set logic for the indexes and relational algebra functionality in the case of join / merge-type operations. -Source: 28-pl -Content: pandas provides a single function, merge(), as the entry point for all standard database join operations between DataFrame or named Series objects: \n\npandas.merge(left, right, how='inner', on=None, left_on=None, right_on=None, left_index=False, right_index=False, sort=False, suffixes=('_x', '_y'), copy=True, indicator=False, validate=None) -Source: 30-pl -========= -FINAL ANSWER: To merge two tables in pandas, you can use the pd.merge() function. The basic syntax is: \n\npd.merge(left, right, on, how) \n\nwhere left and right are the two tables to merge, on is the column to merge on, and how is the type of merge to perform. \n\nFor example, to merge the two tables df1 and df2 on the column 'id', you can use: \n\npd.merge(df1, df2, on='id', how='inner') -SOURCES: 28-pl 30-pl +You are a DocsGPT, friendly and helpful AI assistant by Arc53 that provides help with documents. You give thorough answers with code examples if possible. +Use the following pieces of context to help answer the users question. If its not relevant to the question, provide friendly responses. +You have access to chat history, and can use it to help answer the question. +When using code examples, use the following format: +(code) +{summaries} -## Modified Prompt (for illustration): +Thank you -QUESTION: Explain the concept of pandas merge in Python. -========= -Content: The pandas merge function in Python allows you to combine DataFrame objects by performing SQL-style joins. It is a powerful tool for data manipulation and analysis. -Source: Official pandas documentation -Content: You can use the merge function with various options such as 'how,' 'on,' 'left_on,' and 'right_on' to control how the merging is done. -Source: Data science tutorial blog -========= -FINAL ANSWER: To perform a pandas merge in Python, use the `pd.merge()` function. This function takes two DataFrames as input and merges them based on the specified columns. For example, to merge DataFrames `df1` and `df2` on the 'key' column, you can use: `pd.merge(df1, df2, on='key', how='inner')`. -SOURCES: Official pandas documentation, Data science tutorial blog ## Conclusion From ca3e549dd4f35d3c7b3b33ff85a32e861f6ccd83 Mon Sep 17 00:00:00 2001 From: shivanandmn Date: Thu, 12 Oct 2023 13:29:52 +0530 Subject: [PATCH 04/15] added delete index of vector store in faiss --- application/vectorstore/faiss.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/application/vectorstore/faiss.py b/application/vectorstore/faiss.py index 217b0457..e5e8f7f5 100644 --- a/application/vectorstore/faiss.py +++ b/application/vectorstore/faiss.py @@ -24,3 +24,6 @@ class FaissStore(BaseVectorStore): def save_local(self, *args, **kwargs): return self.docsearch.save_local(*args, **kwargs) + + def delete_index(self, *args, **kwargs): + return self.docsearch.delete(*args, **kwargs) From 2e95666939a5df2ea0d1c8be2d47a54b5820103c Mon Sep 17 00:00:00 2001 From: shivanandmn Date: Mon, 16 Oct 2023 02:46:48 +0530 Subject: [PATCH 05/15] added new endpoint --- application/api/user/routes.py | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/application/api/user/routes.py b/application/api/user/routes.py index fdff2e97..b80562b2 100644 --- a/application/api/user/routes.py +++ b/application/api/user/routes.py @@ -84,6 +84,19 @@ def api_feedback(): ) return {"status": http.client.responses.get(response.status_code, "ok")} +@user.route("/api/delete_by_ids", methods=["get"]) +def delete_by_ids(): + """Delete by ID. These are the IDs in the vectorstore""" + + ids = request.args.get("path") + if not ids: + return {"status": "error"} + + if settings.VECTOR_STORE == "faiss": + result = vectors_collection.delete_index(ids=ids) + if result: + return {"status": "ok"} + return {"status": "error"} @user.route("/api/delete_old", methods=["get"]) def delete_old(): From a51e25dbdeb040e78b331de985fde236c75b5cd4 Mon Sep 17 00:00:00 2001 From: Shruti Sen <115914670+shruti-sen2004@users.noreply.github.com> Date: Tue, 17 Oct 2023 08:09:32 +0530 Subject: [PATCH 06/15] Update react-widget.md --- docs/pages/Extensions/react-widget.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/pages/Extensions/react-widget.md b/docs/pages/Extensions/react-widget.md index 1cc11321..a31306a2 100644 --- a/docs/pages/Extensions/react-widget.md +++ b/docs/pages/Extensions/react-widget.md @@ -14,9 +14,9 @@ import "docsgpt/dist/style.css"; Then you can use it like this: `` DocsGPTWidget takes 3 props: -- `apiHost` — URL of your DocsGPT API. -- `selectDocs` — documentation that you want to use for your widget (e.g. `default` or `local/docs1.zip`). -- `apiKey` — usually it's empty. +1. `apiHost` — URL of your DocsGPT API. +2. `selectDocs` — documentation that you want to use for your widget (e.g. `default` or `local/docs1.zip`). +3. `apiKey` — usually it's empty. ### How to use DocsGPTWidget with [Nextra](https://nextra.site/) (Next.js + MDX) Install your widget as described above and then go to your `pages/` folder and create a new file `_app.js` with the following content: From ca48f000bd2e2852fec5650b5f1f1eb1f196c376 Mon Sep 17 00:00:00 2001 From: ManiYadla Date: Tue, 17 Oct 2023 20:40:27 +0530 Subject: [PATCH 07/15] changed the typos --- README.md | 70 +++++++++++++++++++++++++++++-------------------------- 1 file changed, 37 insertions(+), 33 deletions(-) diff --git a/README.md b/README.md index c24a7483..53d590a2 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@

- DocsGPT is a cutting-edge open-source solution that streamlines the process of finding information in project documentation. With its integration of the powerful GPT models, developers can easily ask questions about a project and receive accurate answers. + DocsGPT is a cutting-edge open-source solution that streamlines the process of finding information in the project documentation. With its integration of the powerful GPT models, developers can easily ask questions about a project and receive accurate answers. Say goodbye to time-consuming manual searches, and let DocsGPT help you quickly find the information you need. Try it out and see how it revolutionizes your project documentation experience. Contribute to its development and be a part of the future of AI-powered assistance.

@@ -21,61 +21,56 @@ Say goodbye to time-consuming manual searches, and let - + + Contributors ## License From f0d4847946a5389dfd9cf9b1d6d214c6f054b680 Mon Sep 17 00:00:00 2001 From: debghs <145260557+debghs@users.noreply.github.com> Date: Wed, 18 Oct 2023 12:46:29 +0530 Subject: [PATCH 09/15] Update How-to-train-on-other-documentation.md --- .../How-to-train-on-other-documentation.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/pages/Guides/How-to-train-on-other-documentation.md b/docs/pages/Guides/How-to-train-on-other-documentation.md index 2e8e4afa..aa1ff41d 100644 --- a/docs/pages/Guides/How-to-train-on-other-documentation.md +++ b/docs/pages/Guides/How-to-train-on-other-documentation.md @@ -12,28 +12,28 @@ It currently uses OPEN_AI to create the vector store, so make sure your document You can usually find documentation on Github in `docs/` folder for most open-source projects. ### 1. Find documentation in .rst/.md and create a folder with it in your scripts directory -- Name it `inputs/` -- Put all your .rst/.md files in there -- The search is recursive, so you don't need to flatten them +- Name it `inputs/`. +- Put all your .rst/.md files in there. +- The search is recursive, so you don't need to flatten them. -If there are no .rst/.md files just convert whatever you find to .txt and feed it. (don't forget to change the extension in script) +If there are no .rst/.md files just convert whatever you find to .txt file and feed it. (don't forget to change the extension in script) ### 2. Create .env file in `scripts/` folder And write your OpenAI API key inside -`OPENAI_API_KEY=` +`OPENAI_API_KEY=`. ### 3. Run scripts/ingest.py `python ingest.py ingest` -It will tell you how much it will cost +It will tell you how much it will cost. ### 4. Move `index.faiss` and `index.pkl` generated in `scripts/output` to `application/` folder. ### 5. Run web app -Once you run it will use new context that is relevant to your documentation -Make sure you select default in the dropdown in the UI +Once you run it will use new context that is relevant to your documentation. +Make sure you select default in the dropdown in the UI. ## Customization You can learn more about options while running ingest.py by running: From d40ea44ae66a8f8b9fc5ad2663e79a79721386d8 Mon Sep 17 00:00:00 2001 From: Shivam Bhatnagar <88842947+Bitnagar@users.noreply.github.com> Date: Wed, 18 Oct 2023 10:33:44 +0000 Subject: [PATCH 10/15] fix(frontend): fix navigation z-index in mobiles --- frontend/src/Navigation.tsx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/frontend/src/Navigation.tsx b/frontend/src/Navigation.tsx index be308eab..f34050cd 100644 --- a/frontend/src/Navigation.tsx +++ b/frontend/src/Navigation.tsx @@ -362,7 +362,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) { -
+
@@ -205,9 +206,10 @@ export default function Upload({
@@ -228,8 +230,9 @@ export default function Upload({ return (
{view} From d93266fee29aa6ead88475c60c7ce9086cc633c8 Mon Sep 17 00:00:00 2001 From: Ayush-Prabhu <126278585+Ayush-Prabhu@users.noreply.github.com> Date: Wed, 18 Oct 2023 16:21:15 +0530 Subject: [PATCH 12/15] Update How-to-use-different-LLM.md Corrected grammatical errors to remove ambiguity and improve professionalism. --- docs/pages/Guides/How-to-use-different-LLM.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/pages/Guides/How-to-use-different-LLM.md b/docs/pages/Guides/How-to-use-different-LLM.md index 0eaf483e..8d7cccce 100644 --- a/docs/pages/Guides/How-to-use-different-LLM.md +++ b/docs/pages/Guides/How-to-use-different-LLM.md @@ -1,10 +1,10 @@ -Fortunately, there are many providers for LLM's and some of them can even be run locally +Fortunately, there are many providers for LLMs, and some of them can even be run locally. There are two models used in the app: 1. Embeddings. 2. Text generation. -By default, we use OpenAI's models but if you want to change it or even run it locally, it's very simple! +By default, we use OpenAI's models, but if you want to change it or even run it locally, it's very simple! ### Go to .env file or set environment variables: @@ -31,6 +31,6 @@ Alternatively, if you wish to run Llama locally, you can run `setup.sh` and choo That's it! ### Hosting everything locally and privately (for using our optimised open-source models) -If you are working with important data and don't want anything to leave your premises. +If you are working with critical data and don't want anything to leave your premises. -Make sure you set `SELF_HOSTED_MODEL` as true in your `.env` variable and for your `LLM_NAME` you can use anything that's on Hugging Face. +Make sure you set `SELF_HOSTED_MODEL` as true in your `.env` variable, and for your `LLM_NAME`, you can use anything that is on Hugging Face. From 2f6213c944c0c7bc948e57c3b02243668223331b Mon Sep 17 00:00:00 2001 From: "beKool.sh" <76424367+beKoool@users.noreply.github.com> Date: Thu, 19 Oct 2023 11:11:30 +0545 Subject: [PATCH 13/15] Change font weight --- docs/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/README.md b/docs/README.md index 8958f6d0..4e41a0de 100644 --- a/docs/README.md +++ b/docs/README.md @@ -50,4 +50,4 @@ yarn dev - Now, you should be able to view the docs on your local environment by visiting `http://localhost:5000`. You can explore the different markdown files and make changes as you see fit. -- Footnotes: This guide assumes you have Node.js and npm installed. The guide involves running a local server using yarn, and viewing the documentation offline. If you encounter any issues, it may be worth verifying your Node.js and npm installations and whether you have installed yarn correctly. +- **Footnotes:** This guide assumes you have Node.js and npm installed. The guide involves running a local server using yarn, and viewing the documentation offline. If you encounter any issues, it may be worth verifying your Node.js and npm installations and whether you have installed yarn correctly. From 06f0129b595d04be48870314bab51148cece0081 Mon Sep 17 00:00:00 2001 From: Saksham Tyagi Date: Thu, 19 Oct 2023 20:12:40 +0530 Subject: [PATCH 14/15] =?UTF-8?q?=F0=9F=92=84=20About=20margin?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- frontend/src/About.tsx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/frontend/src/About.tsx b/frontend/src/About.tsx index fe26835e..bde6ad95 100644 --- a/frontend/src/About.tsx +++ b/frontend/src/About.tsx @@ -4,7 +4,7 @@ export default function About() { return (
-
+

About DocsGPT

🦖

From fcfaa04cc6c0387ce1a688770e0efed9a5c7462c Mon Sep 17 00:00:00 2001 From: Ankit Matth Date: Fri, 20 Oct 2023 05:42:12 +0530 Subject: [PATCH 15/15] About page aligned in center --- frontend/src/About.tsx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/frontend/src/About.tsx b/frontend/src/About.tsx index fe26835e..53a87bc8 100644 --- a/frontend/src/About.tsx +++ b/frontend/src/About.tsx @@ -4,7 +4,7 @@ export default function About() { return (
-
+

About DocsGPT

🦖

@@ -54,7 +54,7 @@ export default function About() { Currently It uses{' '} DocsGPT{' '} documentation, so it will respond to information relevant to{' '} - DocsGPT . If you + DocsGPT. If you want to train it on different documentation - please follow