LLM/DocsGPT
1
0
Fork 0
mirror of https://github.com/arc53/DocsGPT.git synced 2026-04-29 13:31:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into main

Browse Source
This commit is contained in:
Alex
2023-10-08 00:01:43 +01:00
committed by GitHub
parent 5ca5e0d00f 835a04358c
commit 66fd402f00
18 changed files with 71 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
README.md
View File

@@ -24,7 +24,7 @@ Say goodbye to time-consuming manual searches, and let <strong>DocsGPT</strong>
### Production Support / Help for companies: ### Production Support / Help for companies:
We're eager to provide personalized assistance when deploying your DocsGPT to a live environment. We're eager to provide personalized assistance when deploying your DocsGPT to a live environment.
- [Schedule Demo 👋](https://cal.com/arc53/docsgpt-demo-b2b?date=2023-10-04&month=2023-10) - [Get Support 👋](https://airtable.com/appdeaL0F1qV8Bl2C/shrrJF1Ll7btCJRbP)
- [Send Email ✉️](mailto:contact@arc53.com?subject=DocsGPT%20support%2Fsolutions) - [Send Email ✉️](mailto:contact@arc53.com?subject=DocsGPT%20support%2Fsolutions)
### [🎉 Join the Hacktoberfest with DocsGPT and Earn a Free T-shirt! 🎉](https://github.com/arc53/DocsGPT/blob/main/HACKTOBERFEST.md) ### [🎉 Join the Hacktoberfest with DocsGPT and Earn a Free T-shirt! 🎉](https://github.com/arc53/DocsGPT/blob/main/HACKTOBERFEST.md)
@@ -143,11 +143,20 @@ Make sure you have Node version 16 or higher.
2. Install dependencies by running `npm install`. 2. Install dependencies by running `npm install`.
3. Run the app using `npm run dev`. 3. Run the app using `npm run dev`.
## Contributing
Please refer to the [CONTRIBUTING.md](CONTRIBUTING.md) file for information about how to get involved. We welcome issues, questions, and pull requests.
## Code Of Conduct
We as members, contributors, and leaders, pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation. Please refer to the [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) file for more information about contributing.
## Many Thanks To Our Contributors ## Many Thanks To Our Contributors
<a href="[https://github.com/arc53/DocsGPT/graphs/contributors](https://docsgpt.arc53.com/)"> <a href="[https://github.com/arc53/DocsGPT/graphs/contributors](https://docsgpt.arc53.com/)">
<img src="https://contrib.rocks/image?repo=arc53/DocsGPT" /> <img src="https://contrib.rocks/image?repo=arc53/DocsGPT" />
</a> </a>
## License
The source code license is MIT, as described in the LICENSE file.
Built with [🦜️🔗 LangChain](https://github.com/hwchase17/langchain) Built with [🦜️🔗 LangChain](https://github.com/hwchase17/langchain)

2
application/parser/file/html_parser.py
View File

@@ -57,7 +57,7 @@ class HTMLParser(BaseParser):
title_indexes = [i for i, isd_el in enumerate(isd) if isd_el['type'] == 'Title'] title_indexes = [i for i, isd_el in enumerate(isd) if isd_el['type'] == 'Title']
# Creating 'Chunks' - List of lists of strings # Creating 'Chunks' - List of lists of strings
# each list starting with with isd_el['type'] = 'Title' and all the data till the next 'Title' # each list starting with isd_el['type'] = 'Title' and all the data till the next 'Title'
# Each Chunk can be thought of as an individual set of data, which can be sent to the model # Each Chunk can be thought of as an individual set of data, which can be sent to the model
# Where Each Title is grouped together with the data under it # Where Each Title is grouped together with the data under it

2
docs/pages/Developing/API-docs.md
View File

@@ -73,7 +73,7 @@ HTML example:
<input type="text" name="user" value="local" hidden> <input type="text" name="user" value="local" hidden>
<input type="text" name="name" placeholder="Name:"> <input type="text" name="name" placeholder="Name:">
<button type="submit" class="py-2 px-4 text-white bg-blue-500 rounded-md hover:bg-blue-600 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-blue-500"> <button type="submit" class="py-2 px-4 text-white bg-purple-30 rounded-md hover:bg-purple-30 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-purple-30">
Upload Upload
</button> </button>
</form> </form>

2
docs/pages/Guides/How-to-train-on-other-documentation.md
View File

@@ -41,7 +41,7 @@ You can learn more about options while running ingest.py by running:
`python ingest.py --help` `python ingest.py --help`
| Options | | | Options | |
|:--------------------------------:|:------------------------------------------------------------------------------------------------------------------------------:| |:--------------------------------:|:------------------------------------------------------------------------------------------------------------------------------:|
| **ingest** | Runs 'ingest' function converting documentation to to Faiss plus Index format | | **ingest** | Runs 'ingest' function converting documentation to Faiss plus Index format |
| --dir TEXT | List of paths to directory for index creation. E.g. --dir inputs --dir inputs2 [default: inputs] | | --dir TEXT | List of paths to directory for index creation. E.g. --dir inputs --dir inputs2 [default: inputs] |
| --file TEXT | File paths to use (Optional; overrides directory) E.g. --files inputs/1.md --files inputs/2.md | | --file TEXT | File paths to use (Optional; overrides directory) E.g. --files inputs/1.md --files inputs/2.md |
| --recursive / --no-recursive | Whether to recursively search in subdirectories [default: recursive] | | --recursive / --no-recursive | Whether to recursively search in subdirectories [default: recursive] |

4
extensions/chrome/popup.html
View File

@@ -20,7 +20,7 @@
<div class="bg-indigo-500 text-white p-2 rounded-lg mb-2 self-start"> <div class="bg-indigo-500 text-white p-2 rounded-lg mb-2 self-start">
<p class="text-sm">Hello, ask me anything about this library. Im here to help</p> <p class="text-sm">Hello, ask me anything about this library. Im here to help</p>
</div> </div>
<div class="bg-blue-500 text-white p-2 rounded-lg mb-2 self-end"> <div class="bg-purple-30 text-white p-2 rounded-lg mb-2 self-end">
<p class="text-sm">How to create API key for Api gateway?</p> <p class="text-sm">How to create API key for Api gateway?</p>
</div> </div>
<div class="bg-indigo-500 text-white p-2 rounded-lg mb-2 self-start"> <div class="bg-indigo-500 text-white p-2 rounded-lg mb-2 self-start">
@@ -46,7 +46,7 @@
<div class=" flex mt-4 mb-2"> <div class=" flex mt-4 mb-2">
<form id="message-form"> <form id="message-form">
<input id="message-input" class="bg-white p-2 rounded-lg ml-2 w-[26rem]" type="text" placeholder="Type your message here..."> <input id="message-input" class="bg-white p-2 rounded-lg ml-2 w-[26rem]" type="text" placeholder="Type your message here...">
<button class="bg-blue-500 text-white p-2 rounded-lg ml-2 mr-2 ml-2" type="submit">Send</button> <button class="bg-purple-30 text-white p-2 rounded-lg ml-2 mr-2 ml-2" type="submit">Send</button>
</form> </form>
</div> </div>

2
extensions/chrome/popup.js
View File

@@ -3,7 +3,7 @@ document.getElementById("message-form").addEventListener("submit", function(even
var message = document.getElementById("message-input").value; var message = document.getElementById("message-input").value;
chrome.runtime.sendMessage({msg: "sendMessage", message: message}, function(response) { chrome.runtime.sendMessage({msg: "sendMessage", message: message}, function(response) {
console.log(response.response); console.log(response.response);
msg_html = '<div class="bg-blue-500 text-white p-2 rounded-lg mb-2 self-end"><p class="text-sm">' msg_html = '<div class="bg-purple-30 text-white p-2 rounded-lg mb-2 self-end"><p class="text-sm">'
msg_html += message msg_html += message
msg_html += '</p></div>' msg_html += '</p></div>'
document.getElementById("messages").innerHTML += msg_html; document.getElementById("messages").innerHTML += msg_html;

2
extensions/react-widget/dist/index.es.js vendored
View File

@@ -366,7 +366,7 @@ function fr() {
} }
function ye(e) { function ye(e) {
if (Je(e)) if (Je(e))
return g("The provided key is an unsupported type %s. This value must be coerced to a string before before using it here.", qe(e)), ge(e); return g("The provided key is an unsupported type %s. This value must be coerced to a string before using it here.", qe(e)), ge(e);
} }
var W = O.ReactCurrentOwner, Be = { var W = O.ReactCurrentOwner, Be = {
key: !0, key: !0,

2
extensions/react-widget/dist/index.es.js.map vendored
View File

File diff suppressed because one or more lines are too long

2
extensions/react-widget/dist/index.umd.js vendored
View File

File diff suppressed because one or more lines are too long

2
extensions/react-widget/dist/index.umd.js.map vendored
View File

File diff suppressed because one or more lines are too long

4
frontend/src/About.tsx
View File

@@ -51,8 +51,8 @@ export default function About() {
</div> </div>
<p> <p>
Currently It uses DocsGPT documentation, so it will respond to Currently It uses <span className= "text-blue-950 font-medium">DocsGPT</span> documentation, so it will respond to
information relevant to DocsGPT. If you want to train it on different information relevant to <span className= "text-blue-950 font-medium">DocsGPT</span> . If you want to train it on different
documentation - please follow documentation - please follow
<a <a
className="text-blue-500" className="text-blue-500"

52
frontend/src/Hero.tsx
View File

@@ -5,28 +5,26 @@ export default function Hero({ className = '' }: { className?: string }) {
<p className="mr-2 text-4xl font-semibold">DocsGPT</p> <p className="mr-2 text-4xl font-semibold">DocsGPT</p>
<p className="text-[27px]">🦖</p> <p className="text-[27px]">🦖</p>
</div> </div>
<p className="mb-3 text-center leading-6 text-black-1000"> <p className="mb-2 text-center text-sm font-normal leading-6 text-black-1000">
Welcome to DocsGPT, your technical documentation assistant! Welcome to DocsGPT, your technical documentation assistant!
</p> </p>
<p className="mb-3 text-center leading-6 text-black-1000"> <p className="mb-2 text-center text-sm font-normal leading-6 text-black-1000 ">
Enter a query related to the information in the documentation you Enter a query related to the information in the documentation you
selected to receive selected to receive and we will provide you with the most relevant
<br /> and we will provide you with the most relevant answers. answers.
</p> </p>
<p className="mb-3 text-center leading-6 text-black-1000"> <p className="mb-2 text-center text-sm font-normal leading-6 text-black-1000 ">
Start by entering your query in the input field below and we will do the Start by entering your query in the input field below and we will do the
rest! rest!
</p> </p>
<div className="sections mt-1 flex flex-wrap items-center justify-center gap-1 sm:gap-1 md:gap-0 "> <div className="sections mt-7 flex flex-wrap items-center justify-center gap-1 sm:gap-1 md:gap-0 ">
<div className=" rounded-[50px] bg-gradient-to-l from-[#6EE7B7]/70 via-[#3B82F6] to-[#9333EA]/50 p-1 md:rounded-tr-none md:rounded-br-none"> <div className=" rounded-[28px] bg-gradient-to-l from-[#6EE7B7]/70 via-[#3B82F6] to-[#9333EA]/50 p-1 md:rounded-tr-none md:rounded-br-none">
<div className="h-full rounded-[45px] bg-white p-6 md:rounded-tr-none md:rounded-br-none"> <div className="h-full rounded-[25px] bg-white p-6 md:rounded-tr-none md:rounded-br-none">
<img <img src="/message-text.svg" alt="lock" className="h-7 w-7" />
src="/message-text.svg" <h2 className="mt-2 mb-5 text-base font-normal leading-6">
alt="lock" Chat with Your Data
className="h-[24px] w-[24px]" </h2>
/> <p className="w-[300px] text-sm font-normal leading-6 text-black-1000">
<h2 className="mt-2 mb-3 text-lg font-bold">Chat with Your Data</h2>
<p className="w-[250px] text-xs text-gray-500">
DocsGPT will use your data to answer questions. Whether its DocsGPT will use your data to answer questions. Whether its
documentation, source code, or Microsoft files, DocsGPT allows you documentation, source code, or Microsoft files, DocsGPT allows you
to have interactive conversations and find answers based on the to have interactive conversations and find answers based on the
@@ -35,11 +33,13 @@ export default function Hero({ className = '' }: { className?: string }) {
</div> </div>
</div> </div>
<div className=" rounded-[50px] bg-gradient-to-r from-[#6EE7B7]/70 via-[#3B82F6] to-[#9333EA]/50 p-1 md:rounded-none md:py-1 md:px-0"> <div className=" rounded-[28px] bg-gradient-to-r from-[#6EE7B7]/70 via-[#3B82F6] to-[#9333EA]/50 p-1 md:rounded-none md:py-1 md:px-0">
<div className="rounded-[45px] bg-white px-6 py-4 md:rounded-none"> <div className="rounded-[25px] bg-white px-6 py-6 md:rounded-none">
<img src="/lock.svg" alt="lock" className="h-[24px] w-[24px]" /> <img src="/lock.svg" alt="lock" className="h-7 w-7" />
<h2 className="mt-2 mb-3 text-lg font-bold">Secure Data Storage</h2> <h2 className="mt-2 mb-5 text-base font-normal leading-6">
<p className=" w-[250px] text-xs text-gray-500"> Secure Data Storage
</h2>
<p className=" w-[300px] text-sm font-normal leading-6 text-black-1000">
The security of your data is our top priority. DocsGPT ensures the The security of your data is our top priority. DocsGPT ensures the
utmost protection for your sensitive information. With secure data utmost protection for your sensitive information. With secure data
storage and privacy measures in place, you can trust that your storage and privacy measures in place, you can trust that your
@@ -47,15 +47,17 @@ export default function Hero({ className = '' }: { className?: string }) {
</p> </p>
</div> </div>
</div> </div>
<div className=" rounded-[50px] bg-gradient-to-l from-[#6EE7B7]/80 via-[#3B82F6] to-[#9333EA]/50 p-1 md:rounded-tl-none md:rounded-bl-none"> <div className=" rounded-[28px] bg-gradient-to-l from-[#6EE7B7]/80 via-[#3B82F6] to-[#9333EA]/50 p-1 md:rounded-tl-none md:rounded-bl-none">
<div className="rounded-[45px] bg-white px-6 p-6 lg:rounded-tl-none lg:rounded-bl-none"> <div className="rounded-[25px] bg-white p-6 px-6 lg:rounded-tl-none lg:rounded-bl-none">
<img <img
src="/message-programming.svg" src="/message-programming.svg"
alt="lock" alt="lock"
className="h-[24px] w-[24px]" className="h-7 w-7"
/> />
<h2 className="mt-2 mb-3 text-lg font-bold">Open Source Code</h2> <h2 className="mt-2 mb-5 text-base font-normal leading-6">
<p className=" w-[250px] text-xs text-gray-500"> Open Source Code
</h2>
<p className=" w-[300px] text-sm font-normal leading-6 text-black-1000">
DocsGPT is built on open source principles, promoting transparency DocsGPT is built on open source principles, promoting transparency
and collaboration. The source code is freely available, enabling and collaboration. The source code is freely available, enabling
developers to contribute, enhance, and customize the app to meet developers to contribute, enhance, and customize the app to meet

5
frontend/src/Navigation.tsx
View File

@@ -1,5 +1,5 @@
import { useEffect, useRef, useState } from 'react'; import { useEffect, useRef, useState } from 'react';
import { NavLink } from 'react-router-dom'; import { NavLink, useNavigate } from 'react-router-dom';
import Arrow1 from './assets/arrow.svg'; import Arrow1 from './assets/arrow.svg';
import Arrow2 from './assets/dropdown-arrow.svg'; import Arrow2 from './assets/dropdown-arrow.svg';
import Exit from './assets/exit.svg'; import Exit from './assets/exit.svg';
@@ -60,6 +60,8 @@ export default function Navigation() {
const embeddingsName = const embeddingsName =
import.meta.env.VITE_EMBEDDINGS_NAME || 'openai_text-embedding-ada-002'; import.meta.env.VITE_EMBEDDINGS_NAME || 'openai_text-embedding-ada-002';
const navigate = useNavigate();
useEffect(() => { useEffect(() => {
if (!conversations) { if (!conversations) {
getConversations() getConversations()
@@ -111,6 +113,7 @@ export default function Navigation() {
}) })
.then((response) => response.json()) .then((response) => response.json())
.then((data) => { .then((data) => {
navigate('/');
dispatch(setConversation(data)); dispatch(setConversation(data));
dispatch( dispatch(
updateConversationId({ updateConversationId({

4
frontend/src/conversation/ConversationBubble.tsx
View File

@@ -42,7 +42,7 @@ const ConversationBubble = forwardRef<
bubble = ( bubble = (
<div ref={ref} className={`flex flex-row-reverse self-end ${className}`}> <div ref={ref} className={`flex flex-row-reverse self-end ${className}`}>
<Avatar className="mt-2 text-2xl" avatar="🧑‍💻"></Avatar> <Avatar className="mt-2 text-2xl" avatar="🧑‍💻"></Avatar>
<div className="mr-2 ml-10 flex items-center rounded-3xl bg-blue-1000 p-3.5 text-white"> <div className="mr-2 ml-10 flex items-center rounded-3xl bg-purple-30 p-3.5 text-white">
<ReactMarkdown className="whitespace-pre-wrap break-all"> <ReactMarkdown className="whitespace-pre-wrap break-all">
{message} {message}
</ReactMarkdown> </ReactMarkdown>
@@ -148,7 +148,7 @@ const ConversationBubble = forwardRef<
<Like <Like
className={`cursor-pointer ${ className={`cursor-pointer ${
feedback === 'LIKE' feedback === 'LIKE'
? 'fill-blue-1000 stroke-blue-1000' ? 'fill-purple-30 stroke-purple-30'
: 'fill-none stroke-gray-4000 hover:fill-gray-4000' : 'fill-none stroke-gray-4000 hover:fill-gray-4000'
}`} }`}
onClick={() => handleFeedback?.('LIKE')} onClick={() => handleFeedback?.('LIKE')}

18
frontend/src/conversation/conversationSlice.ts
View File

@@ -142,14 +142,14 @@ export const conversationSlice = createSlice({
state, state,
action: PayloadAction<{ index: number; query: Partial<Query> }>, action: PayloadAction<{ index: number; query: Partial<Query> }>,
) { ) {
const index = action.payload.index; const { index, query } = action.payload;
if (action.payload.query.response) { if (query.response) {
state.queries[index].response = state.queries[index].response =
(state.queries[index].response || '') + action.payload.query.response; (state.queries[index].response || '') + query.response;
} else { } else {
state.queries[index] = { state.queries[index] = {
...state.queries[index], ...state.queries[index],
...action.payload.query, ...query,
}; };
} }
}, },
@@ -163,21 +163,21 @@ export const conversationSlice = createSlice({
state, state,
action: PayloadAction<{ index: number; query: Partial<Query> }>, action: PayloadAction<{ index: number; query: Partial<Query> }>,
) { ) {
const index = action.payload.index; const { index, query } = action.payload;
if (!state.queries[index].sources) { if (!state.queries[index].sources) {
state.queries[index].sources = [action.payload.query.sources![0]]; state.queries[index].sources = [query.sources![0]];
} else { } else {
state.queries[index].sources!.push(action.payload.query.sources![0]); state.queries[index].sources!.push(query.sources![0]);
} }
}, },
updateQuery( updateQuery(
state, state,
action: PayloadAction<{ index: number; query: Partial<Query> }>, action: PayloadAction<{ index: number; query: Partial<Query> }>,
) { ) {
const index = action.payload.index; const { index, query } = action.payload;
state.queries[index] = { state.queries[index] = {
...state.queries[index], ...state.queries[index],
...action.payload.query, ...query,
}; };
}, },
setStatus(state, action: PayloadAction<Status>) { setStatus(state, action: PayloadAction<Status>) {

10
frontend/src/upload/Upload.tsx
View File

@@ -41,9 +41,9 @@ export default function Upload({
<p className="mt-10 text-2xl">{progress?.percentage || 0}%</p> <p className="mt-10 text-2xl">{progress?.percentage || 0}%</p>
<div className="mb-10 w-[50%]"> <div className="mb-10 w-[50%]">
<div className="h-1 w-[100%] bg-blue-4000"></div> <div className="h-1 w-[100%] bg-purple-30"></div>
<div <div
className={`relative bottom-1 h-1 bg-blue-5000 transition-all`} className={`relative bottom-1 h-1 bg-purple-30 transition-all`}
style={{ width: `${progress?.percentage || 0}%` }} style={{ width: `${progress?.percentage || 0}%` }}
></div> ></div>
</div> </div>
@@ -55,7 +55,7 @@ export default function Upload({
setProgress(undefined); setProgress(undefined);
setModalState('INACTIVE'); setModalState('INACTIVE');
}} }}
className={`rounded-3xl bg-blue-3000 px-4 py-2 text-sm font-medium text-white ${ className={`rounded-3xl bg-purple-30 px-4 py-2 text-sm font-medium text-white ${
isCancellable ? '' : 'hidden' isCancellable ? '' : 'hidden'
}`} }`}
> >
@@ -189,7 +189,7 @@ export default function Upload({
<span className="bg-white px-2 text-xs text-gray-4000">Name</span> <span className="bg-white px-2 text-xs text-gray-4000">Name</span>
</div> </div>
<div {...getRootProps()}> <div {...getRootProps()}>
<span className="rounded-3xl border border-blue-2000 px-4 py-2 font-medium text-blue-2000 hover:cursor-pointer"> <span className="rounded-3xl border border-purple-30 px-4 py-2 font-medium text-purple-30 hover:cursor-pointer">
<input type="button" {...getInputProps()} /> <input type="button" {...getInputProps()} />
Choose Files Choose Files
</span> </span>
@@ -206,7 +206,7 @@ export default function Upload({
<div className="flex flex-row-reverse"> <div className="flex flex-row-reverse">
<button <button
onClick={uploadFile} onClick={uploadFile}
className="ml-6 rounded-3xl bg-blue-3000 py-2 px-6 text-white" className="ml-6 rounded-3xl bg-purple-30 py-2 px-6 text-white"
> >
Train Train
</button> </button>

1
frontend/tailwind.config.cjs
View File

@@ -24,6 +24,7 @@ module.exports = {
'blue-1000': '#7D54D1', 'blue-1000': '#7D54D1',
'blue-2000': '#002B49', 'blue-2000': '#002B49',
'blue-3000': '#4B02E2', 'blue-3000': '#4B02E2',
'purple-30': '#7D54D1',
'blue-4000': 'rgba(0, 125, 255, 0.36)', 'blue-4000': 'rgba(0, 125, 255, 0.36)',
'blue-5000': 'rgba(0, 125, 255)', 'blue-5000': 'rgba(0, 125, 255)',
}, },

2
scripts/parser/file/html_parser.py
View File

@@ -57,7 +57,7 @@ class HTMLParser(BaseParser):
title_indexes = [i for i, isd_el in enumerate(isd) if isd_el['type'] == 'Title'] title_indexes = [i for i, isd_el in enumerate(isd) if isd_el['type'] == 'Title']
# Creating 'Chunks' - List of lists of strings # Creating 'Chunks' - List of lists of strings
# each list starting with with isd_el['type'] = 'Title' and all the data till the next 'Title' # each list starting with isd_el['type'] = 'Title' and all the data till the next 'Title'
# Each Chunk can be thought of as an individual set of data, which can be sent to the model # Each Chunk can be thought of as an individual set of data, which can be sent to the model
# Where Each Title is grouped together with the data under it # Where Each Title is grouped together with the data under it