LLM/DocsGPT
1
0
Fork 0
mirror of https://github.com/arc53/DocsGPT.git synced 2025-12-03 18:43:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #463 from jbampton/remove-duplicate-words

Browse Source 
Remove unneeded duplicate words
This commit is contained in:
Alex
2023-10-07 14:12:00 +01:00
committed by GitHub
parent 25ec8fb2ab 32ea0213f7
commit ffb7ad1417
7 changed files with 7 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
application/parser/file/html_parser.py
View File

@@ -57,7 +57,7 @@ class HTMLParser(BaseParser):
title_indexes = [i for i, isd_el in enumerate(isd) if isd_el['type'] == 'Title']
# Creating 'Chunks' - List of lists of strings
# each list starting with with isd_el['type'] = 'Title' and all the data till the next 'Title'
# each list starting with isd_el['type'] = 'Title' and all the data till the next 'Title'
# Each Chunk can be thought of as an individual set of data, which can be sent to the model
# Where Each Title is grouped together with the data under it