LLM/airllm
1
0
Fork 0
mirror of https://github.com/0xSojalSec/airllm.git synced 2026-03-07 14:24:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

refine readme, add missing data file

Browse Source
This commit is contained in:
Yu Li
2023-09-15 21:00:38 -05:00
parent b52125ed30
commit 96262af7cc
2 changed files with 5 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
anima_100k/120_topics_en.jsonl Normal file
View File

File diff suppressed because one or more lines are too long

5
anima_100k/README.md
View File

@@ -167,6 +167,9 @@ pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory
pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/xentropy
pip install accelerate
pip install bitsandbytes
pip install evaluate
pip install git+https://github.com/huggingface/peft.git@v0.4.0
pip install wandb
```
#### 推理
@@ -195,7 +198,7 @@ inputs['attention_mask'] = inputs['attention_mask'].cuda()
generate_ids = model.generate(**inputs, max_new_tokens=30,
only_last_logit=True, # to save memory
use_cache=False, # when run into OOM, enable this can save memory
xentropy=True)
)
output = tokenizer.batch_decode(generate_ids,
skip_special_tokens=True,
clean_up_tokenization_spaces=False)[0]