LLM/airllm
1
0
Fork 0
mirror of https://github.com/0xSojalSec/airllm.git synced 2026-03-07 22:33:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Gavin Li
2023-09-14 21:29:06 -05:00
committed by GitHub
parent 57c2b6eee3
commit 2c4eec6aa8
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
anima_100k/README.md
View File

@@ -184,7 +184,8 @@ inputs['attention_mask'] = inputs['attention_mask'].cuda()
# Generate
generate_ids = model.generate(**inputs, max_new_tokens=30,
only_last_logit=True,
only_last_logit=True, # to save memory
use_cache=False, # when run into OOM, enable this can save memory
xentropy=True)
output = tokenizer.batch_decode(generate_ids,
skip_special_tokens=True,