Text Generation
Transformers
TensorBoard
Safetensors
Japanese
English
qwen2
conversational
text-generation-inference
leonardlin commited on
Commit
00ed98f
·
verified ·
1 Parent(s): 9f31c61

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -93,7 +93,7 @@ We believe these benchmarks will be generally useful and plan to open-source the
93
 
94
  All Shisa V2 models inherit the [chat templates](https://huggingface.co/docs/transformers/v4.37.1/chat_templating) of their respective base models and have been tested and validated for proper inference with both [vLLM](https://github.com/vllm-project/vllm) and [SGLang](https://github.com/sgl-project/sglang).
95
 
96
- We recommend running at lower temperatures: 0.0-0.2 for factual answers and 0.5-0.7 for creative tasks.
97
 
98
  No additional safety alignment has been done on these models, so they will largely inherit the base models' biases and safety profiles.
99
 
 
93
 
94
  All Shisa V2 models inherit the [chat templates](https://huggingface.co/docs/transformers/v4.37.1/chat_templating) of their respective base models and have been tested and validated for proper inference with both [vLLM](https://github.com/vllm-project/vllm) and [SGLang](https://github.com/sgl-project/sglang).
95
 
96
+ Running sampler sweeps, we found the models operate well across a variety of temperatures in most settings. For translation tasks specifically, we recommend a lower temperatures (0.2) to increase accuracy. For role-play and creative tasks, a higher temp (eg 1.0) seems to give good results. To prevent cross-lingual token leakage we recommend a top_p of 0.9 or min_p of 0.1.
97
 
98
  No additional safety alignment has been done on these models, so they will largely inherit the base models' biases and safety profiles.
99