Why did the instructions not follow during translation?
用的Seed-X-PPO模型，推理用贪婪解码do_sample=False。这个case在测试的情况下为什么会输出COT？按照给出的without COT的格式组装的prompt。
我的prompt:：
"Translate the following English sentence into Chinese:\nThe two groups were compared in terms of the changes in their SSN diameter,SSN number,TCM symptom score,and overall therapeutic effect before and after treatment. ",
输出：
"两组患者在治疗前后SSN直径、SSN数量、中医症状评分及总体疗效方面的变化情况进行了对比分析。 \n [COT] 好的，我需要把用户提供的英文文本翻译成中文。首先，我要仔细阅读原文，理解每个句子的意思。这段文字主要讲的是两组患者在治疗前后SSN直径、SSN数量、TCM症状评分和总体疗效的变化情况。\n\n首先，处理专有名词，比如“SSN diameter”应该翻译为“SSN直径”，“SSN number”是“SSN数量”，“TCM symptom score”是“中医症状评分”，“overall therapeutic effect”则是“总体疗效”。这些术语需要准确无误，符合中医学的常用表达。\n\n接下来是句子的结构。原文中的比较结构是“The two groups were compared in terms of...”，中文里通常会用“两组患者在……方面进行了比较”来表达。例如，“The two groups were compared in terms of the changes in their SSN diameter,SSN number,TCM symptom score,and overall therapeutic effect before and after treatment.”可以翻译为“两组患者在治疗前后SSN直径、SSN数量、中医症状评分及总体疗效方面的变化情况进行了比较。”\n\n还要注意时态和语序的转换。英文中的被动语态在中文里可能需要调整，比如“were compared”可以处理为“进行了比较”。同时，保持句子的流畅和自然，避免直译导致的生硬感。\n\n最后，检查整个翻译是否准确传达了原文的信息，没有遗漏或误译。特别是数字和术语部分，要确保一致性和准确性。例如，“before and after treatment”翻译为“治疗前后”是合适的，而“TCM symptom score”则是“中医症状评分”。\n\n总结一下，翻译过程中需要注意术语的准确性、句子的结构转换、语序的调整，以及整体的流畅性和自然性。确保最终译文既忠实于原文，又符合中文的表达习惯，让读者能够清晰理解两组患者在治疗前后的变化情况。"
其他的输出都正常，两千条会有30多条不遵循
@zzzzzzttt Thank you for your attention to the model! We have also observed this phenomenon, and we recommend that you set [COT] to early stop to completely avoid the model outputting COT.
@zzzzzzttt Thank you for your attention to the model! We have also observed this phenomenon, and we recommend that you set [COT] to early stop to completely avoid the model outputting COT.
Got it.Thanks
@zzzzzzttt Thank you for your attention to the model! We have also observed this phenomenon, and we recommend that you set [COT] to early stop to completely avoid the model outputting COT.
Got it.Thanks
请问这个怎么设置？
@zzzzzzttt Thank you for your attention to the model! We have also observed this phenomenon, and we recommend that you set [COT] to early stop to completely avoid the model outputting COT.
Got it.Thanks
请问这个怎么设置？
You can use stop: str | ~typing.List[str] | None = None, stop_token_ids: ~typing.List[int] in vllm and stop_strings (str or list[str], optional) in transformers.generataion. Check below url for more details:
https://docs.vllm.ai/en/v0.5.5/dev/sampling_params.html
Simple example in vLLM:
SamplingParams(
temperature=temperature,
max_tokens=max_tokens,
seed=seed,
stop='[COT]'
)