Spaces:

sunwaee
/

MT5-Questions-Answers-Generation-Extraction

Runtime error

App Files Files Community

MT5-Questions-Answers-Generation-Extraction / tokenizer /special_tokens_map.json

sunwaee's picture

added tokenizer (+ <hl>/<sep> tokens)

3143cf6 almost 5 years ago

Raw History Blame Contribute Delete

65 Bytes

{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>"}