turkish-code-detector / added_tokens.json
yeniguno's picture
Add tokenizer
854d3eb verified
{
"()": 32003,
"++": 32001,
"--": 32002,
"=>": 32000,
"[]": 32004
}