WikiGPT-25M / tokenizer_config.json
Ma7ee7's picture
Upload character-level GPT model for WikiText-2
5bef0ce verified
{
"model_max_length": 512,
"tokenizer_class": "CustomCharTokenizer",
"vocab_size": 283,
"chars": "\n !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^`abcdefghijklmnopqrstuvwxyz|~¡£¥§°±²³µ·½ÁÅÆÉÍÎÖרÚÜÞàáâãäåçèéêëìíîñòóôöøúûüĀāăćčĐđėīŁłńŌōśşšūųŻžơưʻʿ̃αβγκμСавекостяاحصلنه्กงณตมยรลัาิ่์გდვზიკორსუცძწხჯ჻ḥṃṅṣṭṯảấầắễệịớửỳ‑–—‘’“”„†…′″⁄₤€₹⅓⅔→−≤☉♭♯〈〉のァアキスットプュリルヴ・動場大戦攻機殻火礮空隊~"
}