Tags
Products tagged: "tokenization"
TokTok: A Spanish Tokenizer
Training a Spanish-only tokenizer and comparing it to GPT-4
VocabVacation: Does Vocab Size Matter?
Training tiny transformers with different vocab sizes to find the sweet spot
Training a Spanish-only tokenizer and comparing it to GPT-4
Training tiny transformers with different vocab sizes to find the sweet spot