Tokenization Inequality

Before an AI model reads a single word you write, it chops your text into pieces called tokens. For English, the pieces are generous. A common word is often one token. For many other languages, the same sentence shatters into far more fragments.

This is not a cosmetic difference. More tokens means higher cost per message, less room in the context window, and often worse output quality. Two people asking the same question in different languages are not paying the same price or getting the same machine.

The unfairness is baked in before training even begins. Tokenizers are built from data that overrepresents English, so every other language inherits the leftovers.