Tokens and words
Convert a word count into an estimated token count, or a token budget into the words it will hold, at a ratio you can set for your own text.
Did this calculator give you what you needed?
What could be better?
The ratio is an estimate, and it moves
English prose sits near 1.33 tokens per word, which is the default here. That figure is an average over ordinary text, and your text may not be ordinary. Code, JSON and tables tokenize far more heavily because punctuation and indentation each cost tokens. Languages that do not use spaces, and languages written in scripts the tokenizer saw less of during training, can cost several times more tokens for the same meaning.
The ratio is a field rather than a constant for exactly that reason. Measure a representative sample of your own text with your provider’s tokenizer once, put the ratio you observe here, and the estimate becomes useful for planning instead of merely plausible.
What to use it for
Sizing a budget before you build, checking whether a document class will fit a window, and sanity-checking a bill are all reasonable. Predicting an exact charge is not: for that, use the token counts the API returns, which are the numbers you are actually billed on.
Frequently asked questions
Why is a token not a word?
A tokenizer splits text into pieces that are frequent in its training data. Common short words are usually one token, longer or rarer words break into several, and a leading space is often part of the token. That is why 1,000 words is roughly 1,330 tokens rather than 1,000.
Does the ratio differ between providers?
Yes. Each family uses its own tokenizer, so the same paragraph can be a different number of tokens on two providers. If you are comparing prices between them, compare the cost of the same text, not the cost per token.
How do I find my own ratio?
Send a representative sample through the API and divide the returned input-token count by the sample’s word count. One measurement on real text beats any published average, especially if your text is code or a language other than English.