File Token Calculator

A file token calculator is a browser-based tool that reads uploaded files locally, extracts their plain-text content, and estimates how many tokens the content would consume when sent to a large language model API. Unlike a simple text box token counter, a file token calculator online handles structured and binary-adjacent formats — PDFs, Markdown, JSON, HTML, source code, CSV, and YAML — applying format-specific text extraction logic and tighter chars-per-token ratios for non-prose formats. This lets developers and data teams get accurate pre-flight token budgets for document processing pipelines, RAG ingestion, and long-context model workflows without writing any code.

1. Upload Files: Drag and drop one or more files onto the drop zone, or click to browse. Supported formats include PDF, Markdown (.md / .mdx), JSON, HTML, XML, CSV, source code (.py, .js, .ts, .go, .rs, .java, .c, .cpp, .cs, .rb, .php), YAML, TOML, SQL, and plain text. Multiple files are processed simultaneously.
2. Text Extraction by Format: Each file type goes through format-specific processing. HTML and XML have tags stripped to isolate readable text. JSON is pretty-printed to normalize whitespace. PDFs have their BT/ET text streams parsed to extract embedded text. Source code files are read as-is since all characters contribute to tokenization. Binary formats like DOCX use XML text extraction.
3. Token Estimation with Format Multipliers: The raw character count is divided by a per-model chars-per-token value, then adjusted by a format multiplier — code files use ~0.85× (more symbols per token), HTML/XML use ~0.80× (tag inflation), and plain prose uses 1.0×. The result is a realistic token estimate for each file.
4. Cross-Model Cost Comparison:Select any model to see input token costs at that model's pricing rate. Switch models at any time — already-uploaded files instantly recalculate without re-reading from disk. The totals bar shows aggregate token count and cost across all files.

• Plain Text & Markdown: Highest accuracy. All characters are readable and tokenized directly. Markdown syntax (headers, bold, code fences) contributes slightly more tokens than clean prose but is fully captured.
• JSON & JSONL:High accuracy after pretty-print normalization. JSON's structural characters (quotes, brackets, colons, commas) increase token count relative to equivalent prose — the calculator accounts for this with a 0.90× multiplier.
• HTML & XML: Good accuracy after tag stripping. The extracted text closely reflects what a model would read from the content. The 0.80× chars-per-token multiplier corrects for the fact that raw HTML inflates character count with tags while reducing token density.
• PDF: Moderate accuracy for text-based PDFs. The calculator extracts text from uncompressed PDF text streams (BT/ET blocks). Scanned PDFs (image-only), password-protected PDFs, and PDFs with compressed streams may show lower-than-actual token counts. For precise PDF token counts, consider extracting text with a PDF library first.

The file token calculator processes every file entirely inside your browser using the Web File API and JavaScript. No file content, extracted text, or token estimates are ever transmitted to any server. Files are read from disk into memory, processed, and the raw binary data is released. This makes it safe to use with proprietary source code, confidential business documents, legal contracts, and any other sensitive files. The tool is completely free, requires no sign-up, and works offline once the page has loaded. There are no file size limits imposed by the tool — browser memory is the only practical constraint for very large files.