Jina AI launches jina-ocr-v1, a low‑cost GPU document parser with MoE and speculative decoding

Jina AI, now integrated with Elastic, has announced the launch of jina-ocr-v1, a document‑parsing model that can turn PDFs, scanned images, tables, charts and invoices into neatly structured Markdown, ready for downstream search and analytics pipelines.

The new model packs 3.4 billion parameters, but thanks to its mixture‑of‑experts (MoE) design it only activates roughly 570 million parameters for each token processed, a strategy that dramatically lowers inference costs on inexpensive GPUs.

Efficient visual encoding

Jina‑ocr‑v1’s visual encoder compresses a full‑page image of 1 024 × 1 024 pixels into just 256 visual tokens. When users enable the dynamic mode, the system can add up to nine local tiles, pushing the token count to a ceiling of 1 156 and preserving fine‑grained layout information without blowing up memory requirements.

Speculative decoding with FastMTP

A novel speculative decoding head called FastMTP looks three tokens ahead and then greedily validates the predictions. This technique guarantees that the final output matches what a standard decoder would produce, while cutting the number of decoding steps roughly in half.

Benchmark results on the OmniDocBench v1.6 suite give jina‑ocr‑v1 a score of 91.14, and the model reaches 83.4 on the olmOCR‑Bench. Although these figures do not place it at the very top of pure quality rankings, the model’s main advantage lies in its throughput‑per‑dollar performance.

Running on an Nvidia A100 40 GB GPU, the developers recorded a processing speed of 2.57 pages per second at a cost of $32 per hour. This was the highest throughput among the fourteen systems evaluated in their internal study, showcasing the model’s efficiency even on high‑end hardware.

On a single Nvidia L4 GPU, FastMTP boosts the token generation rate from 42.7 to 83.1 tokens per second in eager mode, a reported 1.95× speed‑up. The result demonstrates that low‑budget GPUs can approach real‑time document parsing when paired with the right decoding strategy.

Model size and licensing

The model’s BF16 weights occupy about 6.8 GB and are hosted on Hugging Face under a Creative Commons BY‑NC 4.0 license. The license allows free academic and research usage, while commercial deployments require a separate agreement with Jina AI.

• 3.4 billion total parameters
• ≈570 million active parameters per token
• 256‑to‑1 156 visual tokens per page
• FastMTP speculative decoding with 3‑token look‑ahead

According to the official model sheet, jina‑ocr‑v1 supports 108 languages. The developers note, however, that heavily degraded scans remain a challenge and they advise a human verification step for mission‑critical documents.

What this means for English‑speaking organisations

Companies that need to ingest large volumes of heterogeneous documents—legal contracts, invoices, research papers—can now rely on a cost‑effective solution that runs comfortably on mid‑range GPUs such as the Nvidia L4.

The combination of MoE‑driven parameter efficiency and FastMTP speculative decoding translates into higher throughput without sacrificing the fidelity of the extracted Markdown, enabling organizations to automate the first pass of document extraction at a fraction of typical cloud compute costs.

By offloading the bulk of parsing to jina‑ocr‑v1, businesses can reserve human reviewers for only the most problematic scans, reducing manual effort and accelerating downstream workflows.

The structured Markdown output can be fed directly into Elastic‑based search or analytics stacks, simplifying integration and allowing immediate indexing, enrichment and insight generation.

From a deployment perspective, the model’s modest 6.8 GB footprint means it fits comfortably on most modern GPU instances, and the CC BY‑NC 4.0 license eliminates upfront licensing fees for non‑commercial pilots.

Overall, jina‑ocr‑v1 positions itself as a pragmatic middle ground between heavyweight, high‑quality OCR solutions and lightweight, low‑cost alternatives, offering a compelling value proposition for enterprises seeking scalable document ingestion.

In summary, for organisations operating in English‑dominant markets, the new parser delivers a blend of multilingual coverage, GPU‑friendly efficiency and open‑source accessibility that can reshape document processing pipelines while keeping budgets in check.