NTT Tunes an LLM on Small Manuals: 83.1% Accuracy Reported

Japan's NTT announced on 12 November 2026 a new tuning technique for getting the knowledge in a company's business manuals into a large language model (LLM). According to NTT's press release, the method has the LLM itself generate training data from many angles, so a specialised model can be built without collecting vast amounts of documents. AI Watch reports that in tests on NTT's own model tsuzumi 2 the share of correct answers reached 83.1%.

Why does conventional tuning struggle with company data?

Tuning means adjusting an existing LLM to a particular use or task. According to INTERNET Watch, making a model absorb knowledge about business operations has required extremely large amounts of data, far more than the documents any single company owns. NTT's press release notes that a page of A4 text is a few kilobytes, while the usual approach of retraining the whole model uses data running to tens of gigabytes.

AI Watch says a company typically holds about 1,000 PDF files, up to roughly 100 pages each. Simply repeating such a small body of text in training leads, the outlet explains, to overfitting: the model memorises exact wording or article numbers but fails when a question is asked in a different context. NTT adds that filling the gap with similar material from the internet lets in noise unrelated to the business and makes a high-quality dataset hard to assemble.

How does the LLM write its own study material?

NTT drew on how people learn at school. According to the press release, in a language class students do more than read a text: they answer questions about the author's intent and discuss their impressions with classmates. The technique applies the same idea. The LLM acts as a creator of learning material, producing varied training data from the manual and then learning from it.

AI Watch lists three forms of generated data:

• an FAQ format, which turns the manual's contents into questions and answers modelled on real enquiries;
• a table format, which organises and structures several conditions or periods;
• a dialogue format, in which a student and a teacher dig into the background and intent of a rule.

Also according to AI Watch, the generation engine itself uses tsuzumi: from a small batch of source data it produces training data on the order of billions to ten billion tokens (a token is the unit of text an LLM processes). NTT states the technique can be applied to tsuzumi and to other LLMs.

How far can the 83.1% figure be trusted?

NTT measured retention using real business manuals: questions covering the manual's content were put to the model and the share of correct answers compared. Per the press release, tsuzumi 2 scored 65.3% before learning; earlier research, which rephrases texts from other angles, reached at most 73.3%; the new technique reached 83.1%, close to ten points above that prior work.

A lighter model was tested too. According to AI Watch, Llama 3 (3B) rose from about 25% to the high 50s and consistently beat earlier methods. The same outlet says general-domain benchmarks showed no loss of the model's basic abilities after the knowledge was added. One caveat applies: the multiple-choice questions were designed by NTT itself, because, AI Watch explains, no common benchmark exists for measuring a single company's knowledge. These are NTT's own figures, and nothing in the articles read indicates independent replication.

RAG or a specialised model: which one to use?

For exploiting internal documents, retrieval-augmented generation (RAG), which looks up relevant passages before answering, is widely used. AI Watch reports the view of an NTT representative: where absolute accuracy matters most, RAG is recommended, but for work needing very fast responses, such as analysing large volumes of calls in real time at a contact centre, external search adds latency and raises costs because more models and GPUs must run in parallel. A model that already holds the knowledge answers faster and needs fewer GPUs.

According to the same source, the technique does not rule out RAG. The two can be combined, and a model that knows industry and internal background also improves retrieval quality. Target fields include healthcare and finance, where Japan-specific rules and procedures are strict, and manufacturing, with its many in-house terms.

What is the timeline, and what changes for a business outside Japan?

Per the press release, the technique is to be introduced across NTT Group companies. AI Watch says NTT is studying, among other options, a managed cloud service in which GPU-heavy training is run in NTT Group data centres and the tuned model is returned to the customer. The articles read give no price or launch date. The results will be shown at NTT R&D FORUM 2026, held 18 to 25 November at NTT's Musashino R&D centre in Tokyo, where, per AI Watch, a "self-evolving tsuzumi" concept will also be unveiled.

For a company outside Japan, the lesson is mainly about method. If its procedures fit in around a thousand documents, having the model itself write question-and-answer sets, tables and dialogues may make a specialised model feasible where lack of data once discouraged the attempt. The articles read mention no availability outside Japan, and the results remain NTT's own: the prudent step is to measure accuracy on your own documents and your own questions first, then decide how to split work between RAG and a tuned model.