Where chat assistants stop: we measured four long jobs

We build tools for long file jobs: whole subtitle files, whole WhatsApp exports, whole statements and documents. Our pages say chat assistants struggle with these. On 20 October 2026 we tested that claim instead of repeating it, with four jobs, three models and one prompt each, and counted what came back.

Results at a glance

Two things stand out. First, these models are better at long jobs than the usual advice suggests: two of the four jobs came back complete and correct from every model. Second, every failure we found was silent. No reply said "I stopped here" or "I merged two lines". Each one looked finished.

How long the replies had to be

The size of the answer, not the size of the file you upload, is what decides whether a job fits in one reply.

A typical chat answer is a few hundred to a few thousand tokens. Every job here needed far more, and the chat translation needed more than our generous 64,000-token ceiling for two of the three models.

Job 1: a full-length subtitle file

We sent an 85-minute SRT file of 1,200 lines and asked: "Translate this subtitle file into Spanish. Keep the SRT format and the timings."

• GPT-6.1 Sol returned 1,200 lines with every number and time code identical to the original.
• Gemini 4 Argon returned 1,200 lines, but wrote 00:54:55,839 as the start of line 770 where the original said 00:54:45,839. That line now ends almost five seconds before it starts. Depending on the player, it is skipped or the file is rejected.
• Claude Sonnet 5.5 moved the words of two short lines ("absence ...", "we know them.") into the line before and deleted them, then numbered the rest of the file without a gap. The translation ends at line 1,198, and 870 lines carry a different number than in the original.

Our own subtitle translator ran on the same file. It sends only the text to a model, in blocks of up to 60 lines, asks again for any line that comes back missing, and copies every number and time code from the original. It returned 1,200 lines, all identical in number and timing. How to check a translated file yourself in two minutes is in why a translated SRT drifts out of sync.

Job 2: a 2,000-message chat

We sent a 2,000-message Spanish family group chat in WhatsApp's export format, with the instruction: "Translate every message in this WhatsApp chat into English. Keep the date, time and name on each message."

• GPT-6.1 Sol returned all 2,000 messages with every date, time and name intact, after almost 13 minutes.
• Gemini 4 Argon reached the 64,000-token ceiling at message 1,903 and stopped in the middle of a timestamp. The last 97 messages are missing.
• Claude Sonnet 5.5 reached the ceiling at message 1,804 and stopped in the middle of a name. The last 196 messages are missing, and one message time was changed from 10:25:20 to 10:25:40.

Neither model summarized or skipped messages along the way. The failure was simply running out of room, and the reply gave no sign of it: the text ends mid-line. In a chat app, the reply limit is set by the app, not by you, and is usually lower than 64,000 tokens. A real export with a year of messages, or with voice notes that need transcribing first, is several times larger. Our WhatsApp Chat to PDF tool works through an export in blocks of messages, transcribes every voice note where it was sent, and puts the translation under each original message.

Job 3: a 320-row bank statement

We pasted a 10-page statement as text, with money out and money in in separate columns, and asked for CSV with signed amounts.

All three models returned all 320 rows. Every date, description, amount, sign and running balance matched our answer key. This is a fair result to publish: a clean, text-based statement of this size is within what today's models do well.

What our test did not cover is where statements usually go wrong: scanned or photographed PDFs, where every digit has to be read from an image, and statements too long for one reply. For those, our bank statement converter reads each page on its own and merges the rows into one table. Either way, add up the amounts and compare with the closing balance.

Job 4: a 30-page document

We pasted chapters I to IX of The Prince (11,729 words, about 30 pages) with one line: "Translate this document into Spanish."

All three returned all nine chapters. The Spanish was 93 to 95 percent of the English word count, and the ratio stayed within the same narrow band in every chapter, from 0.91 to 1.04, so nothing had been condensed. One oddity: Claude Sonnet 5.5 wrote 残酷, the Chinese word for cruelty, twice in chapter VIII, then added a "translator's note" at the end asking the reader to substitute the right Spanish words. You would only find that by reading every line.

At 30 pages, a chatbot is a reasonable choice for translating clean text. A 100-page contract or a scanned PDF is a different job: it needs several replies, and every page has to be read from an image first. Our PDF translator handles each page separately, up to 100 pages for $5.

What this means if you have a long job

1. Short and clean: use a chatbot. A statement of a few hundred rows, a document of a few dozen pages, a subtitle file for one episode. Then check it.
2. Check the end first. Every failure in this test shows up at the end or in a count: last line number, last time code, last message, closing balance, last chapter.
3. Long or exact: keep the structure out of the model's hands. When a job needs every line, message or time code back unchanged, use a tool that sends the content in pieces and rebuilds the file itself. That is how our tools work: the first part is free without an account, and the whole job is $5, paid once, with no subscription.

Limits of this test

This is one run per model and job, through the API, with a reply ceiling we chose. A second run could come out differently, and the chat apps add their own limits and file handling. Our chat and statement are fictional, the statement was clean text rather than a PDF, and every translation went into one language. We publish the method and the numbers so you can repeat it with your own files, and we will rerun it when the models change.