hamodywe/unfinished

unfinished

A completion is a value plus a reason the model stopped. Your code reads the first one.

---

The problem

const res = await openai.chat.completions.create({ max_tokens: 512, ... });
const data = JSON.parse(res.choices[0].message.content);

When the answer reaches the token limit, the API returns 200, the content is cut off mid-object, and finishreason is "length".

JSON.parse throws. The catch around it decides the model returned bad JSON and retries. The retry sends the same prompt into the same limit and fails the same way. After three attempts the code gives up with "the model kept returning invalid JSON" — and every one of those calls was billed.

Nothing in that story is an error anyone can see. The HTTP status was fine, the SDK did not throw, the model behaved exactly as documented. The only record that the answer was incomplete is the field nobody read.

When the output is prose there is not even an exception. The last sentence just stops. It gets stored, indexed, shown to a user, or fed into the next step of a chain as though it were the whole answer.

$ npx unfinished .

4 model calls · 0 check why the model stopped · 4 would not notice a truncated answer
                                 0% of calls would notice

  3 source files read

error   json-parsed-without-finish-check src/extract.ts:14
  1 response is parsed as JSON without checking why the model stopped
  fix: Check the finish reason before parsing. `finish_reason === "length"`
       (OpenAI), `stop_reason === "max_tokens"` (Anthropic) or
       `finishReason === "length"` (AI SDK) means the answer is incomplete and

src/extract.ts:14 — .chat.completions.create() — openai

warning stream-finish-reason-ignored src/classify.ts:18 1 stream is consumed without reading the finish reason src/classify.ts:18 — streamText() — ai-sdk

## Why existing tools miss it

| Tool | Why this is invisible to it |
|---|---|
| TypeScript | `finish_reason` is an optional field on a type. Not reading a field is not a type error. |
| ESLint | No rule models "this value must be consulted before that one". |
| Evals / prompt tests | They run against inputs that fit. Truncation is a function of *input size*, so it appears in production on the largest document and never in the test set. |
| Observability | Every one of these calls is a success. There is no error to alert on, and the retry loop looks like normal retry behaviour. |

The providers document the fix — OpenAI's JSON-mode guidance says to check `finish_reason` before parsing — and application code almost never does, because nothing ever forces the issue.

npx unfinished .                  # no install
npm install --save-dev unfinished

Node 20.10 or newer. Zero runtime dependencies. No model is called, no API key is needed, no network.

Quick start

unfinished .                 # scan the repository
unfinished src/ai            # or one directory
unfinished src/chat.ts       # or one file
unfinished . --verbose       # list every model call found
unfinished . --json          # machine-readable

Exit code 1 when something at or above the threshold is found, 0 when clean, 2 when the tool could not run.