Jev Lowered One Workflow's API Cost. Caching Reversed the Result.

The cached comparison has the cheaper architecture in the first row: Luna alone, $1.76 per 10,000 classifications. Adding Jev ahead of Luna raised the modeled cost to $2.92. An earlier test had favored the Jev pipeline, so the answer depended on how the baseline handled repeated context.

For a support system sorting requests into queues, that difference changes which component earns its place. A decision model can avoid expensive calls. It can also add a call to work a cheap model already handles economically.

For this comparison, I used the saved results of AI-assisted experiments run under my direction on public benchmarks and synthetic tasks. These were offline experiments, with costs calculated from returned token usage and recorded prices. They did not measure a client's bill, staff time, or production reliability. The comparison table and chart below show the modeled costs, and each section states its limits.

Jev handles a decision inside the workflow

TypeSafe describes Jev as a System One model for focused semantic judgments. Its API returns a choice among named options, a score against a defined scale, or a yes/no probability. A support request can become a category that application code uses to select the next step. Jev doesn't need to write the reply. TypeSafe primitives.

That separation gives it several possible jobs: classify a request directly, check an answer another model produced, or select relevant evidence. Application code still owns execution and required approvals. A confident category is insufficient authority to refund a payment or change an account, the same boundary I discuss in per-tool permissions for AI agents.

The tested version was jev-1.13.0. TypeSafe's documentation, checked September 24, lists $0.042 per million input tokens and free output. That is the component price; the useful comparison includes every fallback and retry. TypeSafe models.

Selective classification passed a narrow test

The classification test asked whether Jev could answer some requests itself and send the rest to a general model. It used BANKING77, a public banking-intent dataset, with examples drawn from its training split. The comparison model was gpt-5.6-luna at low reasoning effort. BANKING77 source.

A confidence threshold of 1.0 was selected on an earlier 231-query sample, then frozen before a fresh 770-query confirmation. That confirmation used ten queries per category, excluding previously tested text and exact training-text matches. This matters because choosing the threshold after seeing the confirmation results would make the result easier to overstate.

Jev handled 409 queries directly, and all 409 matched the reference labels. With Luna handling the remainder, the pipeline matched 731 of 770 labels. Direct Luna matched 730. The complete pipeline had two losses and three wins against direct Luna; its separately executed fallback calls did not always repeat the direct model's answer.

Under the experiment's cold-request accounting, direct Luna cost $11.13 per 10,000 classifications and Jev plus Luna cost $7.18, a 35.5% reduction. Those amounts include the decision step and fallback calls. They exclude engineering and ongoing operation.

The test passed its predefined cost and paired-loss criteria. Still, 409 accepted answers without an observed error don't establish zero future risk. TypeSafe's confidence describes the distribution over the supplied options, and an earlier names-only test produced wrong decisions even at confidence 1.0. The threshold belongs to this task definition and model version. Confidence documentation.

A cached direct model won the next cost comparison

The cold comparison left another practical baseline to test. Category definitions repeat across requests, so a direct model can reuse cached context instead of charging every request as entirely new input. OpenAI documents exact-prefix caching and recommends placing stable content before variable content. Prompt caching.

The separate cache control reused the earlier 231 queries. It put a cache breakpoint after the stable category definitions, kept query-specific examples outside that prefix, gave the two arms separate cache keys, and included initial writes. This was an exploratory control on reused records, not another independent confirmation.

Classification regime	Direct Luna	Jev with Luna fallback	Reference matches
Cold normalization, 770 fresh queries	$11.13	$7.18	730 versus 731
Sustained prefix caching, 231 reused queries	$1.76	$2.92	213 versus 214

Modeled API dollars per 10,000 classifications. Compare arms within each row; the rows use different samples and cache regimes.

With prefix reuse, the Jev pipeline cost 65.8% more. The extra decision step had become more expensive than the cached model calls it avoided. That result gives me a concrete baseline requirement for an evaluation: optimize the direct path before deciding to add a router.

The accounting also needs inspection. The saved OpenAI calculations used a 1.25× cache-write multiplier alongside Luna rates of $0.20 input, $0.02 cached input, and $1.20 output per million tokens. OpenAI's model page confirms those rates as of September 24. The calculations remain estimates from returned usage, without invoice reconciliation; documented prices don't prove the bill a particular account received. Luna pricing.

Checking an answer has a different tradeoff

A second promising use was checking a cheap answer before paying for a stronger one. On 800 FinQA financial-question examples, the saved pipeline generated an answer with gemini-3.1-flash-lite at minimal thinking, checked it with Jev, and selected the saved gemini-3.1-pro-preview answer at low thinking when the check rejected it. This was a replay of component outputs, not a newly timed production pipeline. FinQA source.

Direct Pro and the Jev pipeline each matched 598 numerical references. Modeled cost fell from $52.44 to $31.21 per 10,000 requests, or 40.5%, using the experiment's recorded standard token rates including thinking tokens. But the pipeline introduced eight errors relative to Pro and recovered eight others. Equal totals conceal different failures.

It also accepted 79 wrong cheap answers. The reference audit found 102 test answers inconsistent with execution references; a posthoc analysis excluding those disagreements produced 564 matches out of 698 for the pipeline versus 565 for Pro. That sensitivity check is useful, but agreement between references doesn't certify the remaining answers.

The cheapest direct alternative remains relevant. Flash-Lite alone matched 581 of 800 references at $3.33 per 10,000. Adding checking and fallback bought 17 additional net matches for substantially more API cost. Whether that trade is useful depends on the consequences of the mistakes; the benchmark does not supply their business value.

The next test needs the operating workload

These results leave specific places to investigate Jev: repeated semantic classifications where accepted decisions avoid enough cost, and answer checks where selective escalation improves an outcome enough to justify the extra step. For stable repeated prompts, cached direct Luna won this cost control. A workflow whose decision is already an explicit rule should also include a deterministic baseline before adding inference.

I would start the next evaluation with saved requests from the intended workflow, running without taking actions. Freeze the acceptance policy, preserve individual disagreements, and compare the complete Jev path with the cheapest suitable direct path. Record missed errors and unnecessary escalations separately, because they create different work for the people handling the exceptions.

This study supplies no measured reviewer-time savings. Public datasets can have ambiguous labels or training contamination, and provider-credit interruptions required recovery during the program. The results support selecting a pilot, not promising its return.

Before adding a decision model, measure the direct workflow with its ordinary cost optimizations enabled. Then give the extra component a specific error or expense to remove.