Direct Answer
Jev looks well suited to high-volume inbox triage when the job is to return bounded labels and scores, not write the reply. In Ryan Vogel's test, a 100-email batch reports about 200 milliseconds average latency, 240 ms at the 95th percentile, and 38 completed classifications per second with eight workers. A later 1,000-email run remains near 200 ms per message on the displayed dashboard.
The useful result is speed, but the video is not a complete accuracy or cost benchmark. Ryan manually inspects only a handful of messages, there is no labeled reference set, and the billing view belongs to a shared account. Treat the test as a promising engineering demonstration and a template for a controlled inbox pilot.
Access: Vercel has made Jev available through AI Gateway. The gateway model page lists current access and pricing, which should be checked again before estimating a production inbox workflow.
Watch the 1,000-Email Test
Credit: This article is based on Ryan Vogel's Jev email-classification test and the supplied transcript. Product behavior, pricing, and confidence guidance are cross-checked against TypeSafe and Vercel's official pages. The latency, throughput, and review observations remain creator-reported results from this environment.
What Was Actually Tested
Ryan exports roughly 1,500 of his own emails and asks Jev to answer four structured questions for each message:
- Category: what kind of message is this?
- Priority: how urgent or important does it appear?
- Spam: how likely is the message to be unwanted?
- Reply: how likely is it to need a response?
The first run processes 100 emails with eight workers. Ryan then expands the same setup to 1,000 messages. This is a good fit for Jev's design: the application supplies state plus typed questions, and the model returns choices or scores with probabilities. It is not generating prose, searching the inbox, or sending messages.
| Run | Workers | Average latency | Other visible result | Evidence level |
|---|---|---|---|---|
| 100 emails | 8 | About 200 ms | P95 240 ms; 38/second | Creator's dashboard |
| 1,000 emails | 8 | Stayed near 200 ms | Rapid parallel completion | Creator's dashboard |
How to Read the Latency and Throughput Numbers
Latency and throughput describe different things. The roughly 200 ms figure is the reported end-to-end time for an individual classification request. The 38-per-second figure is aggregate throughput while eight workers send requests concurrently. Those results can coexist because several messages are in flight at once.
The 240 ms P95 is also more useful than the average alone: it means the dashboard reports that 95% of the measured requests completed at or below that level. It still does not guarantee the same performance for every account, region, payload size, concurrency level, or future version. A production test should record P50, P95, and P99 latency, rate-limit responses, retries, and failures over a representative sample.
TypeSafe reports much larger speed and cost improvements on its own workflow benchmark. Its launch article also describes that benchmark as the higher end of likely gains and notes that its model-capabilities team created the workflows. The email test is useful independent evidence of fast responses in one application, but it is not a reproduction of the vendor's full benchmark.
The Video Checks Examples, Not Accuracy
Ryan opens several results and finds plausible high-priority or reply-needed messages, including account and domain warnings. That confirms that the output is inspectable and that some visible examples make sense. It does not establish the system's error rate.
A proper evaluation needs messages labeled by a person before the model runs. Then measure precision and recall for spam and reply decisions, confusion between categories, priority-ranking agreement, and performance on ambiguous threads. False positives matter differently for each action: mislabeling a newsletter is inconvenient, while hiding a security warning or sending an incorrect reply can be costly.
Why the Screen Does Not Prove a Five-Cent Run
During the test, Ryan initially says the usage is five cents. He then notices other team experiments in the dashboard and realizes that it is a shared account. That means the visible amount cannot be assigned cleanly to this 1,000-email run.
Vercel currently lists Jev at $0.04 per million input tokens, with no output-token charge shown. That published rate suggests classification can be inexpensive, but a defensible total needs the exact input-token count for this run and an isolated usage record. The on-screen 1.1 million-token counter is not explained well enough to treat it as the final bill.
Model tokens are only one part of the operating cost. A real inbox tool also needs secure email access, data storage and retention rules, monitoring, retry handling, reviewer time, and a downstream system that applies labels or creates tasks.
A Safer Way to Pilot Jev on Email
- Begin with historical mail: use a consented, representative sample and remove sensitive content that the test does not need.
- Write the labels first: define category, priority, spam, and reply rules with examples and edge cases.
- Create a human reference set: label a few hundred messages before running Jev, then keep that set unchanged for comparisons.
- Use scores to route: let high-confidence, low-risk results add a label; send uncertain messages to review. Do not auto-delete or auto-send during the pilot.
- Measure the complete system: record latency percentiles, throughput, failures, token usage, reviewer corrections, and end-to-end cost.
- Protect the inbox: confirm provider terms, access controls, retention, audit logs, and whether personal or regulated data may be sent to the service.
The best first deployment is a reversible queue: “likely reply needed,” “possible spam,” or “security/account alert.” That captures Jev's speed without allowing one incorrect probability to remove or answer an important message.
Video Chapters
| Time | Chapter | Time | Chapter |
|---|---|---|---|
| 00:00 | A new type of AI | 02:22 | Scaling to 1,000 emails |
| 00:54 | Testing Jev on email | 03:13 | Checking the cost |
| 01:17 | Speed and latency | 03:48 | More possibilities |
| 01:34 | Reviewing classifications |
Sources and Further Reading
- Ryan Vogel: Meet Jev, the AI Built to Make Decisions (video and supplied transcript)
- TypeSafe: Introducing System One Models and Jev
- TypeSafe: Jev introduction and typed questions; confidence guidance
- Vercel AI Gateway: Jev availability and current listed price; official availability announcement
YouTube lists the video's publication date as 15 September 2026. This article was reviewed on 19 September 2026. Access, pricing, and performance may change.