How much AI does a focused task need?
I tried Jev, TypeSafe AI's model for fast, structured decisions, in two roles. It routed work between models in my Hermes workflow, and it evaluated Clara's answers.
Clara, the AI voice course counsellor I’m building, has a natural Aussie voice. I’m working on whether her responses live up to the expectations that voice creates. One idea from my Stanford Online course stayed with me: social cues create expectations automatically. A humanlike voice can encourage trust before the system has earned it.
That work has involved plenty of back and forth between Gemini Flash and more powerful, more expensive models such as Astra. I started wondering how much model capability each part of the work actually needed.
Then Jev from TypeSafe AI caught my attention.
A model built for fast, structured decisions seemed worth trying in two roles: choosing which model should do the work, and evaluating Clara’s responses.
So I tried both.
Jev as a router
First, I added Jev as a router in my Hermes workflow. Routine work stays with Gemini Flash. Heavier work goes to GPT-5.6 Sol. Low-confidence routing decisions and tasks requiring Astra come back to me for confirmation.
That gives Jev a small but consequential job: helping decide where to spend more model capability. I still need to test whether those routing decisions hold up over time.
Jev as an evaluator
Then I tested Jev as an evaluator for Clara. Its speed and structured results made it easy to integrate. Gemini Flash matched one more reference label and gave explanations I could inspect.
The sample was too small to call a winner, but it clarified the trade-off: fast checks versus more visibility into each judgement.
Jev stays in my toolbox. The next test is whether it catches the failures that matter for Clara. I started by testing whether I could trust her answers. Now I’m testing how much I can trust her evaluator.
Next: a two-pass evaluator
My next experiment is a two-pass evaluator:
Jev screens each claim against the reference material, and a stronger model reviews anything flagged as unsupported or uncertain. I’ll start by testing a 0.90 confidence threshold for passing the support check, with everything else escalated. I’ll also send a sample of passed claims for review to check what Jev misses.
The aim is faster, cheaper evaluation. The test is whether those savings come at the cost of missed defects.
What would you need to see before trusting that first pass? Reply on the original LinkedIn post.