Show HN: jevals – replacing LLM judges with typed Jev decisions

Show HN: jevals – replacing LLM judges with typed Jev decisions github.com

Agent evals and guardrails as Jev decisions: one request per trace, a fraction of a cent, fast enough for the agent loop. Runs locally with Kev or Laya. - openlayer-ai/jevals

47 points•gbayomi•10 days ago•6 comments•

#jev #jevals #jev decisions #judges #replacing #typed #llm #replacing llm

6 comments

sshussain27010 days ago

This is going to be a hot use case.

gbayomi8 days ago

Agreed

adityamishra2419 days ago

Interesting idea. How do you handle cases where the decision depends on context that isn't captured by the typed decision type?

gbayomi8 days ago

It’s very dependent on the underlying model right now, but it performs well for the current versions of jev/keV/laya!

jacksun7889 days ago

Curious — how's this compare to laya?

gbayomi8 days ago

It can also run with laya. The idea is to have an easy way to run common evals with decision models and easily switch the “backend” from jev/kev/laya etc

Read the full thread on Hacker News →

6 comments

Related stories