These days, I tend to start my coding sessions by the high level problem I'm trying to solve vs the prescriptive, specific solution I may have in mind. It often surfaces ideas and approaches that I did not know about.

Originally it was so I can label data to fine-tune a VLM, but now a few tiny classifiers that run in milliseconds on cpu.

Now its collecting data to make a domain specific BERT and do what Jev does.

Our resulting RBF models are tiny and fit in L1 cache, with microsecond inference latency.

30KB model, 40-50ms inference. Pretty happy with the results so far!

I can see an entire industry of tiny models like this, now that we have AI to help us do the grunt setup work (validation/training data creation, data cleaning, etc). Or just use a general classifier like Jev/Kev ha

that is generally a very healthy attitude in the AI space anyway in my opinion.

Some of our R&D departments haven't actually finished an interesting project in years because they keep jumping from trend to trend wanting to try out all the latest shit all the time.

In their privacy policy they say

We (1) will not train or fine tune any artificial intelligence or machine learning models on Input, and (2) will not disclose any Input to a third party other than our service providers.

by not allowing to benchmark it, they make it user-hostile, you don't know for what kind of quality you pay

Anything that required knowledge / familiarity mmBERT and ModernBERT post-trains performed much worse.

So it seems like they did some kind of useful expansive pre-training.

Things that were Qwen or Gemma Diffusion did better at those kinds of tasks but were generally pretty inconsistent in terms of whether they could succeed repeatedly (and be stable + reliable) on the many types of tasks that are in the cookbook part of the Jev docs.

If you ask Jev similar input + questions, it's pretty stable. And does a reasonable job on a lot of questions.

This one public benchmark (the only I've seen) seems to give the open versions way too much credit. It wasn't my experience at all.

It gave my a false wrong sense of what might be required to get it working for something at work to avoid needing a new subprocessor as - at least on Cloudflare / OpenRouter Jev is third-party not hosted.

also tried myself: https://console.typesafe.ai/playground?share=shr_1690a3160f1...

Big if. More likely, it seems, is they started with an open LLM model and fine-tuned and repurposed it via their "RLCD" process.

Jev came in, and added that magic of "you dont need to train your classifier or determine the weights" if you dont want to, and just get the classified answer out. I think that's what is making people see this with a glitter in their eyes.

I'm still trying to catch-up on the Jev stuff, but my understanding is that it's basically just a more efficient LLM when all you want is the LLM to produce a classification.

There's more to it, of course, but it's not just "generic" classification ML because it accepts arbitrary inputs and can produce probabilities over arbitrary classes. Not saying this is the first time people have done this, but typically classification tasks are more static and limited.

In the same vein, it's also not just an LLM with structured outputs (which have been a thing for a while) specifically because that is a very inefficient way to approach classification using this kind of architecture. Jev models are much more performant because of how limited they are compared to a full LLM.

So when you want an LLM, but you only really need this kind of classification from the LLM, then Jev makes a ton of sense. This makes sense for me, since I've definitely used LLMs for this kind of classification work and, even then, it kind of felt like using a jackhammer to place some nails, etc.

Happy to be correct, though.

Compared to traditional ML classification, Jev works without training, like a LLM.

LLMs can act as classifiers but they still have to generate text output in the form of a JSON object. This means they have to generate every single curly bracket, quote, command, etc. This turns out to be pretty expensive. On the other hand, Jev uses a different decision head so it doesn't generate text output at all, it outputs logits *only* for the choices provided. So it completely avoids the need to generate text at all, which means no malformed JSON and it's much faster as a result.

Finally, Jev also provides confidence scores that are actually reliable (not made up like LLMs).