DebertaV3's architecture and noising should be even better as a basis because it had a couple inductive biases (cross encoder, disentangled attention and RTD corruptions) that enabled it to have unmatched weight performance ratio on such tasks.

My gut tells me that a better approach to a calibrated 0-shot classifier than shoehorning DiffusionGemma would be starting from another gemma, T5GemmaV2. Take its encoder and do continual training on an RTD objective and a large relational synthetic data mix. Then finetuning (multi-annotator data will help calibration) on as many proper NLI datasets as possible. That still lacks the DebertaV3 disentangled attention's inductive bias, however.

Jev also has its calibrated predictions component which is important. Temperature scaling is probably the easiest first pass. But there's lots of sensible options to improve on that.

ModernBERT might be the easier, more stable starting point than T5Gemma though.

This is very interesting! Seems like a promising direction.

I wonder though if it supports the same claims as Jev: answers are not impacted by other answers to the same questions, nor the existance of other questions? It seems by sharing KV cache all questions will be visible. And I think the diffusion causes the answers to attend to eachother?

Also I think without fine-tuning we can not say that the probabilities it output are actually probabilities. Maybe fine tuning using Brier scoring would do the trick?

Maybe some kind of hierarchical structure of the KV cache can make questions independent, and with smaller diffusion canvas’ generated in batch can be a way to make answers generate independently?

This PR is interesting but it's making the assumption that what Jev has done is based on a diffusion model or that a diffusion model is superior for this work. Which may or may not be the case.

If I understand it though it does mean you can evaluate a bunch of questions simultaneously, which is an advantage.

Also: While I think it's expected/normal to see LLM-generated programs... there's a lot of LLM written comments in that PR, which is sad to see. Auto-human.

Today, he released Laya, a model based on his paper: https://github.com/NandhaKishorM/laya

Jev is, as far as I understand, essentially very optimised for zero shot classification [0]. Something like BERT could be and has been tuned to provide similar "decision making" at a similar latency and cost advantage quite some time back. Advantage over full on LLMs is mainly the efficiency and of something like Jev over e.g. the encoder/decoder based classifier I had in front of an LLM to route to different prompts depending on the users likely needs, that Jev does perform at a more consistent level, allegedly roughly akin to GPT-5.6 Terra, but at the lower cost and latency. Currently testing that, but seems promising, if Jev classifies at or above Terra level, I see no reason not to leverage it.

Can add that I tried using a heavily pruned mt0 based model for structured classification along with structured output for local tagging and simple renaming suggestions. While it does work, the balance is hard to get right for the machine I was targeting as a minimum spec (Macbook Neo), so that's on ice. Focusing on one of the tasks easily goes below 100mb with solid latency across all EU Latin script languages, but the second you add a few, it's simply not in the quality budget, so while LLMs can do anything Jev and similarly focused models can, it comes at a literal cost. Could maybe accomplish the goal with multiple models (BERT+mt0+...), but that get messy.

In general just happy to see a bit of the millions flooding into the industry being used to improve on less flashy but immensely useful solutions. It's amazing that you can technically use LLMs for most tasks, but not every org has a near infinite budget and there is still a lot to gain from applying more recent learnings to old solutions along with just updating their training data to the current year. Also makes business sense, competition on frontier or mid-tier LLMs is vicious, focusing on an underserved niche with clear application is clever.

[0] https://huggingface.co/tasks/zero-shot-classification

It's a non-instruction-tuned classifier model trained on a confidence-aware RL variety that generates its own schema and follows it, with a confidence score output. Think BERT on crack, smart enough to be used as a decision maker (conceptually). They call it "not an LLM" because it's non-generative but of course it's a language model in the same way all non-instruction-tuned classifiers are.

No idea at all what this particular project called openjev is doing, but https://github.com/TheoLeeCJ/openjev and https://github.com/ekzhang/openjev-sglang (neither of which I have any relation to) generate a single token, rather than JSON structured output. I wrote up this technique here: https://sgnt.ai/p/jev/

In Jev you pass options in the input and its output just gives some probability for each. Oai structured output just follows a schema. The exact output is still generated and there is no probability

in short: it's faster, cheaper, smart structured output.

each "question" is answered in parallel instead of a sequential (like an LLM). so if you have an input like:

    {"is_it_hotdog": noul, "is_it_apple", noul}

it answers is_it_hotdog and is_it_apple in parallel and gives a probability.

AI output right now is like a final exam essay response from an anxious student. Instead of being edited for focus and clarity, it's anti-edited to cram in as many details as possible. Instead of worrying that the reader might get bored or confused, it assumes that the reader has no choice but to read the whole thing, even if they get a headache. It doesn't care about picking the most useful perspective on a problem; it cares about covering every possible angle that a grader might use to dock points from it.

It's basically the work you get from a smart, diligent person who is oblivious to any shared goal and approaches every assignment with a CYA attitude.

Berkshire got it right a long time ago.

https://www.berkshirehathaway.com/

There's a "unsloppify site" toggle on top but the unsloppified version looks exactly as vibe coded as the regular one.

I love how the 'unsloppify' button changes the theme but nothing else, and also messes with the layout enough that you can't actually toggle it without scrolling and repositioning your cursor.

I had to re-read a few times to figure out what the site was for and about. Still not sure I understand but that's the problem for them. If I'm a customer, I'm gone cause I can't figure out what it's for and I see this far too often nowadays for a lot of technical sites.

Unsolvable problem.

Why was the aesthetic standard to be pale when workers worked the fields and royals were inside, but tan when workers moved into factories and only the rich could afford to go on a beach vacation?

Aesthetic standards are formed by association. Its why sites that are "well designed" but obviously just use a squarespace or wix template feel so cheap. Why millenial flannel went from hip to standard to outdated. Why purple was the color of royalty before we could synthesize the pigment.

Having good design is about associations. Whatever design LLMs will default to, it will always feel cheap because we will learn over time that that design means cheap. Having good taste is about being ahead of the curve. An LLM cant be ahead of the curve because then that becomes the standard, and theres a new ahead.

You can use LLMs to make novel looking websites by carefully telling it to add certain details, use certain elementd, etc. At that point youve looped back to being a graphic designer.

The problem is the overabundance of text, they can’t let it breathe. Everywhere has to be filled up with bits of hardly-relevant text.

I appreciate a nice brutalist aesthetic like this tbh. It’s also good that there’s a baseline for quality in terms of layout and spacing and contrast and whatnot usually, so the HN webshit meta conversation has shifted from that to whinging about an LLM making it.

The overall arrangement and useless shit LLMs put in the copy is often annoying though.

Same. I can’t really put my finger on what exactly is turning me off though. I mean, apart from the obvious AI-generated text.

Mainly cause you never know what you’re getting. Over time, we trained our minds to believe that a well put site = effort, so at the very least people behind it cared. Now, it takes zero effort to make a site look good. So appearance in general means even less. In fact, now a poorly put together site might mean someone cared, wrote it by hand, flaws and all, to give you the human to human experience.

If there is a silver lining in all this, this might get us to appreciate the flaws in all humans, heck even yearn for them.