the underlying mechanism has to sample tokens (1000/90000 for openai's tokenizer) by some shallow heuristic to build the decision pool (more details are in https://github.com/giga-james/jevgpt/blob/main/docs/experime... !), while standard autoregressive models have access to the entire token vocabulary to predict the next word

jev allows 255 decisions (token vocabulary here), so if we want to support a token vocabulary of 1000, we have to parallelize into multiple jev calls to allow that expansion. the verifier pass simply chooses the best out of the parallel decisions so we can exceed the 255 decision limit. in theory this should allow for up to 255^2 token limit, but if you increase the verifier depth you can increase that exponential factor as much as you want (it'll be slow though)