32,706 tokens
  38min 19s
  14.22 t/s

For my "Please recite Jabberwocky" test the bf16 almost passes but the ternary and even fp8 versions fail badly.

  [ Prompt: 95.0 t/s | Generation: 26.5 t/s ]

Am I understanding correctly that no draft model exists (will never exist or just currently does not exist yet)?

There is no draft file in Huggingface's repository ( https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf/tr... ) and in the file "scripts/download_models.sh" of the demo repository ( https://github.com/PrismML-Eng/Bonsai-demo/blob/main/scripts... ) I see this remark:

  if [ "$_family" = "bonsai2" ]; then
    the projector ships in the same repo; Bonsai 2 has no dspark drafter

  ./llama.cpp/build/bin/llama-server \
    -c "$context_size" \
    -ctk q4_0 \
    -ctv q4_0 \
    --no-models-autoload \
    --models-dir ~/bonsai \
    --reasoning-preserve \
    -ngl 99

If you have found better instructions and they work then use those instead!

Personally I prefer to download models directly rather than running some `./setup.sh` script where I need to then review what it does first.

This is another pet peeve that I have with the way we phrase these things: It should be 9x "as large" and 8x "larger."

The way I'd usually phrase this, to avoid ambiguity, is "this new model is 11% the size (of the original)." Or, the other way, "the old model is 9x the size".

Conversely, 9x [filesize/natural number] is bigger. Every time. At least in the basic maths used by most people. There is no conversion into other units.

Therefore "9x smaller" when talking about a natural number like filesize is a nonsense statement in logic terms. If you strive for unambiguous phrasing - which is a significant part of the programming experience - this logical nonsense might well perturb you.

But english language is a flexible thing and if the phrase communicates your intent to your audience then that's fine by me.

50 percent smaller means either half, or two-thirds the size, depending on Apple Marketing doing the math.

0.999 times smaller means size is nearly zero. 9 times smaller means you get negative memory from loading it.

I really hope they stop using PowerVR in pixel 11.

One such method that I've been meaning to look into further is Tencent's AngelSlim QAT/PTQ approach. They did a Hy4 preview release thats an STQ_1_0 at 2.38 bpw:

https://huggingface.co/AngelSlim/Hy4-preview-GGUF https://arxiv.org/abs/2602.21233

Of course, it's still 213g of VRAM I'd need so it's somewhat out of the range of what I can run locally. In contrast, this new Bonsai is nice because the original was already exciting for making use of low VRAM devices. Could breath new life into some of the older GPUs that were previously close to top of the line just quite VRAM constrained by modern standards and still quite cost effective for now.