HN
login
Greedy Decoding Is Not Precision-Invariant: Cross-Precision Output Divergencearxiv.org

Greedy decoding from large language models is commonly treated as deterministic. We show it is not precision-invariant: the same model, prompt, and decoding algorithm produce different outputs in BF16 versus FP16 on…

1 points•sbulaev•7 days ago•0 comments•
#precision#decoding#greedy#invariant#precision invariant#greedy decoding#cross#output

0 comments

No comments yet.

Related stories