Benchmark for tracking model capability after release. - ninjahawk/livenerf
2 comments
_aavaa_3 days ago
If you don’t trust and are on the lookout for silent nerfing, how are you protecting against it detecting a benchmark and using the un-nerfed model?
ethan19987 days ago
I have no idea how to do it. Keep rerunning benchmarking test can be costly.
Read the full thread on Hacker News →
Related stories
- Livenerf: Has Opus 5.5 been nerfed yet?github.comHacker News · 885 points · 1 day ago
- DEV Community · 0 points · about 11 hours ago
- Hacker News · 1 points · 3 days ago
- Hacker News · 1 points · 9 days ago
- DEV Community · 7 points · 3 days ago
- Hacker News · 1 points · 10 days ago