GPT 5.6 Sol Still Fails on Basic Code Review

GPT 5.6 Sol Still Fails on Basic Code Review chatgpt.com

1 points•mbeavitt•2 days ago•4 comments•

#basic #gpt #still #code #sol #fails #sol still #still fails

4 comments

I'm not going to write some big blog post here, but this chat log is quite entertaining. I wrote a fundamental string algorithm in C and GPT 5.6 Sol on medium effort hallucinated not one but two correctness issues with the code!

Maybe this is not surprising to others but it's been a long while since I caught one of these models making such a glaring error.

chris_money2022 days ago

I'm pretty sure there is a completely different model for Chat vs Coding.

Like if you deploy 5.6 on azure foundry there is a chat model and a coding/reasoning model

mbeavitt2 days ago

I learned something new today

Read the full thread on Hacker News →

4 comments

Related stories