Steven Adler, who worked at OpenAI for four years, performed an interesting analysis of ChatGPT's misbehavior after the model was "fixed" and saw a ton of weird results.
https://substack.com/home/post/p-162910384
Steven Adler, who worked at OpenAI for four years, performed an interesting analysis of ChatGPT's misbehavior after the model was "fixed" and saw a ton of weird results.
https://substack.com/home/post/p-162910384
5 Comments
I thought it was an interesting article that suggests AI companies will continue to struggle to stop misbehavior, and the problem is likely only going to get worse. How do you think we can drive more analysis like this in the future or have companies better monitor AI behavior?
Why do we keep humanizing what this stuff is? It doesn’t “misbehave”. It would have to understand what’s good and bad behavior.
Not to mention a long, long way until ChatGPT provides the correct answer more than 50% of the time.
a basic problem with all these tests to measure how good LLMs are is that companies game the tests immediately
the author complains that OpenAI doesn’t do sycophancy tests, but all that would happen if they did, is that we’d have Sam Altman on stage saying “the new ChatGPT 5 scores 1.1% on sycophancy tests. This model just tells you the unvarnished truth” while the model either keeps licking the user’s boots in other ways the original test did not…or even worse, adopted new weird, unexpected behaviors deformed by its training
so in a way, it’s better if these are given by outside parties, but sooner or later the marketers will demand their AIs do better at these tests, give the AIs the right answers in their training data and deform the creature
the author does take an interesting detour to explain why having the model explain itself is futile; the explanation itself will be victim of the sycophancy bias
Steven’s come a long way from when he got kicked out of Gun n Roses for being a drug addict. Good for him.