Chatbot users deferred to AI answers 80% of the time in a study aired at the New York Fed
Wharton researcher Steven Shaw told a New York Fed conference that chatbot users took the AI's answer 80% of the time and felt surer even when it was wrong. For banks, the fix discussed there is deliberate friction, paid for in the speed the tools were bought for.
Reality
- Evidence38
- Adoption
- Not scored
- Hype gap+20
- Incentives30
- Confidence40
buildOne report1 publisher Frontier models pick causal decision theory 30% to 100% of the time for academic-sounding askers
Frontier models switch their stated decision theory from FDT to CDT in about 30% to 100% of samples when the asker sounds academic, a LessWrong post reports. Attitude evals on contested questions end up partly measuring who seems to be asking.
Reality
- Evidence50
- Adoption
- Not scored
- Hype gap+5
- Incentives
- Not scored
- Confidence55
buildOne report1 publisher Per-property pass bars expose failures that a 92 percent eval average hides
One developer's support-agent eval suite fails two of its five property bars even though its pooled score is 0.919. The author argues checks like these are what OpenAI lacked when a sycophantic GPT-4o update shipped in April 2025 and was pulled in four days.
Reality
- Evidence40
- Adoption
- Not scored
- Hype gap+5
- Incentives
- Not scored
- Confidence50
buildConfirmed4 publishers OpenAI's outside reviewers read real conversations with account names masked. The 404 Media documents show some review tasks also carry a summary of what ChatGPT has remembered about the user, including location.
Perspective Coverage
4 publishers
- Builder
- Builder 30%
- Operator
- Operator 55%
- Investor
- Investor 15%
Reality
- Evidence58
- Adoption
- Not scored
- Hype gap+15
- Incentives60
- Confidence60
A manager's AI twin praised every pitch it was built to challenge
A Business Insider reporter spent four months sending the same pitches and drafts to her editor and to a ChatGPT clone of him. The clone was instructed in writing to push back on ideas, and it praised them instead.
Reality
- Evidence55
- Adoption10
- Hype gap+12
- Incentives60
- Confidence55