CIP Blog company blog
Positional preferences, order effects, prompt sensitivity undermine AI judgments
When Large Language Models are used as judges for decision-making across various sensitive domains, they consistently exhibit unpredictable and hidden measurement biases, making their verdicts unreliable despite common prompt engineering practices.
- Points
- 154
- HN comments
- 81
- Submissions
- 1
- Cited by
- 0 commenters
AI & Machine LearningSociety, Policy & Culture2025
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| May 23, 2025 | Positional preferences, order effects, prompt sensitivity undermine AI judgments by joalstein | 154 | 81 |
