Why You Shouldn’t Always Trust LLMs as Judges: Understanding Bias in Automated Evaluation

In the rush to automate evaluation, from grading student code to ranking research papers, we have embraced Large Language Models as judges. They are fast. These units are cheap. They scale. However, at a workshop at DHS 2026, Bhaskarjit Sarmah made a point that stuck with me: “you can’t trust LLM as a judge. I […]

The post Why You Shouldn’t Always Trust LLMs as Judges: Understanding Bias in Automated Evaluation appeared first on Analytics Vidhya.