Non-experts are increasingly being asked to evaluate output from Google's Gemini AI model; is that worth being concerned about?

There's no arguing that AI still has quite a few unreliable moments, but one would hope that at least its evaluations would be accurate. However, last week Google allegedly instructed contract workers evaluating Gemini not to skip any prompts,

A firm hired to test Google Gemini for accuracy has reportedly changed its policy from letting analysts skip prompts outside of their expertise to evaluating them even if they don't fully understand the responses.