8/21/2026 at 2:04:24 PM
It fails the simple clock test for me which Qwen3.8 27B got (nearly) right. given an image of a clock https://files.catbox.moe/kgwa5e.pngI asked it "what time does the clock show?" (both on reasoning: high)
DS answered: The clock shows *5:10* (and 45 seconds). Here is the breakdown: * *Hour hand (red, shortest):* Pointing at the *5*. * *Minute hand (green, longest):* Pointing at the *2*, which represents 10 minutes. * *Second hand (blue, medium):* Pointing at the *9*, which represents 45 seconds.
Qwen answered: The clock shows *8:10* (with the red second hand on the 5, i.e. *8:10:25*).
- *Hour hand* (short, blue) → 8 - *Minute hand* (long, green) → 2 (10 minutes) - *Second hand* (thin, red) → 5 (25 seconds)
Correct answer is 08:09:25.
by leumon
8/21/2026 at 2:40:54 PM
I’ll keep that in mind next time I need to tell what time it is by asking an llm to read an analog clock.Snark aside, I’m not sure that these gotcha tests are any more useful than asking politicians gotcha questions. Sure, the model can’t tell me what time it is, but it can code the Wang algorithm for noisy audio matching in one shot. Maybe this is just me being an optimist, but this is my hiring philosophy and I guess maybe now my llm philosophy: I’m not interested in seeing how dumb I can make you look, I’m more interested in how smart you can be.
by dghlsakjg
8/21/2026 at 2:30:11 PM
This is not a normal looking clock - most clocks have either one color for all hands (second hand is thinnest and maybe also longest) or one color for hour/minute and one for second. I know that the hand lengths and thicknesses on this image are correct but for some reason I, a totally human person who grew up when analog clocks were still common, see this and think the hand on the 5 is the minute hand. How does the AI do if you just make all the hands black?by emosenkis
8/21/2026 at 2:39:16 PM
then deepseek answers: "The clock shows 8:25. The short hour hand is pointing to the 8, and the long minute hand is pointing to the 5, which represents 25 minutes."and qwen still answers: "The clock shows *8:10* (with the second hand on the 5, i.e., 25 seconds). - *Hour hand* points to the 8 - *Minute hand* points to the 2 (= 10 minutes) - *Second hand* points to the 5 (= 25 seconds) So the time is *8:10:25*, or simply *8:10*."
by leumon
8/21/2026 at 2:43:09 PM
Qwen still got the wrong answer, though.Are we more forgiving because it’s the same type of mistake a human would make?
by dghlsakjg
8/21/2026 at 2:20:08 PM
A good share of humanity would have also gotten this question wrong!by ttul
8/21/2026 at 2:31:05 PM
It's been four years that we are looping those"The professional failed its task!" // "Laymen would have failed it too".
Which makes no sense.
by mdp2021
8/21/2026 at 2:22:51 PM
Yeah, I heard most kids these days can't read analog clocks either.I can't actually remember where I learned to read a clock, it might have actually been in school. I guess that means they don't teach it anymore. (Everyone's phone shows the time anyway...)
by andai
8/21/2026 at 2:11:11 PM
Gemini 3.7 Flash and 5.6-Sol (on all reasoning levels) also answer 8:10:25. The new "stealth" Ox Alpha also replies with the same. Opus 5 replies with 8:10 (no seconds). Not sure why this is so hard for them; Gemini is especially good at vision and I would have expected better from it.by ComputerGuru
8/21/2026 at 2:11:15 PM
welp, damning indictment. not sure if that means DS is super crap, or qwen is super goodby nubg
8/21/2026 at 2:22:12 PM
Neither. Performance of all models is incredibly spikey.by wolttam