alt.hn

5/28/2026 at 9:14:22 PM

Amazon scraps AI leaderboard to stop workers chasing usage scores

 https://www.ft.com/content/b1a62a7f-6df5-4c90-94ce-64ce9c9961b6

by mmarian

5/29/2026 at 4:32:40 AM

I made a high score list for my team to get people to care about catching bugs during code review.

The gamifying started immediately.

If you ever make anything competitive there will be that one guy trying to cheat his way to the top.

The high score list was supposed to be for finding logic errors. This guy was trying to pass off finding typos in text strings as equivalent to finding something that would blow up prod if pushed up.

I really... hate... reality

by novia

5/29/2026 at 7:11:40 AM

If you implement a game, people would play the game. (which is better than if they ignored it)

I would have thought that software engineers, of all people, would understand that one's intention and what they actually implemented in a system can be two different things, and careful effort must be expended to properly implement it, and that "good intentions" alone is not sufficient.

by hnfong

5/29/2026 at 8:09:06 AM

There's playing the game as intended and playing to the rules. People who insist on doing the latter even after being asked not to are infuriating. I've got one friend who got himself permanently uninvited from board game night for these sorts of shenanigans.

by Doxin

5/29/2026 at 3:44:15 PM

Did you try to analyze afterwards why your subordinates tried to do extra work, in that particular case? Maybe they thought that their performance (aka livelihood) would be tied to that leaderboard? And why would they have such a thought? I have some suspicions about the root cause of the issue :)

by Yizahi

5/29/2026 at 5:17:57 PM

I was an IC procrastinating on my real work, not a manager. I think I implemented this in Github during my first sprint on the team. Managers are probably wise enough to the way of humans to not attempt something like this (Goodhart's Law and all that). I know as a fact my peers started pointing to the high score list I made as a way to start justifying asking for raises and things like that, to demonstrate that they were pulling their weight on the team harder than others.

by novia

5/29/2026 at 7:16:40 AM

This isn't always true, if it makes you feel better.

In order for a game like this (what you described is a type of game) to work, all players need to have the same level of trust vectors toward the other players. This type of situation only emerges in very localized instances. The military, for example.

by remarkEon

5/29/2026 at 7:49:04 AM

1. Don’t make the game unless you can enforce the rules.

by brador

5/28/2026 at 9:32:13 PM

Who could have thought that a metric would become a target

by Zopieux

5/28/2026 at 9:56:37 PM

“Leadership” at these megacorps

They are so smart. Great job all.

by qsxfthnkp2322

5/29/2026 at 12:37:31 AM

MBAs are simply unable to learn this.

by mrbungie

5/29/2026 at 4:17:34 AM

The SVP who put this in place, his last name was Goodhart.

by jghn

5/28/2026 at 10:32:16 PM

This is how I learn that they shutdown the awards for Kiro usage.

But more seriously, while this is worded as a result of abuse, it had been up for more than a year at this point as a way to push people to use Kiro (you get badges for each "level" basically). Once you reach a point where everyone is using those tools, it makes no sense to keep it around.

Also it was not related to any performance metric, it was a pure vanity thing of getting virtual awards to display.

by belval

5/29/2026 at 12:37:19 AM

Do you think ai usage will decrease due to this change ?

by pylua

5/29/2026 at 2:25:36 AM

Yes. I've seen the AI usage patterns from people at the top of these kind of leaderboards, they understand that they are not doing things that they expect to produce commensurate value. At best they've inferred (sometimes correctly!) that the existence of the leaderboard means leadership has decided it's OK to burn prodigous amounts of token on any experimental thing one can imagine that might be useful.

by SpicyLemonZest

5/29/2026 at 1:01:19 PM

> Do you think ai usage will decrease due to this change ?

Frivolous usage sure, internally at Amazon there is a subculture (if you can call it that) of award chasers. Using Kiro for mundane task to burn tokens does not sound that far-fetched.

Overall usage though no I don't think so, these tools have some pretty wide adoption at this point and not by people chasing awards.

by belval

5/29/2026 at 5:52:33 AM

Seems reasonable. The intention of these things has always been to use a blunt tool to force adoption. Once you have tool penetration you don't need a reward mechanism. The question (which Amazon has no interest in answering for us, I suppose) is whether this tool worked.

by arjie

5/29/2026 at 2:30:17 AM

Incredible how often perverse incentives are implemented, it's like there's no foresight or understanding of behavior at all.

by david_shi

5/29/2026 at 1:29:03 AM

How many employees were hatetokenmaxxing I wonder

by smrtinsert