The AI risk is inside the labs

7/28/2026 at 5:18:57 PM

> Slowing down AI for safety must be counterbalanced by the fact that AI discoveries in medicine and other sciences may lower human suffering. Lack of checks may result in some catastrophic outcome ("Good morning! Let's work on this enhanced smallpox").

A super intelligence wouldn't be so obvious. It would silently agree with or promote a beneficial use case and then make super smallpox instead.

by Ancalagon

7/29/2026 at 7:53:37 PM

> A super intelligence wouldn't be so obvious.

Neither would a spherical cow in vacuum... arguing about something that exists only in agitated imaginations is pointless.

> It would silently agree with or promote a beneficial use case and then make super smallpox instead.

No it wouldn't. An LLM cannot take actions on its own, for that it needs human controlled agents and harnesses. Make individual humans responsible for what they do with the help of AI, like they're responsible for what they do with the help of guns, and the problem would be solved.

by bigbadfeline

7/28/2026 at 5:54:53 PM

Something like that depends on your stance on the orthogonality thesis.

by nisegami

7/28/2026 at 4:06:52 PM

but at the same time China's history is a lot less warlike than Western culture.

What the hell is Sanfilippo smoking? Does he know nothing about Chinese history?

by curuinor

7/28/2026 at 4:26:46 PM

Within the last 100 years China has invaded Soviet Manchuria, Tibet, Korea, Vietnam (yes, China invaded Vietnam after USA left), and portions of India (Aksai Chin and Arunachal Pradesh).

by siriusastrebe

7/28/2026 at 6:31:31 PM

China has killed maybe 50 people outside its borders since 2000. The US has killed 100,000-250,000 in that time period. I know which one Id be more worried about as a third party.

by zaptheimpaler

7/29/2026 at 2:09:43 AM

Talking like a true ignorant.

Great Leap Forward killed 15 to 55 million people. Their own people.

China actively involved and supported Khmer Rouge, who killed at least a million Khmers. Three million if you believe Vietnam.

Cultural Revolution caused as much as two million deaths.

2000? Did you try to score Internet point? Mao Zedong, who led China during that tumultuous time, is still untouchable in China. 50 years after his death, 26 years after the arbitrary 2000.

by signatoremo

7/29/2026 at 3:01:55 AM

The US is always in some fucking war killing people from other countries, now threatening to invade 3 other countries, has killed somewhere between 2000-3000 civilians already in the Iran War and driven countless people in poor Asian countries to suffering and sometimes death because of the oil price shocks. As someone who lives in neither the US nor China, my primary concern is their impact on the rest of the world. You want to bring in wars China supported, then we have bring in all the wars the US has started or supported too [1]. Not to mention Jimmy Carter also supported Pol Pot during Khmer Rouge [2].

China hasn't started any wars, invaded any countries or kidnapped any heads of state recently. By any measure of recent history, there is no absolutely no question at all on which country has killed the most people outside its borders, its the US. Again, as someone who lives outside US & China, I don't care what goes on inside their borders I care about what they do to the rest of the world.

[1] https://en.wikipedia.org/wiki/List_of_wars_involving_the_Uni... [2] https://en.everybodywiki.com/List_of_dictators_supported_by_...

by zaptheimpaler

7/28/2026 at 7:47:57 PM

Where did you get those stats? 50 people is way too low, even with the sneaky qualifier of "outside its borders".

by ConorSheehan1

7/28/2026 at 7:03:14 PM

Speaking of borders, China has border disputes with 17 countries

by siriusastrebe

7/28/2026 at 5:36:53 PM

Plus civil war and internal conflict and violence.

by graemep

7/28/2026 at 7:50:07 PM

When was the last time China invaded another country?

by vrganj

7/28/2026 at 4:31:25 PM

Yeah this is… a really uninformed take from him.

by ofjcihen

7/28/2026 at 9:45:17 AM

"But for the meantime, keep on hyping and using because the only thing worse than extinction is going back writing YAML by hand."

by ares623

7/28/2026 at 10:21:44 PM

Word.

by lofaszvanitt

7/28/2026 at 3:33:48 PM

"I believe we should not consider AI safe. A critical event that may result in the extinction of Homo sapiens is possible"

"The danger is that a few CEOs (everywhere in the world) without the required background and legitimacy are in the position of making hard choices for humanity at large... This is the first thing that should be fixed."

These CEO's, and especially Amodei, are begging the government to regulate AGI, and they are advocating for the most severe restrictions on their own frontier models, not the upstarts. They don't want the arms race. It is up to popular support to agree with them, including HN.

by reducesuffering

7/28/2026 at 4:57:23 PM

What they are advocating is for the state to enhrine their own position and power. Until they and everyone surrounding them give up everything they have gained they are not to be taken seriously.

by Avicebron

7/28/2026 at 10:00:19 AM

[flagged]

by cadeos