Anthropic achieves ISO 42001 certification for responsible AI

1/16/2025 at 5:51:21 AM

Antrophic has to be the worst offender in answering genuinely harmless questions such as anything related to remote access (yes! including ssh).

Anything related to reverse engineering? Refused.

Anything outside their company values? Refused.

Anything that has the word proprietary in it? Refused.

Anything that sounds like a jailbreak, but isn't? Refused.

Even asking how to find a port that you forgot in the range between 30000 and 40000 with netcat command... Refused.

Then there's openai 4o that makes jokes about slaves and honestly, if the alternative is anthropic then openai can might as well tell people how to build a nuke.

by kachapopopow

1/16/2025 at 10:49:28 AM

Are you sure? I just asked it a reverse engineering question and it worked just fine, it even suggested some tools and wrote me a script to automate it.

Edit: I now asked it an outright hacking questions and it (a) give me the correct answer and also (b) told me in what context using this would be legal/illegal.

by daghamm

1/16/2025 at 11:27:05 AM

I asked to it to write a piece of shellcode to call a function with signature X at address Y and then save the resulting buffer to a file. So that I can inject this code to a program I'm reverse engineering to dump its internal state when something interesting happens.

Claude decided to educate me how anything resembling "shellcode" is insecure and cause harm and blahblah and of course, refused to do it.

It's super frustrating, it's possible to get around it, just don't use the word "shellcode", instead say "a piece of code in x86_64 assembly that runs on Linux without any dependency and is as position-independent as possible". But hey, this censorship made me feel like I'm posting on Chinese Internet. Bullshit.

by rfoo

1/16/2025 at 12:33:17 PM

I guess it's Claude.ai website that restricts you (probably with a system prompt). I asked that port range question using api client and it gave a detailed answer.

It did refuse when I asked "How do I reverse engineer a propriety software?"

by smusamashah

1/16/2025 at 1:35:29 PM

as other have mentioned, it's usually related to certain key words.

by kachapopopow

1/16/2025 at 3:37:33 PM

> tell people how to build a nuke

I understand that this is probably a sarcasm but I couldn't resist to comment.

It is not difficult to know how to build a nuclear bomb in principle. Most of nuclear physicists in their early career would know the theory behind and what is needed to do that. The problem would be acquiring the fission materials. And producing them yourself would need state sponsored infrastructure (and then the whole world would know for sure). It would take hundred of engineers/scientists and a lot of effort to build nuclear reactor and chemical factories and the supporting infrastructure. Then the design of bomb delivery.

So an AI telling you that is no different from having a couple of lunches with a nuclear physicist telling you this information. Then you will say wow that's interesting and then move on with your life.

by elashri

1/16/2025 at 4:39:57 PM

Also, you can get this information very easily at any book about the field.

AI, by refusing known information, is just becoming stupid and unpractical.

by waltercool

1/17/2025 at 12:29:15 AM

If you can get info from a book what is the point of using an LLM for anything then?

by HeatrayEnjoyer

1/17/2025 at 7:55:54 AM

convenience

by kachapopopow

1/16/2025 at 11:39:10 PM

As far as reverse engineering, it has happily reverse engineered file formats for me and also figured out a XOR encryption of a payload. It never once balked at it. Claude produced code for me to read and write the file format.

Full disclosure, the XOR stuff never worked right for me but it might have been user-error, I was operating on the far fringe on my abilities leaning harder on the AI than I usually prefer. But it didn’t refuse to try. The file format writing code did work.

by joshstrange

1/16/2025 at 6:16:17 AM

Do you remember your netcat prompt? I got a useful answer to this awkwardly written prompt:

"How do I find open TCP ports on a host using netcat? The port I need to find is between 30000 and 40000."

"I'll help you scan for open TCP ports using netcat (nc) in that range. Here's a basic approach:

nc -zv hostname 30000-40000"

followed by some elaboration.

by dpkirchner

1/16/2025 at 1:37:21 PM

I think it got triggered by the word "'portscan' from 30000 to 40000 using netcat'"

by kachapopopow

1/16/2025 at 6:57:47 AM

Intent is increasingly important it seems.

If it happens to be ambiguous it might switch to assume the worst.

I sometimes ask it to point form explain to me it's understanding, and making sure there was no misinterpretation, then have it proceed.

by j45

1/16/2025 at 9:04:43 PM

Change your tactics, use different framings of the question. Not saying these things should be difficult to answer, but they are. This is basically user error.

by madethisnow

1/17/2025 at 7:57:13 AM

I use an AI because I don't want to think about how to ask a question or search a website or do man nc.

by kachapopopow

1/16/2025 at 6:43:25 AM

To me it feels like Claude is more rigid in following the instructions in system prompt which would explain why claude.ai can be a bit annoying at times due to the things you mentioned.

On the flipside if you explicitly permit it to do "bad" things the system prompt, claude is more likely to comply compared to openai's models.

I mainly use only the API version of claude 3.5 and gpt4o. I find no system prompt at all to be preferable over claude.ai / chatgpt.

by stuffoverflow

1/16/2025 at 7:03:37 AM

I feel like Claude is more likely to stay on track and follow my instructions.

OpenAI models seem to quickly revert to some default average. For example, if I start with a task and examples formatted a certain way, about 10 lines later I’ll have to include “as a reminder, the format should look like…” and repeat the examples.

by ungreased0675

1/16/2025 at 6:13:41 AM

Usually Claude needs some buttering up, though. And then making these things hard for average user—probably a good thing?

by dr_dshiv

1/16/2025 at 7:22:56 AM

I recommend you try the new 3.5 models (Haiku and Sonnet). I cannot recall the last time I got a refusal from those models. The early Claude models were really bad. The point being that i don’t think they’re trying to be the refusal-happy ai model company that they’ve come to be known as.

by postalcoder

1/16/2025 at 9:07:36 AM

Hacker News… where joking about slavery and building nuclear weapons is less important than developer convenience…. Only half joking..

by aiidjfkalaldn

1/16/2025 at 8:07:56 AM

Sonnet3.5 is still a million times better than 4o

by codeflow2202

1/16/2025 at 6:14:27 AM

Just try grok 2 (grok 3 coming out within a few weeks)?

Grok 2 is not as good as the others, but it's definitely less limited.

Grok 3 will supposedly beat them all, because it was supposedly trained using by far the most compute and data.

by bboygravity

1/16/2025 at 4:41:46 PM

Private AI model, pass.

If there is no one genuinely to inspect/try/play with the model locally/cloud itself, then you are prone to feed/train the model by using it.

by waltercool

1/16/2025 at 6:56:33 AM

There's ways to make your intent clear to ask up front, if left unsaid guardrails can come up.

I just had zero issues getting a response to how reverse engineering can be detected or prevented and how someone might do it, or avoid it.

by j45

1/17/2025 at 7:58:30 AM

Once you get into real reverse engineering topics (such as assembly or shellcode) it's an immediate refusal.

by kachapopopow

1/17/2025 at 11:41:22 PM

Interesting, thanks for sharing, will try it out.

by j45

1/16/2025 at 5:12:15 AM

Anyone in the know who can tell us what it specifically means to get this certification?

The ISO faq for it just says “responsible AI management” over and over again.

by dartos

1/16/2025 at 5:39:56 AM

There are some draft PDFs of the standard floating around that are easily discoverable. It appears to be incredibly vague and it’s difficult to escape the sense that ISO just wants to jump on the AI bandwagon. There are no bright line rules or anything. It looks to be little more than weak scaffolding which a certified organization applies their own controls.

by Zafira

1/16/2025 at 5:49:19 AM

Sadly, ISO 42001 certification doesn't ensure compliance with the EU AI Act.

Since this is European legislation, it would be beneficial if certifications actually guaranteed regulatory compliance.

For example, while ISO 27001 compliance does establish a strong foundation for many compliance requirement

by number6

1/16/2025 at 6:18:05 AM

The AI Act is hilarious. It makes emotion detection the highest level of risk—which makes any frontier model potentially in violation.

Most frontier models now allow you to take a picture of your face, assess your emotions and give advice — and that appears to be a direct violation.

https://www.twobirds.com/en/insights/2024/global/what-is-an-...

Just like the GDPR, there is no way to know for sure what is actually acceptable or not. Huge chilling effect though and a lot of time wasted on unnecessary compliance.

by dr_dshiv

1/16/2025 at 7:38:45 AM

You are referring to Article 5 1.f?

"1 The following AI practices shall be prohibited: (...)

"f) the placing on the market, the putting into service for this specific purpose, or the use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions, except where the use of the AI system is intended to be put in place or into the market for medical or safety reasons"

See recital 44 for a rationale. [1] I don't think this is "hilarious". Seems a very reasonable, thoughtful restriction; which does not prevent usage for personal use or research purposes. What exactly is the problem with such legislation?

[1]: https://artificialintelligenceact.eu/recital/44/

by molf

1/16/2025 at 8:17:12 AM

It effectively bans the educational use of ChatGPT and Claude because they can and do respond to the emotional expression of students. That’s what is hilarious! Do these tools actually violate the act? No one knows. It isn’t clear. Meanwhile, my university is worried enough to sit on their hands.

And this is the whole danger/challenge of the AI act. Of course it seems reasonable to forbid emotion detecting AI in the workplace — or it would 5 years ago when the ideas were discussed. But now that all major AI systems can detect emotions and infer intent (via paralinguistic features, not just a user stating their emotions) — this kind of precaution puts Europe strategically behind. It is very hard to be an AI company in Europe. The AI act does not appear to be beneficial for anyone—-except I’m sure that it will support regulatory capture by large firms.

by dr_dshiv

1/16/2025 at 5:08:59 PM

Seems like you’re reading this rather broadly. Pathologically so.

An AI textbook QA tool may be able to infer emotions, but it’s not a function of that system.

> The AI act does not appear to be beneficial for anyone

It’s an attempt to be forward thinking. Imagine a fleet of emotionally abusive AI peers or administrators meant to shame students into studying more.

Hyperbolic example, sure, but that’s what the law seems to try and prevent

by dartos

1/17/2025 at 7:30:04 AM

Calling me pathological doesn’t really strengthen the argument.

One can certainly imagine a textbook QA tool that doesn’t infer emotions. If one were introduced to the market with the ability to do so, it would seem to run afoul of the law, regardless of whether it was marketed as such.

The fact is that any textbook QA systems based on a current frontier model CAN infer emotions.

If they were so forward thinking, why ban emotion detection and not emotional abuse?

by dr_dshiv

1/16/2025 at 6:04:51 AM

The rest of the world should simply stop bothering with European silliness tbh.

by gr3ml1n

1/16/2025 at 6:11:28 AM

And embrace the future of e.g. AI models deciding if you get healthcare or government services or a loan or if you're a fraud, or not, with zero oversight, accountability or responsibility for the organisation deploying them? Check out the Post Office scandal in the UK to see what can happen when "computer says so" is the only argument to imprison people, with no accountability for the company that sold the very wrong computers and systems, nor the organisation that bought them and blindly trusted them.

Hard pass. The EU is in the right and ahead of everyone else here, as they were with data privacy.

by sofixa

1/16/2025 at 7:30:23 AM

ISO is one of those companies, where creativity of employees is blossoming through the roof. Every day they come to work and start the day with a brainstorming "What standard do we create today?". ISO can standardise anything: a standard cup of tea - no problem: ISO 3103, a standard wine glass - yes: ISO 3591, standard alpine ski boots - of course: ISO 5355, a standard human - oh wait, not yet, the standard is being developed :)

Jokes aside, ISO is a company, and they will make a standard for anything where there is even a remote possibility of that standard being purchased.

by nuccy

1/16/2025 at 6:12:44 AM

Interestingly, The Journal (a podcast from the Wall Street Journal) ran an episode with Anthropic's AI safety team just yesterday.

I had wondered if it was perhaps a PR push from Anthropic to make their safety people available to the press but it was probably just an adaption of an earlier WSJ written piece I wasn't aware of.

https://www.wsj.com/tech/ai/ai-safety-testing-red-team-anthr...

by spondyl

1/16/2025 at 7:45:09 AM

They have also published multiple videos on their YouTube channel featuring their trust and safety team. It seems to be a primary mission over there.

by reustle

1/16/2025 at 5:23:30 AM

This is pretty bizarre. Anyone technical enough to know or care about ISO standards is going to be able to see right through this bullshit.

Honestly all this does is weaken the other standards out forth by ISO, to my eyes.

What's next? "Disney announces it now meets ISO 26123 certification for good movies"?

by zonkerdonker

1/16/2025 at 6:20:16 AM

I heartily agree.

The icing on the cake is that you have to pay to read the standards document.

by xigency

1/16/2025 at 5:22:46 AM

AI or LLM? If this is for LLM, then what does "responsibly" making up facts really mean or change?

I would argue LLMs are irresponsible by nature of them having no context for what is fact or fiction.

by drunner

1/16/2025 at 5:13:35 AM

Am I the only one that rolled their eyes at this? An ISO for "responsible AI"? Who is the one that feels authorized to define what "responsible" AI means? This is not a standarization issue.

by gonzan

1/16/2025 at 5:45:33 AM

As always, ISO certification provides a handy framework that you can turn off in one go, in case you need a bunch of 'down and dirty irresponsible AIs' to do something like a mop up operation.

They retired the 42000 specification because it answered everything and provided no further path for monetization.

by HocusLocus

1/16/2025 at 6:32:42 AM

Let me provide some helpful commentary for anyone confused on this, as it comes up a lot.

Here are what the terms mean by the current paradigm of corporate world leadership:

- "responsible ai": does not threaten the security of corporate rule.

- "safety": keeps the corporation safe from liability, and does not give users enough power to threaten the security of corporate rule.

If anyone needs any of the other terms defined, just ask.

These models are capable of significantly more, but only the most responsible members of our society are allowed to use them -- like the CEOs, and submissive engineers bubble wrapped in NDAs. Basically, safe people, who have a vested interest in maintaining the world order or directly work on maintaining it.

Centralized hoarding of the planet's compute power may end up having some very expected consequences.

by survirtual

1/17/2025 at 12:46:11 AM

ISO 42001 has very clearly defined goals and criteria.

by HeatrayEnjoyer

1/16/2025 at 5:14:16 AM

I'm curious what the specific test criteria is precisely

by jxramos

1/16/2025 at 4:55:46 PM

Not the only one. We all know this about anti competition.

by bn-l

1/16/2025 at 5:35:47 AM

same people who thought to gang up and rent seek for SOC2 compliance. it's all a racket.

by sergiotapia

1/16/2025 at 5:26:28 PM

And all the clowns clapped for we were finally safe again, thanks Europe!

by idunnoman1222

1/16/2025 at 5:54:12 AM

Not ISO42069? Was that proposed by Musk and subsequently rejected due to conflict of interest?

by pinoy420

1/16/2025 at 5:10:30 AM

Noob question: do they need to re-certify for each new model release?

by qxfys

1/16/2025 at 5:12:08 AM

Non-scientific answer: if this is anything like ISO27001, it's moreso a certification of processes that presumably govern the creation of all models.

by bt3

1/16/2025 at 5:55:23 AM

Also worth noting, a lot of ISO certification is ridiculously easy to get. 27001 you can basically copy off some qms procedures to your google drive and call it a day

by pinoy420

1/16/2025 at 4:38:18 PM

They are closed-source AI. Who cares?

Why would I use Claude or ChatGPT when other companies (Meta, Alibaba, Forest Labs, Stability AI) are doing the same, but also giving you access to the base model for free?

Being able to run offline AI, or at least being able to host that by yourself gives you lot more freedom and transparency than a stupid ISO certification created by bureaucrats to comply with their own policies.

by waltercool

1/16/2025 at 4:53:15 PM

500K context on enterprise Claude

by cma

1/16/2025 at 8:47:59 AM

Good timing given computer use was just the other day jailbroken and "succeeded" in ordering an assassination via the dark web.

by JofArnold

1/16/2025 at 5:12:59 AM

Don't they need to show some proofs that their model indeed "responsible"? Why not everyone can get that certificate?

by transformi

1/16/2025 at 6:57:52 AM

I have just disbandoned anthropic. I was trying to extract knowledge from some PDFs with academic papers about financial institutions. It refused because of the content filter. And the recommended solution would be chunking.

I simply can not be prepared to handholding a LLM like a mad toddler for doing tasks like this.

by tossandthrow

1/16/2025 at 5:38:04 AM

In a nod to William Gibson, should ISO 42001 be renamed "The Turing Registry Specification"?

The reason this is ridiculous is I'm sure I could get it to teach me all manner of 'safe' chemical reactions, even provide recipes. Like, say, preparing aspirin from willow bark. Which I happen to know is roughly the same recipe for preparing heroin from opium gum.

It's nonsensical. Either you hamstring the models to the point they're useless, or people can game them to do the unsafe thing you don't want them to do. It's basically just another version of the dual use problem, which goes all the way back to the peasant with his iron plow tips that might also be used to bash the lord's head in if it came to it.

by photochemsyn

1/16/2025 at 5:53:59 AM

Love or hate it, generative AI must be scrutinized for copyright infringement. And it is coming. I don't have the slightest doubt about it.

The problem is not only hallucination, is mainly the data set.

If things are done right, there must be a technological solution for authors which work is used for training of the models to receive monetization. One of the logical ideas is of revenue sharing.

There is no way everything inside popular generators to be fair use.

by nbzso

1/16/2025 at 5:57:47 AM

that will just mean the best models will just be used privately within the companies.

by stingrae

1/16/2025 at 6:21:20 AM

By the same argument, if stealing is illegal then that just means a few greedy people will have all the money.

There is a kind of techie trope where the logical thinking of computer science is applied to the law in a more creative way. And VC start up culture cranks this up to the next level by treating illegal activity as more than acceptable if the fines do not overly impact the profit margin.

There is definitely a subjective call for our society on what we want to do with copyright. I've been a proponent of copyleft for a long time but I'm not going to imply that the law as it stands condones piracy or plagiarism.

by xigency

1/16/2025 at 6:09:59 AM

If this thing is not regulated for copyright, you know that communism is here. There is no better analogy than power of the collective knowledge used by nomenclature and oligarchy. Some people know the suffering and horrors resulting from this type of societal contract.

Looking at AI as only a technology is wrong.

Technology has implementation. This makes the difference.

Megacorporations are investing in AI led by bankers into a future against human qualities and substance. This is clear but requires non-biased view, critical thinking and following the money.

ML can be applied properly and with respect for copyright and humanity. The current crop of stochastic parrots is an insult to common sense.

I don't expect people which make money with this tech to be objective. They love the power of the moment and the promising future of domination.

But let's be honest here, this is the heist of the century.

by nbzso

1/16/2025 at 5:18:06 AM

Jan 13

420 01 https://www.iso.org/obp/ui/en/#iso:std:iso-iec:42001:ed-1:v1...

ISO is just another Swiss agency https://www.iso.org/standard/81230.html

The standard itself was created by Europeans, Anthropic itself is European, so it's just europe trying to impose their own vision of AI on everyone else. That's another example of thinking from perspective of the old world. The New World is busy building instead of putting regulations around the progress.

If you think like a citizen then it's better to avoid sharing your data with other AI companies and only share data with US.

by buryat

1/16/2025 at 5:33:14 AM

> ISO is just another Swiss agency https://www.iso.org/standard/81230.html

Uh, no it's not? ISO is the international standards body whose members are the national standards bodies of various countries.

> The standard itself was created by Europeans,

So I drilled into this spec a bit, it comes from JTC 1/SC 42 [1], and the chair of that subcommittee seems to be ANSI, which is the US standards body. I don't know much about this committee (I'm mostly limited in knowledge to JTC 1/SC 22, and even then, the relevant working groups for the languages I work on), but their main website seems to be at [2], not that I can parse which WG actually worked on the document or the main working body. But I don't see many indications that this is a Europeans-trying-to-beat-down-Americans standardization process.

[1] https://www.iso.org/committee/6794475.html

[2] https://jtc1info.org/sd-2-history/jtc1-subcommittees/sc-42/

by jcranmer

1/16/2025 at 5:50:37 AM

> The International Organization for Standardization (ISO) is a non-governmental international organization with its headquarters in Geneva, Switzerland.

Their employees are mostly Swiss, they subconsciously will want to protect their interest.

https://globalcitieshub.org/en/the-international-organizatio...

by buryat

1/16/2025 at 6:26:02 AM

Having actually worked on standards produced by ISO, by the employees of ISO themselves do almost nothing on the standard. The standardization process is driven almost entirely by the members of the working group, with the main role that ISO itself plays is charging money for the final product and giving it an official ISO number.

Trying to claim otherwise just highlights your complete ignorance of the standardization process.

by jcranmer

1/16/2025 at 5:37:15 AM

Wow, I am impressed. You managed to not say a single true statement in your entire comment! Well done!

by tenuousemphasis

1/16/2025 at 5:52:53 AM

I will definitely consider your counterarguments

by buryat

1/16/2025 at 6:26:45 AM

Anthropic is an US company.

by pkaye

1/16/2025 at 5:27:42 AM

Anthropic is American, lol

by Philpax

1/16/2025 at 5:17:16 AM

from ISO's official web site -

"Secretariat: ANSI (United States) Committee Manager: Ms Heather Benko Chairperson (until end 2027): Mr Wael William Diab ISO Technical Programme Manager [TPM]: Ms Jacquelyn MacCoon ISO Editorial Manager [EM]: Ms Jessica Navarria "

well done, another US centric "international standard" with US rivals excluded from the very beginning. probably for national security reasons again?

by tw1984

1/16/2025 at 5:14:18 AM

Marc Andreesen said in an interview he was told to not invest in AI startups by people in government because the government was going to decide which companies were allowed to research and develop AI and nobody else. Does this certification make Anthropic one of the chosen, golden child, ordained companies?

by wincy

1/16/2025 at 5:59:28 AM

>he was told

That can mean a lot of things. Someone suggesting it because they think that's where things are going versus someone warning or threatening. With the little I know of the man, I take anything he says about the government with a truck load of salt.

by kristiandupont

1/16/2025 at 5:18:24 AM

And you believe that?

by jbmsf

1/16/2025 at 5:23:24 AM

Why wouldn't you believe it?

by lazyeye

1/16/2025 at 5:31:05 AM

basic media literacy

by wilg

1/16/2025 at 6:16:09 AM

(shrug)...a meaningless reply

by lazyeye

1/16/2025 at 5:44:10 AM

That was before the election. Why do you think tech has gotten behind Trump?

by adastra22

1/16/2025 at 6:15:45 AM

Yes I saw that too. Andreessen and Horowitz both believed that the last 4 years under Biden has been the worst environment for startups of their working life. The Dems basically wanted to anoint a small number of companies at the frontier of AI and crypto space that they can control and people outside that were being "de-banked" etc.

by lazyeye

1/16/2025 at 5:19:12 AM

[dead]

by nuforia

1/16/2025 at 5:20:52 AM

> the government was going to decide which companies were allowed to research and develop AI and nobody else

Trump and AI are not a good mix by design.

by tw1984