7/28/2026 at 9:30:15 PM
Hey HN, Michael here, co-founder of Promptfoo and one of the people working on the Codex Security CLI at OpenAI.Thanks for checking this out and for flagging the auth issues. We just open-sourced it, and there's still plenty for us to improve. Expect the product to evolve quickly.
If you try it, I'd really appreciate hearing what works well and what you think we should improve. Happy to answer questions here.
CLI docs: https://learn.chatgpt.com/docs/security/cli
EDIT: If you'd like to help make this better, we're hiring: https://openai.com/careers/full-stack-software-engineer-cybe...
by dangelosaurus
7/28/2026 at 11:11:14 PM
> Have experience shipping production full-stack products across modern web frontends and backend services.I'm amazed that the requirements are so low (or at least this vague) for jobs at companies like these.
Has anyone else had the experience of going to an interview and feeling like you were never asked any qualifying questions?
All the questions were easy, your answers were straightforward, you "got them right", but then were not chosen?
I find on the other side, they're also left with dozens of people who "passed" and then it comes down to a pretty arbitrary decision on who gets hired (if we are talking external, no referral, etc.)
I wonder if they can make job descriptions highly specific to filter the shortlist faster and more effectively (to actually get a shortlist).
Anyway end rant. Cool job, hope you fill it.
by orangelimesoda
7/29/2026 at 4:31:46 AM
I interviewed with them and had a great experience despite it ultimately not being a good fit due to communication issues and a bad flu coming on in the final interview - they, and Michael especially, was very kind and understanding.by J_Shelby_J
7/28/2026 at 11:30:55 PM
This sounds like it’s just an app development role, not a security analysis position, so I’m not sure what your complaint is.by jameshart
7/29/2026 at 3:10:41 AM
> just an app development roleMaybe this is part of the problem
by orangelimesoda
7/29/2026 at 1:21:02 AM
Professional app development requires an understanding of security.by solid_fuel
7/29/2026 at 1:35:18 AM
I think the evidence contradicts you at this pointby corndoge
7/29/2026 at 5:54:31 AM
That's quite an indictment of the common app development practices. (Not that I disagree with your point...)And security is hard. Because it is by definition off the happy path, it is quite often at odds with MVPs and rapid release cycles. Then you add all the ways the users can use your product to attack/abuse others.
Any non-hobbyist app development does indeed require at least a decent understanding of security.
by bostik
7/29/2026 at 3:17:39 AM
So you don't need to know anything about credit cards for someone to enter it into an app? I don't get the take (is it bad sarcasm?)by orangelimesoda
7/29/2026 at 3:27:59 AM
To be fair, for the vast majority of cases, no you don't.It is extremely rare for companies to roll their own payment processing anymore, or even handle PCI scope at all.
by devmor
7/29/2026 at 3:31:50 AM
Credit card entry, I think you should know a few basics like don't put it in MongoDB?? Or nah? It's just like any other user data?How about a background check - can anyone take a user-entered DL and randomly Google stuff to see what they find?
Can I store your SSN in plain text in a text file? Why not?
The user had to upload their ID for IDV but I use Vercel. I guess I have to put it on S3. What should the bucket policy be for all these driver's license photos - there are so many???
by orangelimesoda
7/29/2026 at 7:39:05 AM
> Credit card entry, I think you should know a few basics like don't put it in MongoDB?? Or nah? It's just like any other user data?> How about a background check - can anyone take a user-entered DL and randomly Google stuff to see what they find?
> Can I store your SSN in plain text in a text file? Why not?
You wouldn't be touching any of those unless you work for a handful of providers where that's their whole business. Usually you add the dependency, use their widget, and that's it.
by lmm
7/29/2026 at 3:37:07 AM
> Credit card entryIf you're handling credit card numbers yourself, you're in a shrinking subset of developer roles. I've spent the last 7 years of my career working at payment processors, so I do handle that stuff, but the majority of my industry has built an infrastructure that makes it so most developers don't have to think about that.
To your other examples, not everyone on the team needs to know these things up front. Someone in the review process does, and eventually that knowledge gets disseminated and more people know it to carry it forward in their career.
by devmor
7/29/2026 at 3:45:56 AM
[dead]by redlimetea
7/29/2026 at 3:52:33 AM
To your last question about bucket policies, clearly you just need to scope it down: arn:aws:sts::*:assumed-role/trustme*/*(indeed that first wildcard means any account)
by oenton
7/29/2026 at 5:16:16 PM
Good thing they asked for people with experience of professional app development then.by jameshart
7/29/2026 at 2:44:40 AM
Not having clear, objective criteria enables arbitrary decisions (not against OP, I mean in general.. and in general I dislike this pattern a lot). On the extreme other end of the spectrum would be 100% objective criteria, and companies being forced to pick a random applicant that matches them. If they want only the best, they have to have high expectations, but be able to actually define them. You say "cultural fit", I say "corruption", let's blow this whole joint.by customguy
7/29/2026 at 2:47:44 AM
> You say "cultural fit", I say "corruption", let's blow this whole joint.It's not necessarily corruption. If there's no conflict between principal and agent, it's fine.
Just like when I sent my butler to go and buy a bottle of wine, he can make arbitrary choices, but that doesn't mean he's corrupt or going against my wishes. I trust his judgement, and since it's a repeated game our incentives are aligned.
by eru
7/29/2026 at 2:45:17 PM
If the company got any investment, than the principal has no interest in "cultural fit" and it's always corruption.by marcosdumay
7/29/2026 at 3:32:03 AM
It’s not corruption to simply hire the people you subjectively feel a preference for working for, instead of objective criteria. It’s not public tax funds that fuel salaries, it’s your own money. You get to spend it how you like.Yes, many jurisdictions have outlawed arbitrary discrimination against protected classes (eg race), which is an entirely different matter, and not what we are discussing here.
by sneak
7/29/2026 at 7:33:47 AM
Yeah, and it's not necessarily mobbing to only tell people you really like about your party. But where it occurs, people never admit it to themselves and justify it in such a way, so that justification is meaningless. All wars of aggression are called a defensive emergency measure. Hundreds and thousands and rarely would anyone say "we'll take this because we can and you're helpless". No matter how glaringly obvious it is, it's never admitted.In the same way, I can acccept "cultural fit" as a summary of things a person can describe, sure. But I think more often than not it's just a thought-terminating cliché. It can also just mean "I cannot verbalize my reasons and/or don't want to admit to them".
You can say if something fits only if you either can describe both sides in sufficient detail and where it wouldn't fit, e.g. a plug and a socket. But if it's dark, you barely see anything, and just have a "hunch", then "fit" doesn't even apply. It's like telling someone you won't let them through a door because they wouldn't "fit" anyway -- okay, so let them try, if they actually won't fit you don't need to read tea leaves and gate keep based on that.
Preferring to go with a more safe and familiar and obvious candidate, fine. But don't pretend it's because the others won't "fit".
Culture, in so far as it deserves the name, shapes the people exposed to or in it, as well as the other way around. E.g. if only people who fit the culture can work at a company, no company can exist in the first place, because for there to be a culture there need to be people there. So that leaves setting the culture in stone after it grew to a certain size, and only looking for more of the same, which also isn't great, but at least still honest.
If a culture is so brittle it cannot integrate people who aren't already a product of it, that may be a legitimate choice of the company, but my assessment to find that lame is also valid.
I feel the same way about immigration troubles, tangentially. We moan because people we don't actively try to get to know don't care for our rules which we don't enforce in a confident, but respectful manner. We basically require sterile, bland input because we have no immune system worth speaking of and no way to process and refine what comes in.
by customguy
7/29/2026 at 8:35:37 AM
I agree the phrase "culture fit" is weasel-wordy, but...> You can say if something fits only if you either can describe both sides in sufficient detail and where it wouldn't fit
Many human dynamics, including sexual attraction, love, and even just who will be fun or easy to work with, are dynamics we don't fully understand, and cannot fully specify. In all these cases "I'll know when it see it" is perfectly reasonable, and need not be hiding an untoward motive. Which is not say, ofc, that it can't be hiding such a motive. That happens too.
by jonahx
7/29/2026 at 8:33:51 AM
All fair points and I guess I would dislike it more if it hadn’t benefitted me more. In general, if I can get face-to-face with a human then I have a huge advantage, and any barriers to getting to that point are a net negative to me.At the start of my career I was under-credentialed, and had to rely on lax requirements to get myself in front of people. I might never have gotten anywhere if my first few employers hadn’t been willing to overlook a lack of degree or commercial (rather than open-source) experience, directly as a result of a funnel with a wide entrance.
by petesergeant
7/29/2026 at 6:13:11 AM
It depends whether your goal is to hire for specific knowledge (hence specific questions) or for overall mindset and abilities (hence broader questions where you are able to extract the way a person thinks)by gbalduzzi
7/28/2026 at 11:47:41 PM
If you didn't already know, jobs at highly competitive companies tend to have vague job requirements because they expect to be able to apply your raw intelligence to changing demands quickly. There's no point being hyper-specific about the exact software packages because that's not what they want. What they want is someone who, after talking to an interviewer for 30 minutes, leaves them with the thought "Wow, this person can do anything we need of them. They can probably tell us what we need too and take ownership of large projects. Hire!"by pertymcpert
7/29/2026 at 12:17:43 AM
Agree. Being smart, competent, and high in conscientiousness is more important than any highly specific “qualification”. It’s not about checking a bunch of boxes. If you have a track record of getting shit done, you’ll have something to contribute.by moscoe
7/29/2026 at 3:15:06 AM
> There's no point being hyper-specific about the exact software packages because that's not what they wantOkay. This makes it sound like they're more sophisticated but it seems more like they are less sophisticated, less specific, and a lot more vague in the job descriptions they themselves create.
If you look at any technical role, game dev or something where people are building important things at scale - there are a lot of specifics. Libraries, methodologies, where if you didn't know them you are nowhere near a fit.
I'm just wondering. It's OpenAI. Surely there is some domain-specific something beyond "has experience shipping front-end and back-end services" since that includes basically everyone.
It makes this job look like a Starbucks role.
by orangelimesoda
7/29/2026 at 3:32:37 AM
I see where you're coming from, but as someone who regularly interviews engineers, I don't care about specific tech stacks when evaluating a candidate very much either. I can only think of two positions I've worked in where such a thing really mattered.A good engineer can adapt and catch up without a lot of lead time. For a contractor, I'd be much more specific - but for someone who's going to join my team? I'm looking for a candidate that can demonstrate their problem solving ability, creative thinking and communication skills.
Other than having some kind of experience in the general domain we work in, those "soft skills" are far harder to find than specific tech experience.
by devmor
7/29/2026 at 2:10:10 PM
Yeah, no. If you are digging deep into database internals, query optimization and schema design, it helps to have someone with some experience in the domain. Otherwise your team will spend a few years learning from first principles.by a34729t
7/29/2026 at 4:12:04 AM
This is insane to me.Not every engineering job is entry level or as simple as most fullstack crud. Deeper into industry you find highly specific well defined positions for a given domain. Soft skills matter more the higher the ladder but id take a killer senior who can be difficult over a team of mediocre staff engineers.
by rustystump
7/29/2026 at 3:39:32 AM
[dead]by berrylimetea
7/29/2026 at 1:52:01 AM
This. Being too specific on the stack requirements is a red flag imo.by _superposition_
7/29/2026 at 2:19:44 PM
> raw intelligenceI'm not sure how this would apply. Are you implying that if the company operates on Python, you can hire someone with great "raw intelligence" who have only developed C++ all their life, and they can start contributing on day 1?
You need to clearly list what the position entails, otherwise you're wasting time.
by swat535
7/29/2026 at 3:15:46 PM
They are listing what the position entails. In this case it's a full-stack web dev role. "only c++ all their life" is more or less not possible if you have experience doing thatLikewise from another job post: "Have a strong background in kernel-level systems". Not possible if your whole resume is building web apps, may be possible for someone that has only used c++ professionally.
by yesb
7/29/2026 at 2:40:37 AM
I agree, I recently interviewed at Synthiolabs, and they asked me two questions: one about RAG and two about graph RAG. They rejected me even though I answered correctly, and the interviewer was also a college kid.by hacket04
7/28/2026 at 10:29:50 PM
This looks great, thanks for open-sourcing it!How does it deal with the current guardrails 5.6 Sol has on finding vulnerabilities? When I use it in the Codex app it would sometimes say it found a vulnerability, but it cannot tell me what it is.
by vladoh
7/29/2026 at 12:00:25 AM
Thanks! You've run into a real limitation: the CLI doesn't bypass the model's cybersecurity guardrails. If GPT-5.6 Sol finds a vulnerability but refuses to explain it, switching from the Codex app to the CLI won't automatically fix that.For authorized defensive work, Trusted Access for Cyber (TAC1/Daybreak) can reduce refusals depending on the model and the account or organization where access is provisioned. It isn't a blanket bypass.
If you're an open-source maintainer, you can apply for conditional Codex Security access here:
https://openai.com/form/codex-for-oss/
For enterprise teams, the public Daybreak onboarding guide is here:
https://help.openai.com/en/articles/20001261-enterprise-dayb...
If you have an example of "found a vulnerability but won't tell me what it is," I'd love to take a look too. You can send it to use with /feedback (or message me).
by dangelosaurus
7/29/2026 at 9:24:45 AM
> If you're an open-source maintainer, you can apply for conditional Codex Security access here:> https://openai.com/form/codex-for-oss/
Hey, Lead maintainer of vim here. Applied twice already never heard anything back. This is a frustrating experience!
by chrisbra80
7/29/2026 at 2:57:28 AM
>> For authorized defensive work, Trusted Access for Cyber (TAC1/Daybreak) can reduce refusalsOr perhaps a better option is to use something like Kimi K3 and cancel the GPT subscription altogether.
by theplumber
7/29/2026 at 5:44:11 AM
Or try Grok, 4.5 seems pretty capable, should be close to K3 in many coding tasks. I use it for code review of what other "stronger" models shit out (like Sol) and it constantly finds even pretty big bugs or just not robust enough solutions (Sol tends to overengineer, yes, but I'm not so sure it overengineers the right parts, so far my experience woth it has been mid. Except it understanding my drawings and collages and it being capable of far better frontend/design dev than 5.4 or even 5.5 was).by Culonavirus
7/29/2026 at 10:23:17 AM
Sounds like that's the only solution. I'm so sick of this safety nonsense I was going to switch from Anthropic to OpenAI because of it. I'm so disappointed to see it's just more of the same.Model finds a vulnerability in your code but "refuses" to tell you. Words can hardly express the sheer absurdity of it.
by matheusmoreira
7/29/2026 at 1:45:22 PM
It’s like they want to squeeze more money from you with the cyber crap…reminds me of all the DRM stuff around music distribution…by theplumber
7/29/2026 at 8:52:51 AM
> GPT-5.6 Sol finds a vulnerability but refuses to explain itI think it would be a good practice to refund the session cost in that case. Otherwise a customer just spent some money in order to get exactly nothing.
by egorfine
7/28/2026 at 10:48:54 PM
I tried it, it started a scan but stopped after hitting the rate-limit of my account. It gave up after just a minute of retrying (rate limits are tokens per minute, so... :P).It said "Partial output was kept at <...>", but I dont see a obvious way of picking it up in a new scan? (The failed run cost me ~$13)
by Quai
7/28/2026 at 11:28:41 PM
Yeah, you're right. A per-minute rate limit shouldn't kill a scan after a minute, and "partial output was kept" makes it sound like you can pick up where you left off. You can't yet, unfortunately. --max-cost can limit estimated spend, but we still need proper retries and resume. Sorry you spent $13 finding that out. Please send me an email and I'll help make it right.by dangelosaurus
7/29/2026 at 2:08:20 PM
Off topic: Just some thx and kudos to you guys. I used Promptpoo at the beginning of the year - it was exactly, what I needed, very much still a niche thing hardly anyone was using.I totally missed the acquisition - but well deserved. I am currently re-evaluating PF again for my upcoming project, and happy to see that it is more than simply thriving.
by _the_inflator
7/29/2026 at 2:42:06 PM
unfortunate typo lolby MattRix
7/28/2026 at 10:06:47 PM
Does it require hitting OpenAI's APIs or can one also stand up a local OpenAI compatible LLM endpoint?by strictnein
7/28/2026 at 10:10:41 PM
We are actively working on officially supporting this. Because it's open source it is pretty easy to point a coding agent at it now and switch out the model.by dangelosaurus
7/28/2026 at 10:20:29 PM
Exciting! Is there any open GitHub issue we can track?by ignoramous
7/29/2026 at 12:22:56 AM
Not yet, unfortunately. We only just opened the repo, and there isn't a public issue specifically tracking local or OpenAI-compatible endpoint support.The issue tracker is here: https://github.com/openai/codex-security/issues
If you open an issue with the endpoint or model you want to use, I'd be happy to follow up there.
by dangelosaurus
7/29/2026 at 2:40:18 AM
Thanks! https://github.com/openai/codex-security/issues/52by ignoramous
7/28/2026 at 9:49:45 PM
When would I use this over the plugin in codex? Which I think can be invoked from cli as wellby gizmodo59
7/28/2026 at 9:56:27 PM
The plugin, including when invoked through the Codex CLI, is great for scanning the repo you're currently working in. The standalone Security CLI/SDK uses the same scanner, but is built for running security across many repos over time: org-wide scans, historical results, deduplication, false-positive tracking, budget controls, and CI integration.We've been talking to hundreds of engineering and security teams, and their feedback is shaping what we build.
Like Promptfoo, our goal is practical tooling that fits into the workflows teams already have.
by dangelosaurus
7/28/2026 at 9:37:17 PM
Been watching your progress for a while, glad OpenAI have looked after you and the team and you still get to ship!by robotswantdata
7/29/2026 at 12:24:14 AM
Thank you, that means a lot. Being able to keep building practical, open-source security tooling was important to us.Really glad we got to ship this, and there's still a lot we want to improve in Codex Security and in Promptfoo!
by dangelosaurus
7/28/2026 at 10:41:32 PM
How does it fare against its own codebase?by 6thbit
7/29/2026 at 7:27:54 AM
Hi! Any chance you may have tangential positions opening in Zürich?by noname120
7/29/2026 at 12:43:56 AM
Why does this need an entirely separate repo instead of being a feature in the existing Codex project?by waterTanuki
7/28/2026 at 10:13:23 PM
> co-founder of Promptfoo and one of the people working on the Codex Security CLI at OpenAI.> Thanks for checking this out and for flagging the auth issues.
Offtopic, but this right here is why I don't believe any marketing around "great amazing models that one-shot everything and programmers are no longer needed".
You just have to look at what these labs routinely produce, and their own products.
Edit to respond to @simonw whose comment I saw before he retracted it ;)
This comment is tied directly to consistent continuous claims by the LLM labs. Their own products disprove their own claims, and it would indeed be nice if fewer people believed them :)
by troupo
7/29/2026 at 3:54:56 AM
[dead]by imrozim
7/29/2026 at 7:51:56 AM
Hi! Any remote internship for a high schooler? lolby sudo_cowsay