5/20/2025 at 1:44:51 AM
So one of their servers had a /heapdump endpoint that publicly served a heap dump of the server? This whole saga is out of control.This group didn’t really “publish” anything, though. They’re offering access to journalists through a request form. They’re also not saying how much actual message content they have because the 410GB of heap dumps makes for a bigger headline number.
by Aurornis
5/20/2025 at 1:49:31 AM
Can you imagine co-opting a trusted and secure (and free) bit of software and just making it worse at seemingly every turn?And charging for it?!
I’m not sure what is more embarrassing: to be the company or to be a user.
by mingus88
5/20/2025 at 10:34:46 AM
This is why Signal is so opposed to third-party apps (or forks) that connect to their service.If you want to keep the branding of Signal being the secure app, you need to make sure that all Signal users are actually using a secure version of Signal.
If an insecure fork (like this one) becomes too popular, most groups will have at least one member using it, and then the security is gone.
by miki123211
5/20/2025 at 3:37:39 PM
That was Apple's same reasoning for shutting down that iMessage client app. These leaks seem to justify their concerns.by pchristensen
5/20/2025 at 8:02:15 PM
Nah, that was to keep their users hostage and force them to buy a iPhone.by xandrius
5/20/2025 at 10:51:13 PM
This is a shallow dismissal of an argument that should be given more consideration.Sure, this is HN, we know one of the effects of locking the ecosystem and coloring in-system messages differently is to encourage people to be in the ecosystem.
At the same time, you ALSO need to consider that obviously there will be leaks.
Malicious/advertising apps will target the new messaging interface to gain more data on their victims, etc.
by fn-mote
5/20/2025 at 11:36:15 PM
Safe encrypted group chat with stangers is an oxymoron.Locking down a platform is not an acceptable solution to the above conundrum - it doesn't matter if the user is using an official device/app whatever if they are untrusted. They can always turn around and leak everything you say without any technical measures.
Should we have no security? No, if you want to color messages differently based on perceived platform, fine. This is just an illustration that no technical measures can replace the fundamental trust necessary in these types of situations.
by smaudet
5/21/2025 at 2:16:59 PM
Hm, my understanding is that TeleMessage archival works with iMessage in the same way it does with Signal.The third-party federation problem is real, but the vulnerability caused by TeleMessage isn't solved by removing federation.
by aesh2Xa1
5/20/2025 at 6:10:51 PM
If your product is a strong brand then that would make total sense.I believe the main criticism against Signal is that they should focus on getting widespread traction of secure messaging, and that perhaps the brand can be a relatively distant concern.
by xorcist
5/20/2025 at 12:02:36 PM
That doesn't seem to be a problem for protocols and having a single implementation can lead to bugs that defy spec yet cause no issues obviously.by calvinmorrison
5/20/2025 at 1:09:08 PM
But you're not branding or selling implementationsby ctxc
5/20/2025 at 1:10:23 PM
*protocolsby ctxc
5/20/2025 at 2:07:44 AM
Why would the company be embarrassed? The users (i.e. high level U.S. officials) did no due diligence. Of course a private company is going to take the easiest and cheapest route. If it goes bad, just shut down and spin up a new entity.Some speculate this was intentional intelligence gathering by the Israelis which is plausible too.
by hypeatei
5/20/2025 at 2:42:40 AM
> Some speculate this was intentional intelligence gathering by the Israelis which is plausible too.How does this make sense? If they were gathering data, why would they add a public download? Surely the Israeli officials would not want foreign powers to access this?
Per Hanlon's razor, I don't think this is attributable to anything other than incompetence.
by n2d4
5/20/2025 at 3:21:02 AM
Two things can be true at once. Them using their access to unencrypted messages for nefarious purposes and them being incompetent at the same time leaving that endpoint open.by barbazoo
5/20/2025 at 3:23:30 AM
There’s room for both sides of the razor. The heapdumpz could be there maliciously, but incompetently made globally accessible.by jojohohanon
5/20/2025 at 3:37:30 AM
From the Wired article: "The archive server is programmed in Java and is built using Spring Boot, an open source framework for creating Java applications. Spring Boot includes a set of features called Actuator that helps developers monitor and debug their applications. One of these features is the heap dump endpoint,"So the heapdumps being available is a Spring Boot feature so it does not appear to be malicious.
by pigbearpig
5/20/2025 at 2:53:50 PM
I'm the original author of the Spring Boot feature for heapdumps: https://github.com/spring-projects/spring-boot/pull/5670.It seems that users commonly misconfigure Spring Boot security or ignore it completely. To improve the situation, I made this PR: https://github.com/spring-projects/spring-boot/pull/45624.
When the PR was created in 2016, endpoints were marked as "sensitive" and, for example, the heapdump endpoint would have to be explicitly enabled. However, Spring Boot has evolved over the years, and only the "shutdown" endpoint was made "restricted" in the later solutions. My recent PR will address that weakness in Spring Boot when users misconfigure or ignore security for a Spring Boot app so that heapdumps won't get exposed by default.
by flarecoder
5/20/2025 at 7:23:52 PM
I don't get why 2+ years after Log4J we are still dealing with this from Java libraries developers.Your end users are not security savvy, they will never be security savvy and you need to protect them from themselves instead of handing them loaded handgun. This language more than most is filled with people punching buttons for paycheck.
- Signed, Angry SRE who gets to deal with this crap.
by stackskipton
5/20/2025 at 6:55:41 PM
In my opinion, the original sin of Spring Boot Actuator is allowing server.port and management.server.port to be the same. It makes it too convenient for developers to skip the security review that would be done for opening a non-standard port.I think it would be wise to either disallow the ports being the same, or if they are the same, only enable the health endpoint.
by testplzignore
5/20/2025 at 11:52:21 PM
I'm more of the opinion that developers will make smart choices, when motivated.Sure, punching buttons for money is a widespread issue in the industry, but devs also like convenience.
Security has the hard problem that it's infuriatingly difficult to troubleshoot (ever tried to write security policies for an app or figure out how to let an app through a firewall, or set of firewalls?), and there's a bit of a culture of "security by obscurity".
So it's kind of expected that this is the behavior...
Sure some people will really just not care, mistakes will be made, but secure defaults, easy to configure and simple to understand are features not often seen from security products generally. This is driven by poor motivations from security folk who want to protect their industry...
by smaudet
5/20/2025 at 5:07:11 AM
This feature must be explicitly enabled, it is not on by default nor by accident.by evrflx
5/20/2025 at 5:19:51 AM
huh, I sure seem to be needing to debug this a lot, I guess I'll just leave it turned on all the time that way I can say a few seconds next time. Larry Wall says one of the virtues of being a great developer is laziness!by bryanrasmussen
5/20/2025 at 6:00:05 AM
[dead]by szundi
5/21/2025 at 12:22:49 PM
Based on [1] it seems like one `management.endpoints.web.exposure.include=*` is enough to expose everything including the heapdump endpoint on the public HTTP API without authentication. It's even there in the docs as an example.Looks like there is a change [2] coming to the `management.endpoint.heapdump.access` default value that would make this harder to expose by accident.
Let's look for `env` next...
[1] https://docs.spring.io/spring-boot/reference/actuator/endpoi...
[2] https://github.com/spring-projects/spring-boot/pull/45624
by terom
5/20/2025 at 2:59:15 AM
I mean, it could theoretically have been to provide plausible deniability, but it seems extremely more likely to have been incompetence and carelessness (and if they were also sending everything to Israel, it was probably through some unencrypted ftp upload).by g-b-r
5/20/2025 at 5:05:10 PM
Imagine you ran a spy agency and you were infiltrating signal, Facebook, Google, aws, cloudflare, and so on.Would you have them make a secure back door that could only be intentionally designed, and potentially traced back to you?
Or would you just have them be incompetent in plausible, deniable ways?
Nobody’s getting shot for espionage because they chose log4j and it had the shell shock bug.
by michaelt
5/20/2025 at 3:23:14 AM
I mean, one doesn’t preclude the other. This could be an incompetent intentional intelligence gathering.by notpushkin
5/20/2025 at 8:10:53 AM
The Israeli would have made it secure so only them can access the data because knowing someone else's secret is worth something only when it's still a secret, if china, Russia and everyone can read the log of the American government it's worth nothing.by aucisson_masque
5/20/2025 at 2:26:47 AM
>Some speculate this was intentional intelligence gathering by the Israelis which is plausible too.Which does not bode well for the customers' counter intelligence abilities
by dylan604
5/20/2025 at 5:23:53 AM
> The users (i.e. high level U.S. officials) did no due diligence.But why would they? It's not their job. They have massive IT staff supporting them. "High level U.S. officials" are just executives; the pointy-haired bosses to the pointy-haired boss. Only difference is these wear little decorative pins over their breast pocket.
Every Fortune 500 company has dedicated IT staff for execs; someone you can call 24/7 and say "my shit's broke" and they respond "we just overnighted you a new phone".
These people couldn't even install an app on their MDM-controlled device, now the narrative has become we expect them to be making low-level IT decisions too?
Next week we'll be scrutinizing Pete Hegseth's lack of thoughts on rotating backup tapes.
by donnachangstein
5/20/2025 at 5:36:38 AM
> ... narrative has become we expect them to be making low-level IT decisions too?I think that's a misdirection.
The narrative is that:
a) they were using a compromised piece of software
b) they should not have been using that software - not (necessarily) because it was compromised, but because it wasn't US DoD accredited for that use case.
(I understand your point that these guys are not tech savvy, and do not need to be, but they should be regulation-savvy (clearly they either are not, or willingly broke those regulations), and they should be following organisational guidelines that presumably cover the selection and use of these tools types.)
by Jedd
5/20/2025 at 5:48:04 PM
Yeah, and the purchase approval process is in place specifically so that someone who knows what to look for has looked at it and verified that it's an acceptable configuration.This is the exact same problem as Clinton's blackberry enterprise server. Doing it right was hard and time consuming, so they ignored that and did what they wanted.
Only we should be a lot more demanding that our officials in 2025 have a better basic understanding of the importance of computer security than in 2005.
by da_chicken
5/20/2025 at 11:16:56 AM
> now the narrative has become we expect them to be making low-level IT decisions too?If their staff makes bad decisions, that’s their failure too.
We expect them to be ultimately responsible for what happens on their watch.
Was it Truman who said, “Woah, don’t bring the buck anywhere near me, it stops with my assistant”.
by nkrisc
5/20/2025 at 5:41:34 AM
It is too early to tell, but given that these people openly attack scientists and other experts (they don’t agree with), I wouldn’t be surprised if they ignored advise of their IT experts.by danieldk
5/20/2025 at 11:09:26 AM
It's not too early to tell, we knew from the beginning that the use of Signal (let alone its clone) was not authorised to be used for such communications.Yes, there's a fleet of people who are supposed to make such tech decisions. The people involved specifically went against those rules. The existence of a group chat using an authorised app is a violation on its own, adding a journalist to it is a violation on top of a violation.
Adding a journalist was accidental, but using such an app (despite it not being approved) is very intentional.
by input_sh
5/20/2025 at 5:54:46 AM
IT staff that knew it was illegal to provide them tools for a conspiracy were fired or silenced. So the only people left were their cronies, who instantly complied with their illegal request, to the best of the cronies' abilities. For such national failures, the buck has to stop at the very top, not on some IT monkey.This is typical for highly corrupt governments and autocracies, they crumble from within because the autocrats can't trust random, competent people so their inner circle becomes saturated with people who are selected on the basis of loyalty not competence, and these people end up making the most important decisions and running the country.
by cornholio
5/20/2025 at 6:06:47 AM
Would tend to agree with most of that, but I think the assertion is Petey needed to ask his IT leadership to do the due diligence before diving in, not that he needed to decide using his own depth of skills and experience.I assume he did and they said it was a bad idea - the memo they'd released a few weeks prior about Signal vulnerabilities seems to suggest a lack of faith in that approach - but he was already banging away on his phone with all the grocery reminders and definitely not battle plans he needs to keep pushing out. Which is also how it feels in the enterprise space these days.
Strange thing to see our bureaucracy start to behave like a corporation instead of the other way around.
by 3rdDeviation
5/20/2025 at 5:40:17 AM
Their massive it staff provides them with a way to communicate securely and they ignore it deliberately so that their communications are not preserved for history or for future court cases.by hristov
5/20/2025 at 6:33:41 AM
One man's low Integrity (in the "CIA triad" sense) of communications is another man's improved plausible deniability.by TeMPOraL
5/20/2025 at 8:03:17 AM
[dead]by bigbacaloa
5/20/2025 at 2:21:47 AM
The changes to the application are intentional by all parties because message archiving was required by law.by kube-system
5/20/2025 at 2:36:58 AM
Sure, but they were not required to be done incompetently and insecurely.by brookst
5/20/2025 at 4:14:49 AM
The fundamental concept of plaintext archiving (escrow) of messages from e2ee messaging apps is insecure by most definitions.They could have used user-custody public key cryptography, where the end devices have the pubkey of the customer, and archive only re-encrypted messages to TM that they can’t read.
That is not, of course, what they did. They just archive them in plaintext.
by sneak
5/20/2025 at 12:08:45 PM
I don't think it is. I can archive my own messages and E2E security on the messaging layer means I don't have to trust the operator of the messaging service to not read my messages because they can't. The choice of how I archive the messages is completely orthogonal to the choice of messaging platform security. I could choose to use an E2EE approach if I want but in that case it probably wasn't even desired as the point was to have these be archived for audit purposes. (Of course they are more secure options such as archiving to an audit key, but this is still orthogonal to the concern of the messaging protocol)by kevincox
5/20/2025 at 3:05:54 AM
Well, I suppose technically this /heapdump endpoint does satisfy that archive requirement.by _kb
5/20/2025 at 7:24:35 AM
User for sureby yapyap
5/20/2025 at 6:08:11 AM
(read with sarcastic tone) But hey, this is a 'lite' version or a 'red' version (icon is red) or a 'purple' version (icon is purple), so I am cooler that then others that have the standard.I haven't used WhatsApp for 'a very long time' as I have exited the FB ecosystem, but back in the day I remember seeing "lite" or "WhatsApp+" or other variations of the software. I wouldn't be surprised that those "lite" or "+" come with baggage.
by HenryBemis
5/20/2025 at 1:41:25 PM
> They’re also not saying how much actual message content they have because the 410GB of heap dumps makes for a bigger headline number.That's very important to say. I went through one of these massive data dumps recently and it was literally all cached operating system package updates and routine logs. Nothing at all of interest.
It's easy to cut the size on a heap dump. When it's not done it seems sketchy. But it could be a 512GB dump and already pruned, so I could be wrong.
by BearOso
5/20/2025 at 5:06:42 PM
Most of the the heap dump will be filled with stuff like java.util.String!blahjava.util.ArrayList!Though the heap dump would have messages in flight at the time. It's obviously not as useful if you are just trying to grab messages for a specific person.
Frankly the most useful part might be any in-memory secret keys, which could be useful for breaking deeper into the system.
by harrall
5/20/2025 at 1:47:45 AM
Aren’t those Israeli software companies all supposed to be top notch, ex Mossad, yadda yadda? Doesn’t sound like it.I hope the message dump is juicy.
by barbazoo
5/20/2025 at 3:42:48 AM
And SBF of FTX fame was ex-Jane St so obviously was a serious finance professional. This is why using past employers as a shorthand for capability is unwise.by msy
5/20/2025 at 8:55:34 AM
In fairness, FTX had a profitable bankruptcy [1]. So it's still better to be scammed by Jane Street alumni than to be scammed by the usual alumni of Goldman Sachs, JP Morgan etc[1] https://www.bloomberg.com/news/articles/2024-05-15/ftx-bankr...
by sillystu04
5/20/2025 at 7:17:25 PM
It's not profitable. They are getting their money back from value of the assets in 2022 when they went bankrupt but most of crypto assets have gone up significantly in value so it's 2.5 years of lost profit.by stackskipton
5/20/2025 at 10:42:38 AM
How is that fair? It was luck from the AI investment. Pure luck.by coolcase
5/20/2025 at 7:35:22 PM
Regardless of how you feel about SBF and FTX, claiming an early investment into Anthropic is "luck" rather than being ahead of the curve feels off the mark.by fredoliveira
5/20/2025 at 7:54:27 PM
That is dodging the point. The guy ripped people off. By luck they got the fiat value of their investment at some past date back. Yes if a single investment pays off well enough to negate fraud losses on that scale over a short time scale. It's fucking luck.by coolcase
5/20/2025 at 12:28:32 PM
It wasn’t the only smart investmentby bn-l
5/20/2025 at 2:39:41 AM
I thought Israel has mandatory military service, so ex-mossad or ex-military signals intelligence doesn't really say much? Presumably they're directing people based on their skill set, so you'd expect most hackers to end up in mossad for their mandatory service.by gruez
5/20/2025 at 12:00:02 PM
> Presumably they're directing people based on their skill setBig presumption.
If I were israeli, there’s no way in hell anybody with half a brain would want me near their spy agency.
When a gov is committing a genocide, their decisions are based on control and fear, not getting the best out of people.
Edit: downvote all you want. Israel is still committing a genocide. No hospitals left standing. Killing aid workers, journalists, and doctors. A million people on the brink of starvation. Literally salting the earth to prevent crops from being grown. That is war crimes, ghettoization, and genocide.
by kennywinker
5/20/2025 at 2:03:13 AM
That's not a great generalisation for the whole country. How many ex Mossad people interested in doing actual implementation in tech companies do you think there are? It's like "aren't those US software companies all supposed to be top notch, ex NSA yadda yadda?"by viraptor
5/20/2025 at 2:12:14 AM
They do start a lot of tech companies specifically: https://en.wikipedia.org/wiki/Unit_8200#Companies_founded_by...The US only has voluntary military service, so the dynamics are different
by conradev
5/20/2025 at 3:39:18 AM
The CEO/Founder of TeleMessage Guy Levit was the head of the Planning and Development Department of an elite technical unit in the Intelligence Corps of the IDF according to bio.by lysp
5/20/2025 at 3:38:41 AM
One problem that smart people tend to make is in thinking that being really smart in one area is generalizable to all others. Just because they're good at AppSec doesn't mean they're good at networking or operating a webserver.by oceanplexian
5/20/2025 at 4:50:33 AM
I agree with this. It's surprising how often I encounter people with that belief, because I was disabused of it very early on in my career; this industry is chockablock with people who are brilliant in 1 area and deficient in others.by ripley12
5/20/2025 at 10:45:46 AM
That's why you need teams. Red team for example! Security team. App developers. Code reviews. You need all the process too. Security that relies on one genius is fragile.by coolcase
5/20/2025 at 12:26:49 PM
Aka "halo effect"by czl
5/20/2025 at 5:21:45 AM
That sounds more like a stupid person than smart lolby karn97
5/20/2025 at 9:13:28 AM
you can be smart in one area and stupid in others. the "not knowing you're stupid in others" is part of the "stupid in others".by stefs
5/20/2025 at 9:19:24 AM
I'm not sure why you'd expect intelligence agency types to be particularly good at engineering, tbh.by rsynnott
5/20/2025 at 11:35:09 AM
Spooks in general like to project a veneer of competence, downright invincibility. Entertainment media, journalists, experts play a big role in this. And by and large it works.It’s especially true for spooks of a certain entity. Also, it’s easy to confuse brazenness, being protected from consequences, and usually downplayed or secret Western complicity with competence.
by rainworld
5/20/2025 at 11:45:23 AM
I mean, I'm sure they're competent in some stuff, but being competent in one field doesn't generally mean being magically competent in _all_ fields.by rsynnott
5/20/2025 at 3:37:11 PM
I'm not sure about this case, but maybe the assumption here is that these are people from a technical branch of Mossad, such as Unit 8200, which does SIGINT. I've interviewed 3 of them for your typical Big Tech SWE position, and to a candidate, they were very strong engineers. I never got to work with them, however, because they always got better counteroffers...by keeda
5/20/2025 at 9:52:57 AM
> Aren’t those Israeli software companies all supposed to be top notch, ex Mossad, yadda yadda?Working with a few companies like these, I can tell you that the marketing is top-notch, and very aggressive. The products not so. Most get better with time.
by ExoticPearTree
5/20/2025 at 5:33:25 AM
"All supposed to be".This is a country of 10 million people, a rather heterogeneous one at that. There are going to be better and worse companies.
by underdeserver
5/20/2025 at 7:30:26 AM
They are top notch - at working for profit and for the interests of their country.by H8crilA
5/20/2025 at 6:32:14 AM
After all the concern over China and TikTok, why is the USG using a foreign chat program at all?by treebeard901
5/20/2025 at 10:47:37 AM
SuperPAC and other corruptionby coolcase
5/20/2025 at 10:40:31 AM
Yeah the /leakitbaby endpoint was meant for just them, not the world! Doh!by coolcase
5/20/2025 at 9:36:31 PM
It only takes one guy doing one stupid thing to have a security incident. Yeah, processes should be in place, but no process is perfect.by elzbardico
5/20/2025 at 1:54:55 AM
[flagged]by Calwestjobs
5/20/2025 at 2:30:07 AM
This article doesn't mention Mossad, though. Do you have any other sources?by basilgohar
5/20/2025 at 6:36:51 AM
yes, Shin Bet : https://en.wikipedia.org/wiki/Yigal_Amirby Calwestjobs
5/20/2025 at 3:14:19 AM
[flagged]by MPSFounder
5/20/2025 at 2:52:35 AM
Sounds like someone had a Java app and mistakenly exposed all of the JMX endpoints over HTTP. It's not the default configuration, and likely done out of carelessness.by jfim
5/20/2025 at 3:39:50 AM
From the Wired article, it may not have even been a mistake, depending on the version of Spring Boot."Spring Boot Actuator. “Up until version 1.5 (released in 2017), the /heapdump endpoint was configured as publicly exposed and accessible without authentication by default."
by pigbearpig
5/20/2025 at 8:46:07 AM
This sounds utterly insane. Is Actuator a standard part of Spring Boot or is it an optional package of some kind?by davedx
5/20/2025 at 9:11:26 AM
Imaging putting up a firewall to mitigate this, then docker compose helpfully opening the ports for you. Security comes in layers.by teekert
5/20/2025 at 1:51:58 PM
This feature of docker compose is insane.by callamdelaney
5/20/2025 at 5:57:25 PM
Right!? I learned with a colleague: Didn’t you restrict everything to the Tailnet? Yes, feel free to check UFW. Hmm, then why does nmap show all this stuff when scanning from the lan? Wtf??by teekert
5/22/2025 at 10:54:09 AM
Similar here, UFW setup to only enable access via Caddy to our http services - wait, why can I connect directly to our redis instance?Took a while to workout that for some reason docker-compose is messing directly with iptables to shoot holes in the firewall we'd configured. Figured out you have to write your compose in some super special way to disable that functionality. Compose should never ever open network ports, ever in my book - to do so without a warning or anything though is like I said, insane!
by callamdelaney
5/20/2025 at 6:31:26 AM
This was also part of the exploit chain in the "Volksdaten" incident.by formerly_proven
5/20/2025 at 2:57:01 AM
Or intentionally. There could be an APM agent which just lets you run heap dumps any time you want, or they enabled heap-dump-on-crash, or had a heap dump shutdown hook, etc. There's a lot of ways to trigger dumps. If we're talking about a full dump, and the apps were using most of the memory allocated to their container/VM/etc, 410GB is actually not that many dumps (we're probably talking uncompressed). At 4GB/dump, that's around 100, over possibly several years.I just wonder where they were storing them all? At one place I worked, we jiggered up an auto shutdown dump that then automatically copied the compressed dump to an S3 bucket (it was an ephemeral container with no persistent storage). Wonder if they got in through excessive cloud storage policies and this was just the easiest way to exfiltrate data without full access to a DB.
by 0xbadcafebee
5/20/2025 at 1:23:34 PM
Is this a heapdump of servers or of clients? I can imagine that might have been intended as a place for crashing clients to logby trebligdivad
5/20/2025 at 2:11:40 PM
TeleMessage is most likely an intelligence asset, and a burned one now that Trump's people stopped using it. A fake hack is the safest way for the agency responsible to leak the messages collected.by kleton
5/20/2025 at 7:33:14 AM
if a heap dump is a copy of all the bytes in memory, then wouldn't "thousands of heap dumps" likely be larger than 410GB?napkin math:
410GB/1000 dumps = 410MB per dump?
410GB/2000 dumps = 205MB per dumpby kbouck
5/20/2025 at 10:45:45 AM
Might be filtered somewhat, like extracted all ASCII text then compile that into the dump, rather than just the raw dump files.Edit: reading the description on the dump again, seems exactly what they did:
> Some of the archived data includes plaintext messages while other portions only include metadata, including sender and recipient information, timestamps, and group names. To facilitate research, Distributed Denial of Secrets has extracted the text from the original heap dumps.
by diggan
5/20/2025 at 10:49:01 AM
Kubernetes pods?by coolcase