3/26/2026 at 4:14:56 PM
by tanelpoder
3/26/2026 at 7:25:28 PM
I wonder how this compares to grafana pyroscope, which is really good for this sort of thing and already quite mature:by SEJeff
3/26/2026 at 7:59:50 PM
As far as I'm aware, Pyroscope itself is not a profiler, but a place you can send/query profiles. OpenTelemtry is releasing a profiler, so they don't compare. One can be used with the other.by ollien
3/26/2026 at 5:11:33 PM
Relatedly: Has anyone profiled the performance and reliability characteristics of rsyslogd (Linux and FreeBSD distributed syslogger, maybe other platforms too) in its mode where it’s shipping logs to a central node? I’ve configured and used it with relatively small (high single digit nodes, bursts of activity to a million or two requests per minute or so) set-ups but have wondered if there’s a reason it’s not a more common solution for distributed logging and tracing (yes it doesn’t solve the UI problem for those, but it does solve collecting your logs)Like… has anyone done a Jepsen-like stress test on rsyslogd and shared the results? I’ve half-assedly looked before and not been able to find anything.
by genthree
3/26/2026 at 7:38:10 PM
We're doing this with a few dozen GiBs of logs a day (rsylog -> central rsylog -> elasticsearch). It works reliably, but the config is an absolute nightmare, documentation is a mixed bag and troubleshooting often involves deep dives into the C code. We're planning to migrate to Alloy+Loki.by jbaiter
3/26/2026 at 6:50:04 PM
People don’t care about syslog. 98% of my colleagues haven’t heard of it.by nesarkvechnep
3/26/2026 at 7:28:28 PM
You are drawing a global conclusion from a tiny sample!by malux85
3/26/2026 at 6:47:26 PM
Very excited for this. We've used the Elixir version of this at $WORK a handful of times and have found it exceptionally useful.by ollien
3/26/2026 at 4:48:59 PM
> Continuously capturing low-overhead performance profiles in productionIt suprises me that anything designed by the OTel community could ever meet 'low-overhead' expectations.
by secondcoming
3/26/2026 at 5:19:30 PM
The reference implementation of the profiler [1] was originally built by the Optimyze team that Elastic then acquired (and donated to OTEL). That team is very good at what they do. For example, they invented the .eh_frame walking technique to get stack traces from binaries without frame pointers enabled.Some of the OGs from that team later founded Zymtrace [2] and they're doing the same for profiling what happens inside GPUs now!
[1] https://github.com/open-telemetry/opentelemetry-ebpf-profile...
[2] https://zymtrace.com/article/zero-friction-gpu-profiler/
by tanelpoder
3/26/2026 at 5:13:55 PM
OTel Profiling SIG maintainer here: I understand your concern, but we’ve tried our best to make things efficient across the protocol and all involved components.Please let us know if you find any issues with what we are shipping right now.
by felixge
3/26/2026 at 4:52:42 PM
Anything to actually add?by phillipcarter