alt.hn

8/17/2026 at 5:54:59 PM

GPU Offload in Rust: Portable, Safe, and Fast

 https://arxiv.org/abs/2608.13759

by linggen

8/17/2026 at 9:20:18 PM

> This module is under active development. Once upstream, it should allow Rust developers to run Rust code on GPUs. We aim to develop a rusty GPU programming interface, which is safe, convenient and sufficiently fast by default. This includes automatic data movement to and from the GPU, in a efficient way. We will (later) also offer more advanced, possibly unsafe, interfaces which allow a higher degree of control.

I really appreciate the work and the effort that went into this. However, such an approach has previously not really worked for C++ with LLVM offload. Why would it work for Rust?

by whateverboat

8/17/2026 at 10:19:48 PM

> However, such an approach has previously not really worked for C++ with LLVM offload. Why would it work for Rust?

They're very different languages, with different semantics. Without reading more than the synopsis of the paper, they're 100% leveraging the substructural type system and will have a really tight requirement for you to use a certain kind of Rust code at the CPU/GPU boundary.

by ux266478

8/18/2026 at 5:42:46 PM

To answer my own question apparently,

With C++ the last time, the problem was that there was no MLIR and it's dialects. So, the lowering could not take advantage of any vendor specific features like CUDA graphs and co-operative groups. That has apparently now change with MLIR being a possibility, but with C++, it is still in the experimental phase with CIR not finalized yet, even though people are working quiet hard on it.

With Rust, I am not sure if Rust lowers to MLIR, but if it does, then things should work otherwise not. However, knowing that all GPU vendors are working very hard on CIR and lowering to MLIR, and seeing how much time it is taking them, I would be surprised (pleasantly so) if Rust --> MLIR --> Backend would be faster.

Nvidia has started experimenting with Native Rust so it is still a possibility but still a pleasantly surprising one.

by whateverboat

8/18/2026 at 7:11:36 AM

Unfortunately, the Rust description itself is inconsistent. It claims to be "sufficiently fast by default", yet "sufficiently fast" depends entirely on the requirements of a specific user project. And then it also plans to provide options that do not guarantee memory safety when the default speed is insufficient. It is already common for Rust projects to sprinkle memory unsafe code around when performance is needed.

by erupti

8/18/2026 at 9:45:16 AM

> It claims to be "sufficiently fast by default", yet "sufficiently fast" depends entirely on the requirements of a specific user project.

I think that's why the "by default" is there; the goal is to offer a safe/convenient API that performs well enough that by default you don't need to reach beyond said safe/convenient API. And if you happen to be in a situation where the default performance of the safe/convenient APIs is insufficient, more advanced APIs will be provided.

It's a mirror of Rust's general design goals, if anything.

by aw1621107

8/18/2026 at 1:42:25 PM

But it still depends on the requirements of the specific user project, whether the default is sufficiently fast, ESL. It is still entirely inconsistent. Basic logic 101, clear as day. Why do you even try to contest this?

> It's a mirror of Rust's general design goals, if anything.

Do you claim that "it's a mirror of Rust's general design goals, if anything" to be forced into memory unsafe, extra-difficult Rust code being required, when the going already gets tough regarding performance requirements? That is not a great design goal, and it is not conducive to memory safety nor high quality software.

by erupti2

8/20/2026 at 2:44:44 PM

Could you please stop creating accounts for every few comments you post? We ban accounts that do that. This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html.

You needn't use your real name, of course, but for HN to be a community, users need some identity for other users to relate to. Otherwise we may as well have no usernames and no community, and that would be a different kind of forum. https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...

by dang

8/18/2026 at 2:34:49 PM

> But it still depends on the requirements of the specific user project, whether the default is sufficiently fast

I mean, "by default" means "by default", not "in all cases". Situations that the default doesn't address are not inconsistent with the existence of a default; it's just that said situations are expected to be a relative minority.

Perhaps a more concrete example would help: say the API the devs come up with is sufficiently fast enough for 99 common use cases and not fast enough for 1 uncommon one. I don't think it'd be inconsistent to call said API "sufficiently fast by default" since "sufficiently fast" is an accurate description of the API for "normal" use (i.e., that's the "default" state).

> Do you claim that "it's a mirror of Rust's general design goals, if anything" to be forced into memory unsafe, extra-difficult Rust code being required, when the going already gets tough regarding performance requirements?

No, I think that's a rather... imaginative interpretation of what I said.

by aw1621107

8/18/2026 at 10:54:27 AM

I wonder if they're looking to achieve easy speedups. I can see lots of value enabling performance gains where normally people wouldn't bother because it's too much effort. I don't think this will take away work from those who hand-optimise their kernels and scheduling, this is to enable GPU acceleration for those, who otherwise wouldn't.

by eptcyka

8/17/2026 at 9:48:19 PM

> However, such an approach has previously not really worked for C++ with LLVM offload. Why would it work for Rust?

I think that will depend on the exact reason(s) C++ with LLVM offload didn't work out? If Rust differs from C++ in a way that addresses pain points/failure modes/etc. from the C++ attempt, for instance, then perhaps it isn't unreasonable to think Rust could succeed where C++ didn't (c.f., Mozilla's pre-Rust attempts to parallelize Firefox's CSS styling engine). Inversely, if Rust doesn't do things differently in the right way perhaps one might expect the effort to also not work out. Or maybe the problems are entirely non-technical and things could work out in either language.

by aw1621107

8/18/2026 at 7:14:27 AM

[flagged]

by erupti

8/17/2026 at 11:05:48 PM

Seems to work out well for Mojo, so I'd guess it's more an issue with C++.

by boxed

8/18/2026 at 11:47:04 AM

Well mojo handles it by introducing an additional step of lowering code to MLIR as an intermediate representation. Something that can be done with C++ as well

by nylonstrung

8/18/2026 at 2:08:13 PM

In fact, clang is in the process of adding an MLIR layer as well, ClangIR.

by pjmlp

8/17/2026 at 11:22:01 PM

> However, such an approach has previously not really worked for C++ with LLVM offload

... isn't Metal shading language just C++17 compiled with LLVM ? working on every Mac and iPhone in the world is not what I would call "not really worked". Likewise, SYCL works just fine.

by jcelerier

8/17/2026 at 11:48:03 PM

bruh lol this is so wrong and so confident i don't want to even attempt to explain how wrong you are.

> just C++17 compiled with LLVM

i invite you to attempt to compile/run absolutely any C++17 codebase on your iphone's GPU lol!

by mathisfun123

8/18/2026 at 7:28:26 AM

For starters Metal Shading Language is based on C++14, so naturally any C++17 will fail.

by pjmlp

8/18/2026 at 1:20:08 AM

I mean "any C++ codebase" doesn't make sense in general. I run C++23 code on ESP32, that doesn't mean I'm gonna build KDE or chrome for it, and that doesn't make it any less C++.

by jcelerier

8/18/2026 at 12:55:26 AM

Wouldn't any metal app have to be written in C++17?

by nish__

8/17/2026 at 10:52:22 PM

[dead]

by winningChild

8/17/2026 at 10:52:47 PM

[dead]

by winningChild

8/17/2026 at 9:46:03 PM

So... why go through LLVM at all instead of having the MIR target PTX/HIP C directly then?

If they really wanted a vendor neutral solution for Rust GPU, that already exists: you write the CPU side code, including buffering, allocation, concurrency, etc through Vulkan binding and consume the compute kernel in SPIR-V from HLSL/GLSL/WGSL etc. As it stands, the way they use Rust here feels more like using it like TypeScript types/interfaces than anything else.

Again, the size of most operations that should be done on the GPU is known ahead of time before compilation, so it's very much possible to statically allocate memory at compile time instead of going through all this trouble to write what's essentially a Rust shaped DSL for GPU compute.

by YuechenLi

8/18/2026 at 12:34:29 AM

Because it's convenient? Shader and vulkan semantics can be quite limiting and annoying to write.

Maybe it doesn't matter in a post AI world but perhaps it will allow better abstractions.

No need to yuck someone else's yum.

by minraws

8/18/2026 at 7:51:09 AM

The compute shader side of Vulkan is actually fairly smooth compared to graphics. It's not really that different from low level CUDA or OpenCL. The Vulkan complexities overwhelmingly concern rasterization and raytracing.

by gmueckl

8/17/2026 at 10:37:20 PM

People go through trouble to write Python-shaped DSL for GPU compute. We will go "why o why?", but apparently such things are necessary to succeed in the market.

by sanxiyn

8/17/2026 at 11:03:07 PM

Well, I suppose fake Python is better than fake C++ at least.

But yeah, I think Python's dominance in science/ML will eventually pass, just as FORTRAN and Matlab did before.

by YuechenLi

8/17/2026 at 10:42:13 PM

I write all my code in Rust because I am a Rustacean. In many of my custom LLM inference engine projects, the biggest fight has always been bindings. I don’t want to maintain and write bindings; also, if I use an existing project that provides bindings, then I have to wait for the owner to update or fork it and then maintain it on top. It has been a big headache. Running Rust core on GPU sounds like something I will try from day one. Kudos to the team and will watch it closely.

by bicepjai

8/18/2026 at 1:51:33 PM

> I write all my code in Rust because I am a Rustacean.

Why is Rust the only language to have literal acolytes? It's as if people believe they are part of some collective computer religion ushering in the messiah. Weird, man.

by MisterTea

8/18/2026 at 2:19:45 PM

Hahaha, I have heard that argument. I generally love different programming languages. If you have written code a lot in C/C++ and have battle scars with security and null pointer handling, then Rust is definitely a breath of fresh air. It makes you feel why did I have to be even in that fight? It’s like driving stick your whole life in heavy traffic, and someone just handed you an automatic :) you didn’t realize how much mental overhead that was until it was gone.

Respect for addressing something that fundamental at the design level. If something deserves praise, it’s okay to praise it. Give it a try before writing it off as religion.

by bicepjai

8/18/2026 at 8:18:16 PM

> If you have written code a lot in C/C++ and have battle scars with security and null pointer handling, then Rust is definitely a breath of fresh air.

Whats new is old. Ada was that breath of fresh air for me. Safety through a type system with built in concurrency since 1983. Spark gives you provable safe code.

> It’s like driving stick your whole life in heavy traffic, and someone just handed you an automatic :)

Eh, depends on the vehicle (weight, gearing, tires). My 2006 Civic was a dog in traffic but my 2002 Pathfinder was a breeze.

by MisterTea

8/18/2026 at 2:35:38 PM

The longer you starve the more delicious the food tastes. Programming language advances were ignored by a significant number of folks for decades. The sudden relief of getting something like rust can leave a strong impact.

People also like community and rust has a decent one. It's not wrong for folks to want to feel part of a community they have shared interests or values with.

by surajrmal

8/18/2026 at 8:23:36 PM

It's cute you think that Rust is the only language that has people who are whole converts.

by stouset

8/18/2026 at 9:42:33 PM

Show me the proof.

by MisterTea

8/19/2026 at 10:02:44 AM

The onus isn't on them to correct your incorrect generalizations. It is on you as a person to guide yourself. You can do it, I believe in you.

by explodes

8/20/2026 at 12:32:47 AM

Have you never heard of Pythonistas, or Ruby fans, or Perl fans back in the day, or the how Java (plus a healthy dose of XML) were going to be the One True Language?

by pezezin

8/18/2026 at 9:11:19 AM

Ownership tracking should map well to GPU memory lifetimes. That's one place Rust has a real edge over C++.

by Driftbench

8/17/2026 at 9:12:59 PM

is this mainly about making host binaries self-contained for heterogenous workloads?

also, seems like this is mostly targeted towards HPC audience?

by boywitharupee

8/19/2026 at 8:57:44 AM

> multi-vendor GPU compilation framework

Technically true, since it supports NVIDIA and AMD.

But we have a different definition of portability, if I cannot bring a Metal device and expect it to work.

by Archit3ch

8/18/2026 at 2:34:55 PM

I’ve been having success doing this with AdaptiveCpp (formerly OpenSYCL) bindings in Rust to get PG OLAP and geospatial workloads to run on a metal/cuda gpus - pure rust the whole way through sounds really nice!

by contrahax

8/17/2026 at 8:27:39 PM

does anyone know Mojo well enough to comment how Rust + gpu-offload compares to it?

by maxchisto

8/17/2026 at 8:29:58 PM

Mojo is not fully open sourced yet, but it will eventually be, would be an interesting comparison though.

by giancarlostoro

8/17/2026 at 8:47:12 PM

Mojo's OSS status doesn't prevent us from evaluating its memory model, writing and benchmarking kernels in it, etc

by maxchisto

8/17/2026 at 8:49:36 PM

Sure, and I realized after I posted the std lib is opened up, not sure how much of it will reveal the underlying Mojo specifics though.

by giancarlostoro

8/17/2026 at 8:00:12 PM

> the rust-gpu project has to emulate pointers[8], which we consider a blocking issue for most HPC benchmarks.

Why is it a blocking issue?

I feel like this is very aligned with the goals of rust-gpu.

by jasonjmcghee

8/17/2026 at 8:44:41 PM

Pointers are sort of needed for high performance memory management for HPC targets for existing design patterns, maybe we can think of better solutions down the line but it's hard for me to say anything I just use/abuse CUDA pointers as well.

by minraws

8/17/2026 at 8:49:45 PM

Julia has pretty good design heritage for how to deal with this sort of thing. you build the right abstractions and everything works (the main key is making sure the compiler elides bounds checks)

by adgjlsfhk1

8/18/2026 at 9:22:27 AM

[flagged]

by Arsen-V

8/18/2026 at 10:28:29 AM

[dead]

by andreypk

8/17/2026 at 8:34:02 PM

The NVIDIA+AMD support is the part I find really interesting. I know OpenMP and SYCL can already target multiple GPU vendors, but doing this while keeping Rust's safety model seems pretty compelling. I'm curious how portable the performance is in practice.

by Alephinitesimal

8/17/2026 at 8:14:28 PM

[dead]

by jheriko

8/17/2026 at 7:14:48 PM

[flagged]

by iberator

8/17/2026 at 7:17:56 PM

Genuinely curious what these constant changes are. In fact I don’t feel they’re moving fast enough to give us improvements to core.

by aabhay

8/17/2026 at 7:28:59 PM

It was changing quickly in the earliest releases, about 10 years ago. Like one day I pulled our repo and there was new "?" syntax, but that was a feature I'd been wanting anyway.

Edit: Oh, async/await was a bigger and more recent one, 2019. I've heard that this wasn't an easy decision for them but was kinda needed.

by frollogaston

8/17/2026 at 7:44:18 PM

A few years in between releases with a sane versioning system seems fine to me. If you don’t want new features don’t build with that new toolchain.

It’s not javascript where you need to support all possible browsers.

by aabhay

8/17/2026 at 7:55:22 PM

Teams will disagree over what toolchain to use, and you will read others' code, so this doesn't dodge the issue. Otherwise there'd be no complaint about C++. I don't think Rust is bloated though, every feature has a very good reason.

by frollogaston

8/18/2026 at 12:33:51 AM

Would they disagree, new toolchains work great with old code? And even add 95% of the features to older editions, just not new keywords or inference defaults. I think in C++ people complain about the effort to migrate --std, rather than having to learn variants or concepts. It does add to the complexity of course, and it's useful to agree on a consistent style, I just mean Rust doesn't have the particular issues with everyone having to migrate in lockstep, or anyone needing to for new tools

by speedstyle

8/18/2026 at 1:03:44 AM

Same with C++, new standards will support old code. But they'll disagree on using the new features in the new toolchains. Or more likely, they'll just use the new features. So there's no option of sticking to old feature sets, you will encounter the new ones at the very least.

Even with a style guide in C++... Google is known for having one. I was forced to change how I use basic strings multiple times there. They also changed the rules around refs (&).

by frollogaston

8/17/2026 at 7:19:00 PM

That's an opinion. One you're very welcome to hold, but it's not universal by any means. Lots of people enjoy rust.

by jjice

8/17/2026 at 8:09:16 PM

Fascinating how many people still overcomplicate offloading to GPUs.

by rfgplk

8/17/2026 at 8:46:29 PM

How so? I don’t know anything about this area.

by MBCook

8/17/2026 at 9:26:08 PM

What's the easy way here? Linking CUDA into your Rust binary?

by konradha

8/17/2026 at 9:31:16 PM

:)

This might be an relevant read: https://smolmachines.com/engineering/gpu-over-vsock

by binsquare

8/17/2026 at 11:10:58 PM

I just read this tool up and down but still can't conceive of a use case.

by dc443

8/18/2026 at 11:16:22 PM

that's fair

the connection I'm making is that both sit on the same interface: the CUDA driver api (module load, launch, memcpy).

the paper is about compiling kernels using safe rust instead of CUDA C++, what I did is reimplement the other side of it using rust for the calls from inside a VM get forwarded over vsock to the host driver via api-remoting.

by binsquare

8/18/2026 at 11:49:39 AM

I have no idea how spinning up a MicroVM solves the problems this paper talks about and I doubt you actually have programmed GPU kernels ever

by nylonstrung

8/18/2026 at 6:07:05 AM

No, it’s not relevant to simple GPU offload, stop spam linking your slop blog.

by foltik

8/18/2026 at 11:21:57 PM

Will take it to heart

by binsquare