alt.hn

1/16/2025 at 2:45:25 PM

Red Hat Woos VMware Shops with OpenShift Virtualization Engine

 https://www.nextplatform.com/2025/01/15/red-hat-woos-vmware-shops-with-openshift-virtualization-engine/

by rbanffy

1/16/2025 at 7:09:38 PM

They’ve been hammering us pretty hard, especially some folks in the leadership chain who worked with them before. While I have no direct beef with the product, the reality is that our Enterprise workload (and in fact, most Enterprise IT workloads in my experience) are VM-first, not container-first.

My research conclusion at the time was that, while OpenShift is a great product worthy of consideration, it really only shines in organizations that are heavily invested in microservices or Kubernetes. If you (or more specifically, your vendors) haven’t migrated into that state, it’s not worth it compared to a RHEL server license and their KVM+Cockpit solution for bog standard VMs.

by stego-tech

1/17/2025 at 1:16:16 AM

This is a VM only version of OCP. No containers or microservices.

So if you haven’t migrated into that state mentioned but want a Hypervisor that isn’t VMware and have enterprise support.

by mogwire

1/16/2025 at 10:49:21 PM

or proxmox.. i was going to say for small shops, but i have heard of some larger deployments recently.

by woleium

1/16/2025 at 9:01:05 PM

Red Hat just hit us with 300% price increases for OpenShift across the board right after we went live in production after a little over a year of implementation. The entire org is very, very unhappy about it.

by breakingcups

1/17/2025 at 1:18:22 AM

300% and you added no additional compute? Prices went up but not 300% unless you got a major discount and added compute.

by mogwire

1/16/2025 at 10:47:57 PM

An IBM by any other name would smell the same?

by woleium

1/16/2025 at 3:27:34 PM

What're the FLOSS technologies underlying OpenShift? Sounds like KVM?

by lenerdenator

1/16/2025 at 3:45:55 PM

Openshift is Kubernetes+.

Openshift Virtualization Engine is kubevirt (aka kvm/libvirt).

by natebc

1/17/2025 at 1:39:25 AM

That's a shame. We found a significant performance loss with kubevirt and nobody has ever really been able to explain it or make it go away. 15% throughput loss on an open source mq.

by denkmoon

1/17/2025 at 7:09:08 AM

Do you have the RH case number / Jira issue? I can take a look.

by rwmj

1/17/2025 at 9:53:36 PM

OKR for the open source kubernetes, kubevirt for the virtualisation part.

by jaitaiwan

1/16/2025 at 3:31:13 PM

Probably kubevirt if I had to guess (which would then use KVM under the hood)

by samcat116

1/16/2025 at 4:29:04 PM

Yes you are correct, it is kubevirt and leveraging KVM as the hypervisor coupled with the QEMU/Libvirt userspace pieces.

by houseofzeus

1/16/2025 at 4:31:29 PM

everything ultimately boils down to kvm or qemu, usually

by whalesalad

1/16/2025 at 4:33:06 PM

If I were building a datacenter today I would go with proxmox. It's "just debian" under the hood and can be customized and controlled a multitude of ways (UI, CLI on the box, Terraform, API, etc)

by whalesalad

1/16/2025 at 7:03:59 PM

I'm using it now, even paying for it, and it works very well but I only have one wish. That they could distribute an image based distro, or a slimmed down appliance ISO. It just seems unnecessary to have an actual OS on each host. Specially since I've been impressed by Talos and Openshift for a while.

by INTPenis

1/16/2025 at 5:50:25 PM

Doesn't Proxmox use a separate kernel package compared to Debian? That's kinda annoying because it ends up making the distro a 'Frankendebian' at best. Even using an up-to-date kernel from the stable backports repositories is a lot better than that.

by zozbot234

1/16/2025 at 6:10:15 PM

They use a slightly modified Ubuntu kernel (https://github.com/proxmox/pve-kernel), with things like ZFS added. They also really are good about using proper Debian tooling, and so their kernel doesn’t cause any weird dependency issues.

Right now they install proxmox-kerne-6.8.12-6 by default (using pseudo-packages called proxmox-default-kernel and proxmox-kernel-6.8 pointing at it), and offer proxmox-kernel-6.11.0-2 as an opt-in package (by installing proxmox-kernel-6.11)

I’ve been using the latest opt-in kernels on all of my Proxmox nodes for a few years now, and I’ve never had any issues at all with that myself.

by brirec

1/16/2025 at 6:13:49 PM

> things like ZFS added

That's a big gotcha - ZFS is non-free so of course it cannot be part of Debian proper. Hopefully we'll get feature parity via Btrfs or Bcachefs at some point in the future.

by zozbot234

1/16/2025 at 9:33:16 PM

> ZFS is non-free so of course it cannot be part of Debian proper

ZFS is under the CDDL which is a perfectly good free and open-source software license, just some people view it as incompatible with GPL (IANAL, but this is apparently somewhat controversial; see the wikipedia page) so Debian doesn't distribute ZFS .ko files for Linux in binary form. They do, however, have an official package for it[1], just using DKMS to compile it locally.

[0] https://en.wikipedia.org/wiki/Common_Development_and_Distrib...

[1] https://packages.debian.org/sid/zfs-dkms

by yjftsjthsd-h

1/17/2025 at 6:17:10 AM

If some people see it as incompatible, its not perfectly good then, is it.

by worthless-trash

1/17/2025 at 6:48:11 AM

CDDL is a good license. GPLv2 is a good license. They happen to be (maybe) incompatible. That doesn't make either of them bad. I mean, would you argue that GPLv2 is bad because it's not compatible with CDDL?

by yjftsjthsd-h

1/16/2025 at 6:22:41 PM

Thats incorrect, it is free software but incompatible with the GPL.

by Lariscus

1/16/2025 at 6:12:23 PM

It certainly has an optimized kernel for its use case. I believe it also includes ZFS by default. I wouldn't be surprised if the Proxmox developers would prefer to upstream these defaults, but they likely would introduce regressions for the common use case that Debian optimizes for.

Ultimately, I use Proxmox as a hardware hypervisor only, so I don't mind that it uses its own kernel. Everything I run is in its own VM, with its own kernel that is setup the way I want.

by Maskawanian

1/16/2025 at 7:02:35 PM

I'd be worried how proxmox would scale past a few racks. The bones are all good, but I'm not sure how much scale testing their API layer has had.

by samcat116

1/16/2025 at 4:37:30 PM

> If I were building a datacenter today I would go with proxmox.

I use Proxmox as well in a small-ish deployment, but have also heard good things with Xcp-ng.

At a previous job used OpenStack.

by throw0101c

1/16/2025 at 4:51:10 PM

Or if you'd like to help preventing linux monoculturalization of datacenters, MNX Triton or vanilla SmartOS are very good options too.

by NexRebular

1/16/2025 at 4:55:50 PM

The cool thing about proxmox is that it is - again - "just debian" so there is really no vendor lock-in. Yes they do have commercial support/update subscriptions but the community offering is open (https://github.com/proxmox). So I do not worry too much about lock-in or monoculturalization. At the end of the day it is a wrapper around fundamental components of Linux. They do not have any proprietary secret sauce that would F you down the road.

Correction I see now that the projects you reference are Solaris based. I am down with that cause too - but if you are a BSD/Solaris shop expect to do a lot of things on your own. The linux virtualization space is substantially larger (not necessarily suggesting it is better...)

by whalesalad

1/16/2025 at 8:20:47 PM

As an aside: Solaris is something you can buy from Oracle, which they forked from OpenSolaris 15 years ago. SmartOS is a distribution of illumos, which also forked from the same code 15 years ago. They have since diverged, in some areas dramatically, so we (the illumos community) don't bill ourselves as being Solaris based.

by jclulow

1/17/2025 at 4:52:59 AM

Is there a VDI solution that runs on proxmox? I've only found UDS Enterprise.

by polski-g

1/16/2025 at 3:40:37 PM

Well, the Enterprise-hypervisor market is certainly wide open right now, due to Broadcom turning the financial screws on VMware customers hard. There are, broadly, two categories of potentially-profitable VMware customers up for grabs:

-Large enterprises that previously purchased hardware-with-accompanying-VMware-licenses from OEMs like Dell-EMC: Broadcom refused to even honor pre-acquisition license keys from these sources, leaving many private data centers in the lurch, unless they paid a huge premium for a new Broadcom-originated annual subscription (whereas the original key was one-off)

-Service providers with an ongoing "small-percentage-of revenue per year, payable in arrears" agreement, that were suddenly forced into a "hard vCPU and vRAM limit" subscription, payable for at least 2 years upfront.

However, the magic word for both customer segments is "vMotion", i.e. live-migration of VMs across disparate storage. No OSS and/or commercial (including Hyper-V) solution is able to truly match what VMware could (and can, at the right price) do in that space...

by antithesis-nl

1/16/2025 at 4:58:35 PM

> However, the magic word for both customer segments is "vMotion", i.e. live-migration of VMs across disparate storage. No OSS and/or commercial (including Hyper-V) solution is able to truly match what VMware could (and can, at the right price) do in that space...

Someone's gonna start working on that soon. Necessity is the mother of invention.

To me, this will be the UNIX wars moment for virtualization.

Originally, UNIX was something AT&T/Bell Labs mainly used for their own purposes. Then people wanted to use it for themselves. AT&T cooked up some insane price (like $20k in 1980s money) for the license for System V. That competed with the BSDs for a while. Then, some nerd in a college office in Finland contributed his kernel to the GNU project. The rest is history.

UNIX itself is somewhat of a niche today, with the vast majority of former use cases absorbed by GNU/Linux.

This feels like an effort by Broadcom to suck up all of the money in the VMWare customer base, thinking it's too much of a pain in the ass to migrate off of their wares. In some circumstances, they're not wrong, but there's going to be teams at companies talking about how to show VMWare the door permanently as a result of this.

Whether Broadcom is right that they can turn a profit on the acquisition with the remaining install base remains to be seen.

by lenerdenator

1/16/2025 at 11:53:22 PM

You have to understand that Broadcom isn't actually Broadcom the chipmaker. It's a private equity firm that used be named Avago Technologies before it bought Broadcom. So squeezing until there's nothing left is the plan.

https://digitstodollars.com/2022/06/15/what-has-broadcom-bec...

by azurelake

1/17/2025 at 11:13:40 PM

> The truth is Broadcom is not a semiconductor company. Nor is it a software company. It is a private equity fund, maximizing cash flow from an endless series of acquisitions. This is disheartening to many in the semis industry and probably confusing to those in software.

I hate when finance people talk like this.

No, it's not confusing to people in software. We're well aware of your (finance) industry's reputation of sucking capital out of necessary, competitive companies for your own personal gain. If we thought we could get away with it, we'd do something about it.

by lenerdenator

1/16/2025 at 8:12:23 PM

I consider UNIX/POSIX becoming niche, including GNU/Linux, for the so called cloud native workloads.

The large majority of managed languages being used in such scenarios, compiled to native or VM based, have rich ecosystems that abstract the underlying platform.

Moreso, if going deep into serverless, chiseled containers, unikernel style, or similar technologies.

Naturally there is still plenty of room for traditional UNIX style workloads.

by pjmlp

1/16/2025 at 4:35:46 PM

What can VMware do in this vmotion space?

The docs says open source can do a live migration, see https://www.linux-kvm.org/page/Migration and https://docs.redhat.com/en/documentation/red_hat_enterprise_...

by noja

1/16/2025 at 11:44:57 PM

The majority of vmware customers could get by with qemu/kvm + pacemaker/corosync, but that requires hiring people who can read a manpage.

by b5n

1/16/2025 at 5:35:26 PM

Vaguely like MS Active Directory vs Kerberos. The really big thing 'vmotion' provides is 'things just work' vs flexibility of options but more effort required.

by mjevans

1/16/2025 at 4:53:52 PM

Third is academia. Even with a marginal academic discount, it doesn't come close to offsetting the price hikes. When the choice is between slashing personnel budget and minimizing the usage of/completely migrating away from VMware before the next renewal, there's really no choice in the matter, especially with the difficulty of firing employees.

by tart-lemonade

1/16/2025 at 11:08:26 PM

Another magic word that Vmware has which no other vendor does the same is NSX. (network virtualization).

proxmox is lightyears behind this usecase, and so are most other vendors. Especially if you are building private/public clouds with multi tenancy in mind.

NSX is really well designed and scales nicely, (it even has MPLS/EVPN support for Telecom Service Provider integration).

Most open source and other commercial offerings have solved both the compute and storage aspect quite well. But on the networking front, they a really not comparable.

Proxmox for instance, only supports a vxlan encapsulation or vlans, without support for a proper control plane like EVPN. Heck, route injection by BGP is only doable by DIY'ing it ontop of proxmox.

"just using vlans" is not going to cut if you want to really scale across datacenters and with multiple tenants. NSX does this all really nicely without having to touch the network itself at all thanks to encapsulation and EVPN route discovery.

by kazen44

1/16/2025 at 4:44:15 PM

Hashicorp has some Nomad drivers for Qemu and a beta version that uses libvirt. However it's fairly immature and lacks a lot of features they would need to be competitive there.

Since IBM already has OpenShift I'm not sure how much time and effort they want to put into Nomad virtualization, but I'd love it as an alternative to Kubernetes.

by SteveNuts

1/16/2025 at 4:32:40 PM

libvirt+qemu can do live migration across disparate storage; I know quite a lot of people want to use it. I'm curious in which ways you find the vMotion stuff does well with that.

by trebligdivad

1/16/2025 at 6:13:33 PM

I would think another issue for a lot of existing large VMware customers is NSX (incl. dynamic firewalling), orchestration (vRA/vRO), vROPS, and the gazillion integrations that have been built with tight coupling at both ends. Swapping that out is not an easy task, and the "digital transformations" some of the available products are banking on take a lot of time in real life.

by mvdwoord

1/16/2025 at 8:06:18 PM

Xen (and XenServer and XCP and the Citrix banded stuff) have been able to live migrate VMs for a really long time. Definitely not something special to VMWare.

Realistically, all the legacy workloads (those that are singletons and can't be load-balanced, need an active GUI session etc) are going to to be problems forever, even if you keep VMWare around.

by oneplane

1/16/2025 at 7:40:25 PM

" Broadcom refused to even honor pre-acquisition license keys from these sources, leaving many private data centers in the lurch"

How is this not contract violation?

by UltraSane

1/16/2025 at 8:32:56 PM

Sometimes it turns out that contract law only applies to the poorer of the parties engaged in a contract.

by plagiarist

1/16/2025 at 8:05:51 PM

That was my question as well. This seems blatantly illegal.

by bigstrat2003

1/16/2025 at 8:13:06 PM

The courts will make that decision in 5 years or so, but it doesn't help people who need their stuff working today.

by jandrese

1/16/2025 at 5:55:06 PM

>However, the magic word for both customer segments is "vMotion", ... . No OSS and/or commercial (including Hyper-V) solution is able to truly match what VMware could (and can, at the right price) do in that space...

Storage vMotion requires a hefty license (as does DVS and the other useful things, such as containers). Proxmox does it all out of the box at a very reasonable price point.

Hell, VMware wont even let you use LLDP until your pissing money out of all orifices. You get CDP only for "free".

After 20+ years of being a VMware fanboi I am migrating all my customers to Proxmox. I've had enough.

by gerdesj

1/16/2025 at 6:03:29 PM

> Proxmox does it all out of the box at a very reasonable price point.

Live migration of containers via the CRIU featureset (checkpoint+restore in userspace, which is now part of the mainline Linux kernel) is also an interesting theoretical possibility - AIUI the Kubernetes folks are at least thinking about supporting it. (Live migration fits remarkably well with containers since it requires comprehensive namespacing of all system resources - abstracting away from any dependence on the local machine - which is also how containerization works to begin with.)

by zozbot234

1/17/2025 at 6:19:10 AM

If live migration is the key to vmotion, I think that this has been working for some time between like-cpu deployments.

by worthless-trash

1/16/2025 at 3:50:34 PM

I think Oxide's stuff does live migration, but I don't know how well it runs on non-Oxide hardware.

by zellyn

1/16/2025 at 4:32:34 PM

I think most of these solutions including OpenShift Virtualization, Hyper-V, Proxmox, etc. do live migration. What the previous post is talking about is some of the more advanced VMware live migration features like storage live migration and cross-cluster live migration and some of the automations layered over the top of them.

by houseofzeus

1/16/2025 at 4:08:40 PM

As far as I can tell, Oxide is all about compute, not storage -- i.e., as long as you can guarantee a stable storage layer, your Oxide racks will be fine.

VMware used to go a bit further, in that they allowed your compute nodes to fail and/or your storage nodes to fail, without adverse effects.

If Oxide can do that, out-of-the-box, right-now, they'll be having a field day. Otherwise, my reservations about Oxide's business model remain...

by antithesis-nl

1/16/2025 at 4:33:59 PM

I _think_ their storage layer is fault tolerant. Not sure though. Hopefully one of them will weigh in here.

by zellyn

1/16/2025 at 5:21:53 PM

This is not my area of expertise, so I'll quote from our website: https://oxide.computer/product/storage

The storage service uses OpenZFS for all data storage. This marries Oxide’s distributed data storage and multi-node failure resiliency with the dependability and efficiency OpenZFS has earned in its 20 years of running demanding workloads.

The Oxide control plane monitors performance metrics as another early signal of component failure. As sleds and SSDs are rotated in and out, the Oxide control plane migrates storage regions to ensure the appropriate redundancy.

OpenZFS checksums and scrubs all data for early failure detection. Virtual disks constantly validate the integrity of your data, correcting failures as soon as they are discovered.

by steveklabnik

1/16/2025 at 9:43:21 PM

I figured ZFS would do the low-level redundancy, but I was under the impression that Crucible does the higher-level stuff, and I don't know much about it.

by zellyn

1/17/2025 at 7:37:39 PM

Yeah, I believe so too, but I haven't ever worked on it so I don't know a ton about it either.

by steveklabnik

1/16/2025 at 4:09:38 PM

vMotion is about host failure. If a blade server dies, the vms can be spun up on another blade. Storage is same.

by wesapien

1/16/2025 at 4:14:39 PM

You're thinking of high-availability, which registers the vmx file of the VM to a physical server that isn't dead, then powers the VM back on. Whereas vMotion is either a cold or live-migration of the VM + memory state (if the VM is powered down, there is no memory state to migrate).

by blown_gasket

1/16/2025 at 4:14:28 PM

> vMotion is about host failure

No? It's also about load balancing and draining compute/storage resources in preparation for maintenance.

Most pertinently: as long as your alternative doesn't cover any vMotion use-cases, customers will remain 'in talks' with Broadcom...

by antithesis-nl

1/16/2025 at 5:13:10 PM

Fun that people -still- don't understand the feature set that VMware provides.

He/She is thinking about VMware Fault Tolerance

by toredash

1/16/2025 at 3:49:39 PM

[dead]

by mjtechguy

1/16/2025 at 8:21:42 PM

Moving off VMware (yay!) Onto open shift (crap.)

by more_corn