[cfarm-users] Future of cfarm186/cfarm187/cfarm188 at OSUOSL

[Zach van Rijn]

From: Baptiste Jonglez via cfarm-users <cfarm-users at lists.tetaneutral.net>
To: <cfarm-users at lists.tetaneutral.net>
Cc: "Baptiste Jonglez"<baptiste at bitsofnetworks.org>
Date: Wed, 21 Jan 2026 15:36:08 -0600
Subject: Re: [cfarm-users] Future of cfarm186/cfarm187/cfarm188 at OSUOSL

 > On 15-11-25, Baptiste Jonglez via cfarm-users wrote:
 > > ...
 > 
 > So, it seems we should rather go with option 2: have many small virtual
 > machines instead of one big physical x86 server.
 > 
 > I see three ways to run that:
 > 
 > (a) ask OSUOSL to use their OpenStack cluster: it delegates physical
 >     maintenance to competent people.  Also, each VM can have its own public
 >     IP address, which simplifies things a lot.  However, it would only work
 >     with "common" operating systems.  I will ask OSUOSL what OS they can support.

+1

 > 
 > (b) run virtualization ourselves on one of the big OSUOSL physical servers.
 >     However, it requires more maintenance on our end (e.g. maintaining a
 >     Proxmox setup),

This is fairly straightforward but inevitably is extra maintenance and the complexity can increase depending on storage/networking configurations. Maintenance is not bad.

One argument in favor of Proxmox (e.g.) is the ability to create both VMs (independent kernel) and containers (independent userland). In some cases, resources can be scaled up or down without rebooting.

 > and we probably won't be able to have a separate IP per
 >     VM.  Also, it would waste hardware resources: even 10 VMs would be far
 >     from filling up one server.

Why couldn't we do this? Each guest would have its own MAC address, so that can be done at either the network level or the hypervisor level if a block of IPs is assigned to the host.

 > 
 > (c) find somebody that already has the expertise and infrastructure to run
 >     VMs with unusual OS, and ask if they could provide some for cfarm.
 >     Ideally, each VM would need its own public IP address.

Several of our hosts can likely already do this.

Adélie, for example, has adequate machine capacity but lacks IPv4 space only due to choice of ISP and project budget. At a previous data center, IP addresses were not a limitation. Unfortunately data center costs were unsustainable at that previous location due to demand.

 > 
 > I would rather go with (a) for common OS and (c) for uncommon ones, and
 > avoid (b).

In other words, you wish to avoid (b) due to not wanting to run/maintain Proxmox or VMware/etc.?

I am not volunteering myself to do this maintenance but I will say from having done this for ~15 years it is manageable for a single person and I am happy to educate others about it.

 > Here is the OS wishlist collected from the thread:
 > 
 > - Haiku, Minix, Hurd and Sortix (especially Hurd)
 > - Trisquel
 > - Illumos distributions (OpenIndiana, OmniOS, Tribblix)
 > - Fedora or Gentoo

These seem reasonable but I am personally unfamiliar with Haiku. Instead, I have some questions:

1. How much storage/disk/memory do you think each guest should have, and should there be shared storage between them?

2. Are you willing to update the deployment scripts to integrate with any of these, if they aren't already supported?

3. Would it be acceptable for a VM host to not be involved with administration of each machine? I wouldn't want to force a host to suddenly be responsible for maintaining half a dozen new machines.

> Here, "big" means dual-socket Intel Xeon Platinum 8280, so 56 cores / 112
> threads, with 768 GB RAM. 

4. This machine would be owned by and live at OSUOSL but we can do anything we want with it?

5. Depending on how the hypervisor is configured, memory and CPU can be shared and restricted as needed.


Zach