[cfarm-users] more on POWER9? (was: Re: cfarm110 and cfarm111 decommissioning on February 27)

[Luke Yasuda]

Hi all,

On 2026-02-14 03:34, Sean McGovern via cfarm-users wrote:
> I guess this really spells the end for big-endian POWER.

Maybe not so fast?

In January, I proposed a few new VMs on my Talos 2 to cfarm-admins, but 
there was not really enough opinions on the VM specs and the choice of 
OSes. So I'm yeeting it to this mailing list to ask for more thoughts:

<---------------------------------------------------------->

Hardware: Talos 2 with two 02CY227 (POWER9), 176 threads in total. 1TB 
RAM in total (purchased when DDR4 was still affordable). Running Debian 
12 ppc64el (PAGESIZE=4k, bookworm-backports), and of course with the 
companion of my ported version of proxmox (a little buggy but usable). 
KVMs are started with "-machine 
'cap-hpt-max-page-size=4096,type=pseries,cap-nested-hv=on'".

Just for this time, for the convenience of provisioning new machines, I 
made a three-tier system: VMs are divided into large, medium, small.

Large: 64 cores, 64GB RAM, 2TB /home.
Medium: 32 cores, 32GB RAM, 1.5TB /home.
Small: 16 cores, 16GB RAM, 1TB /home.

... and we can expand from there e.g., if the memory or disk space is 
too tight on any of the VMs. Every VM gets a 64GB root partition like 
cfarm420..430 and also can be expanded later. The /home storage will 
have to use NVME over TCP or iSCSI, exported from "the big fat AMD 
server" where cfarm420..422 are on. I bought two 3.84TB SATA SSDs some 
time last year on the second hand market, and the SSDs are the identical 
model that cfarm420..422's /home are using.


cfarm currently has:
cfarm111,119 (RIP): POWER7,8, AIX
cfarm110 (RIP): PWOER7, Centos 7.9, big endian
cfarm29: POWER9 (Blackbird), Debian 13 ppc64el
cfarm112: POWER8, Centos 7.9, little endian
cfarm120: POWER10, AlmaLinux 9.7, little endian
cfarm121: POWER10, Debian port ppc64, big endian
cfarm135: POWER9, AlmaLinux 8.10, little endian


Newly proposed: for the number of VMs and the variety OSes:

Large: 2
1: Debian "testing forever" (essentially rolling release), ppc64el, 
PAGESIZE=64k (non-default)
2: Debian forky, ppc64el, PAGESIZE=4k (default)

Medium: 1~2
1~2: Almalinux 10 and/or Rocky 10, little endian

Small: 4
1: Alpine newest stable release, little endian
2: FreeBSD current, little endian
3: FreeBSD current, big endian (although I heard unconfirmed rumors on 
IRC #talos-workstation that they are ending big endian support??)
4: NetBSD newest stable release, don't know endianness

<---------------------------------------------------------->

Notes:
a. OpenBSD doesn't work with KVM on POWER, it can only work with bare 
metal Talos 2 or Blackbird. AIX is obviously not available.
b. Keep in mind that like cfarm420..430, /home storage is on a ZFS 
volume, the transparent zstd/lz4 block level compression ratio is good, 
somewhere between 1.5~2.5.
c. Do we need another medium or small VM that runs Debian port ppc64 
(big endian), like cfarm121?
d. I'm not very familiar with the difference among POWER{7..11}. Would 
this (big fat) POWER9 board with a variety of VMs be useful?
e. Proposals for non-linux non-gnu systems are of course welcome, but it 
may come with a steep learning curve for the maintainer, so please be 
patient. They'll probably be "small".


Other general misc updates:
イ. cfarm420..430 had an important network changes last month. mosh is 
available again. IPv6 is available again. The network latency is reduced 
with wireguard tunnels. It was one of the most complicated networking 
setup I've ever done. If I can get FTTH with a non-shitty ISP, the 
wireguard tunnel wouldn't be necessary.
ロ. On the subject of getting FTTH with an(other) ISP of my choice, I 
lost one battle against the homeowner association. Preparing for the 
next battle.
ハ. At the same time of proposing POWER9 VMs, I also proposed 2 new hurd 
VMs but I've made 0 progress so far. The first issue is that I have not 
finished (or started) to write my external watchdog to detect fatal 
crashes. The second issue: Baptiste mentioned that ansible's hurd 
support on creating users is non-existent now, so more work on 
implementing that with ansible is needed.
二. I'll consider accepting donations in the future if the economy is bad 
and I'm desperate enough, to cover the cost of hosting.


Cheers,
-- 
Luke Yasuda
About me: https://jing.rocks/about/
GPG Fingerprint: 4E09 8D19 00AA 3F72 1899 2614 09B3 316E 13A1 1EFC
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: OpenPGP digital signature
URL: <http://lists.tetaneutral.net/pipermail/cfarm-users/attachments/20260215/88d7ed8e/attachment.sig>