[cfarm-users] Setting up GitLab CI for git.git on the farm

[Segher Boessenkool]

On Mon, Nov 26, 2018 at 10:46:39PM +0100, Baptiste Jonglez wrote:
> On 25-11-18, Segher Boessenkool via cfarm-users wrote:
> > On Sun, Nov 25, 2018 at 01:14:31PM +0100, Stefan Ring via cfarm-users wrote:
> > > On Sun, Nov 25, 2018 at 10:25 AM Baptiste Jonglez via cfarm-users
> > > <cfarm-users at lists.tetaneutral.net> wrote:
> > > >
> > > > According to ansible [https://cfarm.tetaneutral.net/machines/list/] gcc112
> > > > has 160 cores, and gcc135 has 128 cores.  Is ansible getting this wrong?
> > > 
> > > 8 threads per core. It really does not make sense to target more than
> > > one job for each core. Things will just get horribly slow.
> > 
> > No, that's not true, up to 4 jobs per core still gives considerable
> > speedup (and 8 a little too, depends).  Performance _per thread_ is lower
> > of course, but aggregate is higher.  SMT4 gets about twice as much work
> > done as single-threaded (which means each thread gets about half as much
> > done, but the total doubles).  Very roughly, depends on what you are doing
> > exactly, etc.
> > 
> > lscpu gets it right on all these machines btw (110, 112, 135); what does
> > ansible use?
> 
> Ansible parses /proc/cpuinfo:
> 
>   https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/facts/hardware/linux.py#L181
> 
> Do you know how lscpu gets its information?

It uses /proc/cpuinfo, and /sys/devices/system/cpu/**, and
/sys/devices/system/node/node*/cpumap.  Says strace.

Maybe /sys/devices/system/cpu/cpu*/topology/thread_siblings is where it
finds what things really are cores and what things are threads?  The source
will tell (but who knows what lscpu does is the most proper either!)

Anyway, $KERNEL_SRC/Documentation/ABI/testing/sysfs-devices-system-cpu
and $KERNEL_SRC/Documentation/cputopology.txt should help.

> If somebody comes up with an ansible patch that gets the number of
> CPU/cores/threads right on most farm machines, we can easily backport it
> to the live cfarm system (we are already doing this with a patch from
> Anatoly to properly support SPARC64, for gcc202).

So much bait, and not enough time ;-)


Segher