[cfarm-users] Setting up GitLab CI for git.git on the farm

[Ævar Arnfjörð Bjarmason]

On Sun, Nov 25 2018, Segher Boessenkool via cfarm-users wrote:

> On Sun, Nov 25, 2018 at 10:23:41AM +0100, Baptiste Jonglez wrote:
>> On 24-11-18, Segher Boessenkool via cfarm-users wrote:
>> > On Sat, Nov 24, 2018 at 11:28:58PM +0100, Ævar Arnfjörð Bjarmason via cfarm-users wrote:
>> > > I've tried to be conservative about resources. It's all nice -n 19'd,
>> > > and with a conservative -j value relative to the number of cores:
>> > > https://gitlab.com/git-vcs/git-gitlab-ci/blob/b8d4645891aa/ci/gitlab/run-on-gcc-farm.sh#L62-163
>> >
>> > At least for the Power machines, that isn't conservative at all.
>> > -j1 is conservative.  -j24 is not conservative on a machine with 20 CPUs
>> > (gcc112), or 32 CPUs (gcc135).  The AIX (gcc119) jobs seem to run for over
>> > an hour on half the machine?  That's no good :-(
>>
>> According to ansible [https://cfarm.tetaneutral.net/machines/list/] gcc112
>> has 160 cores, and gcc135 has 128 cores.  Is ansible getting this wrong?

Thanks all. I definitely don't want to misuse resources here. Will tweak
it way down. I was going by e.g. 128 on gcc135 and $(getconf
_NPROCESSORS_ONLN), but can see that lscpu gives the numbers discussed
her.

> gcc110 is a Power7 with 16 cores, 64 threads.
> gcc112 is a Power8 with 20 cores, 160 threads.
> gcc135 is a Power9 with 32 cores, 128 threads.
>
> (Running more than 4 threads per core on a Power8 does not really help,
> so you can count gcc112 as 80 threads).

I was curious to see what this was like on Power8 so on gcc135 I ran
this as a one-off:

    for j in 8 16 32 64 96 128 160 192 224 256; do for i in {1..3}; do git clean -dxf; time perl -MTime::HiRes=time -wE 'my $n = shift; my $t0 = -time(); system "make -j$n CFLAGS=-O0 NO_CURL=Y"; $t0 += time(); open my $fh, ">>", "/tmp/perf.txt"; printf $fh "%.2f\t%d\n", $t0, $n' $j ; done; done

I.e. compile with -O0 3 times for 8, 16 etc. values of -j:

    [avar at gcc135 ~]$ sort -n /tmp/perf.txt
    2.86    128
    2.87    256
    2.90    192
    2.91    160
    2.92    192
    2.93    128
    2.93    160
    2.93    224
    2.93    224
    2.94    160
    2.95    192
    2.96    128
    2.96    224
    2.98    256
    3.02    96
    3.02    96
    3.04    256
    3.06    96
    4.05    64
    4.13    64
    4.16    64
    4.97    32
    4.97    32
    5.05    32
    6.27    16
    6.28    16
    6.33    16
    9.83    8
    9.85    8
    9.86    8

> Running many tasks using SMT hurts tasks that want to run single-threaded,
> of course.  Linux is quite good at distributing things nicely, but
> oversubscription hurts.
>
> Automated tasks on shared systems should try to stay out of the way.