[cfarm-users] Setting up GitLab CI for git.git on the farm

[Ævar Arnfjörð Bjarmason]

On Mon, Nov 26 2018, David Edelsohn wrote:

> On Sun, Nov 25, 2018 at 6:16 PM Ævar Arnfjörð Bjarmason
> <avarab at gmail.com> wrote:
>>
>>
>> +CC cfarm-admins@
>>
>> On Sun, Nov 25 2018, David Edelsohn via cfarm-users wrote:
>>
>> > On Sat, Nov 24, 2018 at 5:53 PM Segher Boessenkool via cfarm-users
>> > <cfarm-users at lists.tetaneutral.net> wrote:
>> >>
>> >> Hi!
>> >>
>> >> On Sat, Nov 24, 2018 at 11:28:58PM +0100, Ævar Arnfjörð Bjarmason via cfarm-users wrote:
>> >> > I got access to the farm a while ago to test free software projects I
>> >> > work on, mainly git.git. I wanted to send a headsup about what I've been
>> >> > up to.
>> >> >
>> >> > I'm setting up something where the integration branches of git.git are
>> >> > smoke tested on various machines on the farm. See the Git ML
>> >> > announcement at
>> >> > https://public-inbox.org/git/875zwm15k2.fsf@evledraar.gmail.com/
>> >> >
>> >> > Currently I'm running things on these machines:
>> >> > https://gitlab.com/git-vcs/git-gitlab-ci/blob/b8d4645891aa/.gitlab-ci.yml#L16-57
>> >> >
>> >> > I've tried to be conservative about resources. It's all nice -n 19'd,
>> >> > and with a conservative -j value relative to the number of cores:
>> >> > https://gitlab.com/git-vcs/git-gitlab-ci/blob/b8d4645891aa/ci/gitlab/run-on-gcc-farm.sh#L62-163
>> >>
>> >> At least for the Power machines, that isn't conservative at all.
>> >> -j1 is conservative.  -j24 is not conservative on a machine with 20 CPUs
>> >> (gcc112), or 32 CPUs (gcc135).  The AIX (gcc119) jobs seem to run for over
>> >> an hour on half the machine?  That's no good :-(
>> >
>> > The AIX systems unfortunately are I/O bound.
>> >
>> > One can run set up a temporary directory in the /scratch RAMDISK for
>> > some improvement.
>>
>> Thanks. The /scratch directory exists on gcc119 and speeds things up by
>> a *lot*, but doesn't exist on the other AIX machine, gcc111. Would it be
>> possible to have /scratch be a ramdisk there too?
>
> It's possible, but gcc111 is an old system with an older release of
> AIX. There's not a lot of reason to focus on it.

Thanks. I don't have any burning desire to test things there, but the
reason I test on these obscure setups is because it often reveals edge
cases in POSIX assumptions & other portability issues. So testing on
older OS's also helps. But per the "[cfarm-users] GCC111 filesystem is
very slow" it looks like others might have similar issues.