Distbuild is a distributed build system, it is a set of Baserock systems working together to satisfy a set of build requests. System builds are naturally parallelisable, assuming the dependencies for a, b and c are satisfied, then a Distbuild network with 3 workers can build a, b and c in parallel.
There are 3 components:
Initiators are Baserock devel machines, they submit build requests to Controllers. Controllers manage build requests, they decide what needs to be built and tell Workers what to build. Controllers also perform some basic build scheduling to avoid wasting build resources. All components of the distbuild network make use of a single trove, this trove is used to share the definitions for the systems being built as well as the artifacts produced by the workers. Once a worker completes a build the resultant artifacts are transferred to the trove's artifact cache.
Note: This guide assumes you have some experience deploying KVM virtual machines, if not please read Deploy to KVM
First, build the appropriate Baserock
build-system-xxx variant for your
architecture. (Note that in releases up to and including Baserock 14.40, these
$ morph build systems/build-system-x86_64.morph
The following example cluster morph deploys an x86-64 distbuild network with one controller node and two worker nodes.
Note: To deploy a distbuild network you will need the trove's worker keys, deployment will copy these keys onto the nodes to allow them to access all trove content.
In this cluster morph we set ATTACH_DISK to attach artifact storage space to the distbuild workers and FSTAB_HOMES to mount the storage into the right place. It's up to you where this storage comes from, typically you will want at least 50G. In this example we have allocated a logical volume for each worker.
name: x86_64_distbuild_network kind: cluster description: A cluster morph for an x86_64 distbuild network systems: - morph: systems/build-system-x86_64.morph deploy-defaults: TROVE_ID: $trove_id TROVE_HOST: $trove_url CONTROLLERHOST: &controllerhost $controller_url DISTBUILD_CONTROLLER: no DISTBUILD_WORKER: yes INSTALL_FILES: distbuild/manifest WORKER_SSH_PUBKEY: worker.key.pub WORKER_SSH_KEY: worker.key VERSION_LABEL: distbuild DISK_SIZE: 5G RAM_SIZE: 2G VCPUS: 2 FSTAB_HOMES: LABEL=src /srv/distbuild auto defaults 0 0 deploy: controller: type: extensions/kvm location: kvm+ssh://user@vmhost/distbuild-controller/path/to/vm/controller_vm.img DISTBUILD_CONTROLLER: yes DISTBUILD_WORKER: no HOSTNAME: *controllerhost WORKERS: $worker0_url, $worker1_url FSTAB_HOMES: no worker0: type: extensions/kvm location: kvm+ssh://user@vmhost/distbuild-worker0/path/to/vm/worker0_vm.img HOSTNAME: $worker0_url ATTACH_DISKS: /dev/vg/worker0_src worker1: type: extensions/kvm location: kvm+ssh://user@vmhost/distbuild-worker1/path/to/vm/worker1_vm.img HOSTNAME: $worker1_url ATTACH_DISKS: /dev/vg/worker1_src
$ morph deploy distbuild-cluster.morph
To run a build using your distbuild network:
$ morph distbuild --controller-initiator-address=<controllerhost> <system>
Distbuild places build logs for each chunk in the current working directory, for example the log for a build of pango-misc would be stored in build-step-pango-misc.log, so you could view build progress using:
$ tailf build-step-pango-misc.log