Rockbuild

About

Rockbuild is a system for running an ad-hoc heterogeneous distributed build farm over the internet (long latency and low bandwidth).

The main purpose of Rockbuild is to give developers quick compile results after commit.

Rockbuild was written for the requirements of the Rockbox project. Rockbox has a huge array of hardware targets, each with a different source code configuration. It is very time consuming for each developer to have to test compile every modification on all those targets before committing it. Therefore the focus of Rockbuild is to, as quickly as possible, give developers feedback to their changes. "Did my commit break any build?"

Build speed is priority one. In the Rockbox build farm of ~50 clients, the 200+ build targets that take nearly an hour for a single machine to build is completed by the farm in a handful minutes.

Features

Components

Rockbuild, at its' core, consists of two perl scripts:

rbmaster.pl
The script running on the master server. It tells each client what to build and tells old clients to upgrade.
rbclient.pl
The script running on each build client. It connects to the master server and recieves instructions for what to build.

In addition to this, there are a number of helper scripts and tools for managing the build results, creating the build tables etc.

Source

# git clone git://github.com/zagor/rockbuild.git

https://github.com/zagor/rockbuild

Authors

Rockbuild was written by Björn Stenberg and Daniel Stenberg.

License

Rockbuild is distributed under the GNU General Public License.

How is Rockbuild different from <other system>?

Generally, the goals are different. Many build systems are "Contiuous Integration" tools, focusing heavily on automated testing.

Rockbox, being a device driver-intensive embedded project, has a very wide array of source code configurations. The focus on Rockbuild is therefore to provide compile results for all configurations as soon as possible after each commit.

This might match the needs of your project, or it may not. Luckily, there are many other systems to choose from:

distcc
distcc preprocesses each source file on the master, hands over the file to a remote client to be built and then gets the object file back. Hence to be efficient, it requires (relatively) slow per-file compiles and very fast networks. Rockbuild uses big jobs with little network traffic and is therefore much faster over slow networks.
Buildbot, Hudson, Continuum
These systems all use a fixed list of build clients. Rockbuild allows new clients to join without preregistering with the master. Also, none of them match builds to client speeds. Rockbuild gives heavy jobs to fast clients and lighter jobs to slow ones. This makes a huge difference in build completion time.
Integrity, CruiseControl
These tools run all build jobs on the server. Rockbuild's main focus is to complete the build rounds quickly by distributing the workload.
Samba and PostgreSQL custom build farms
These are traditional rigid constructions where each client sets up cron jobs to run builds regularly. This approach can not produce the immediate commit feedback Rockbuild was designed for.