Flock | the social web browser

FBuilder

[edit] What is Fbuilder?

fbuilder is the building system for Flock builds. It feeds tinderbox and downloadable builds.

fbuilder is an alternate tinderbox client. It builds software and publishes it, but does not report on its status in human readable way. fbuilder is far more generic.

[edit] Installing fbuilder

fbuilder has all the same requirments as the Flock build environment. See Building Flock from Source.

Before you install fbuilder, be sure to have a non-privleged user named "build" on your machine.

if build is to report to Tinderbox, then build@tinderbox.flock.com must have the ssh-keys to trust the build user.
if build is to publish builds for archive or download, build@archive.hq.flock.com must have the ssh-keys to trust the build user.

Follow these steps:

[edit] get the code

Check out fbuilder into ~build/fbuilder.

$ svn co svn+ssh://svn.flock.com/home/svn/master/tools/trunk/fbuilder fbuilder

[edit] install starup files

Follow instructions for your platform.

[edit] Linux
```
# cd ~build/fbuilder/startup/linux  
# make install-redhat
or
# make install-debian

# /etc/init.d/fbuild start
```
[edit] Windows

tips for vmware buildserver
WARNING, if you install cygwin in an alternate path such as C:\Cygwin-Install You must change these directions to reflect that and you must change the start-build.bat script

As a user with adminstrator access
1. click: Start -> Control Panel
2. select: Scheduled Tasks
3. click: File -> New -> Scheduled Task
4. rename: fbuild
5. right click: fbuild -> Properties
6. In tab: Task
7. enter text in "Run": C:\cygwin\home\build\fbuilder\startup\win\start-build.bat
8. enter text in "Start in:" C:\cygwin\home\build\fbuilder
9. enter text in "Run as": HOSTNAME\build
10. click: Set password
11. enter text in "Password" and "Confirm password": the password for the build user
12. click: OK
13. uncheck: Run only if logged on
14. check: Enabled (schedule task runs at specified time)
15. click: Schedule (tab)
16. select: Schedule Task -> At System Startup
17. uncheck: Show multiple schedules
18. click: apply
19. click: OK
As a build user you can just click on start-build.bat. Warning: It will not work well if it is already running, and this .bat file destroys the lock mechanism.

Additionally if building an official Flock binaries
You need to install Microsoft Platfrom SDK 2003 R2 (appropriate for Windows XPpro)
Download Link
Once this is installed. Signtool must be run as a user with adminstrator privleges once. After this time the build user can run the program as long the as the build user is "logged in."

[edit] Mac
```
# cd ~build/fbuilder/startup/mac
# make
# make install
# SystemStarter start MacFbuild
```
Disable spotlight!

[edit] config

Create a config at ~build/conf/tinderbox-build.conf
```
$ svn co svn+ssh://svn.flock.com/home/svn/private/fbuilder-conf/trunk/conf conf
$ cd conf
$ ln -s DIR/CONFIGFILE tinderbox-build.conf
```

[edit] Running fbuilder

[edit] arguments

The primary application is ~build/fbuilder/bin/tinderbox-build, it only accepts two somewhat exclusive optional arguments

tinderbox-build --config=ABSOLUTE_PATH_TO_FILENAME

tinderbox-build --thread-config=ABSOLUTE_PATH_TO_FILENAME

If no argument is given it trys to read the confif file in ~build/conf/tinderbox-build.conf, which is by tradition a symlink to a particular config
If a lock file for the config is present (default value, ~build/tinderbox-build.pid) it will refuse to run.

[edit] thread-config

This is a simpler config file than a stander tinderbox-build config.

It lists paths to config files separated by new lines. If the path does not begin with a / it assumes it is a relative path to $HOME/conf

It will then iterate through the configs in the order they appear in the file (if the config exists on the filesystem). When it gets to the end of the list it will re-read the thread-config file to check for updates.

This feature is fully backwards compatible with daisy chained configs and once the chain has completed once it will move on to the next line in the thread-config.

[edit] config file

Config file parameters can override anything in %global in a tinderbox-build.

Syntax of the config file:

Lines begining with # are commments
Blank lines are ignored
The variable must be in the same form as in tinderbox-build.
Trailing spaces are ignored.
After an equals sign (=), trailing spaces are ignored, then the first non-space character until the end of the line is the value.
The config file is re-read at the end of any build, or a new config file if sepcified in the old one.

Example:

# example comment
SU = /usr/bin/su
TEST_SCRIPT = /usr/bin/true
daemonize = 0
BUILD_ARGS_STRING = F_SCM_BRANCH=trunks/buzzard
buildname = MacOS 10.4 (admins-mac-mini.local)
MAIL = /Users/build/conf/public/scripts/deliver
ARCHIVE = true
MAX_BUILDS = 1
CONFIG_FILE = /Users/build/conf/cardinal/parrot.conf

A few special config options

# include is non-recursive and pulls in 1 other config file, the name must be surround in double quotes
# this is parsed at the time of reading the config file
#
include "ABSOLUTE_PATHNAME"
#
# This is actually the NEXT configfile to be read, it doesn't read it the first pass but does the second time
# you should make all configfile in a chain use the same pidfile, if the CONFIG_FILE specified does not exit
# the tinderbuilder will keep on using its last config until the file does exist
#
CONFIG_FILE = ABSOLUTE_PATHNAME
#
# this is is always appended to BUILD_ARGS_STRING, this exists only to allow template include files
# build arguments are passed along to helper scritps as argument to build, deliver, and otheres
#
BUILD_ARGS_STRING1 = BUILD_ARGUMENT=VALUE

[edit] IPC signals

SIGNALS handled by tinderbox-build

TERM & QUIT are synonymous for tinderbox-build and they mean, shutdown gracefully and remove the pidfile

[edit] restarting fbuilder

There are two major strategies. Reboot the machine and let the startup scripts do it for you, or kill the processes listed in the PIDFILE (defaul value ~build/tinderbox-build.pid) -QUIT. There are specfic problems with each platform.

[edit] Windows

reboot

you can only send SIGNAL ipc from withing the same login session, and sometimes only within the same shell depending on how buggy cygwin has become. This makes killing a build that started out of standard startup procedure impossible, because cygwin cannot communicate with the processes. start-build.bat just wipes out the default pidfile and relaunches tinderbox-build, which makes it unsafe to run by hand as two build ast the same time cause problems.

[edit] Macintosh

kill -QUIT `cat ~build/PIDFILE`

The mac gets tinderbox-build process get automatically restarted every 10 seconds by macbuildd. macbuildd exist to provide bootstrap context so that the disk image utilities can be run. Basicilly all it has to do is sit there as root process and have direct parentage of the build script. It has it own lock files in /var/tmp/ (default /var/tmp/macbuildd.pid) macbuildd will clean up its pidfile when sent a QUIT signal.

[edit] start secondary macbuildd

fbuilder/startup/mac/start-macbuildd.sh PATH_TO_TINDERBOX_BUILD_CONFIG_FILE MACBUILDD_SESSION_ID

This script requires sudo access as macbuildd must be run as root

$ start-macbuildd.sh /Users/build/config/cardinal/locale/crane-pl.conf locale

This starts a new macbuildd using the tmp file /var/tmp/locale-macbuildd.pid, and will launch tinderbox-build as user build with --config=/Users/build/config/cardinal/locale/crane-pl.conf argument.

[edit] Linux

kill -QUIT `cat PIDFILE`
~/fbuilder/bin/tinderbox-build --config=PATH_TO_CONFIGFILE

/etc/init.d/fbuild restart

this requires the build users password as it tries to su to build to start the build

[edit] Setup a New Build Tree

Putting this in a separate document for now

BuildTreeSetup

[edit] Getting access to Tinderbox

If things are to make it to tinderbox or tinderbox build more steps need to be taken.

1 First ssh key trust must be setup between the build user on the build server to the archive server (archive.hq.flock.com) and tinderbox messageing server (tinderbox.flock.com).

2 The next step is update builds.txt on the archive server (archive.hq.flock.com:/archive/builds/installer/builds.txt)

trunk/crane.hq.flock.com                trunk/mac
trunk/chicken4                          trunk/windows
trunk/cuckoo                            trunk/linux
trunk/chicken2                          trunk/windows_incremental

This format

Left Side

a directory relative to the file builds.txt where builds are deliver. BRANCH/HOSTNAME

Right Side

destination directory relative to the tinderbox build application. This is a directory where select builds will be placed based on the rules in publish-public

3 create the relative directories if you re creating a new server i.e. on archive.hq.flock.com as the build user.

$ mkdir archive.hq.flock.com:/archive/builds/installer/BRANCH/HOSTNAME

[edit] Preparing tinderbox to recieve builds

This makes the tree live.

[edit] Create the new tree

checkout tinderbox (to your own machine, with SVN installed) website code and edit

$ svn co svn+ssh://svn.flock.com/home/svn/tools/tinderbox/trunk tinderbox

edit tinderbox/tinderbox2/default_conf/TreeData.pm

add your new tree definition to %VC_TREE

%VC_TREE = (

...

              'BRANCH_BRANDING' =>  {
                  root => '/home/cvs/svn/master/flock',
                  module => 'all',
                  branch => 'branches/BRANCH',
                 },

[edit] create the builds download directory

next create the build directory for the builds and copy in the download application

$ svn cp  tinderbox/tinderbox.flock.com/builds/trunk_sulfur tinderbox/tinderbox.flock.com/builds/BRANCH_BRANDING

the customize the message down in the html

$ vi tinderbox/tinderbox.flock.com/BRANCH_BRANDING/index.php

...

 These are testing builds for branches/BRANCH

commit changes

$ svn ci tinderbox

[edit] make tinderbox changes live on server

$ ssh -l USERNAME tinderbox.flock.com
$ sudo su - tinderbox
$ svn up .

make the build directories own by group build and writable

$ cd ~tinderbox/tinderbox.flock.com/builds/
$ sudo chgrp build BRANCH_BRANDING/{linux,mac,windows}
$ sudo chmod g+rwx BRANCH_BRANDING/{linux,mac,windows}

out of users tinderbox's crontab tinder.cgi --daemon-mode gets run every 5 minutes and that will produce.

http://tinderbox.flock.com/BRANCH_BRANDING/status.html

[edit] edit the tinderbox message of the day

You need to Administer the tree

http://tinderbox.flock.com/cgi-bin/tinderbox/admintree.cgi?tree=BRANCH_BRANDING

after creating a password and adding someone else to be an admin at the bottom set the message of the day so people can find the builds.

A HREF="/builds/BRANCH_BRANDING">Download</A> builds created by the code at A HREF="http://svn-mirror.flock.com/trac/flock/browser/branches/BRANCH">branches/BRANCH</A>.

due to spam protection on this wiki you must manually add the < in front of both "A"

[edit] Customizing fbuilder

fbuilder/scripts

[edit] Network Architecture

Diagram to help explain things on the left hand side are at rackspace and things on the right hand side are behind the firewall in the flock office.

note that when within the office you tinderobox/builds donwload links offer you to download directly from the build archive server http://archive.hq.flock.com/ rather than make you download it over the office Internet collections.

Repackage builds

Retrieved from "http://wiki.flock.com/wiki/FBuilder"

Category: Contribute

This page was last modified 16:38, 26 May 2008.
About Flock Developer Portal
Disclaimers

Flock | the social web browser

FBuilder

Contents

[edit] What is Fbuilder?

[edit] Installing fbuilder

[edit] get the code

[edit] install starup files

[edit] Linux

[edit] Windows

[edit] Mac

[edit] config

[edit] Running fbuilder

[edit] arguments

[edit] thread-config

[edit] config file

[edit] IPC signals

[edit] restarting fbuilder

[edit] Windows

[edit] Macintosh

[edit] start secondary macbuildd

[edit] Linux

[edit] Setup a New Build Tree

[edit] Getting access to Tinderbox

[edit] Preparing tinderbox to recieve builds

[edit] Create the new tree

[edit] create the builds download directory

[edit] make tinderbox changes live on server

[edit] edit the tinderbox message of the day

[edit] Customizing fbuilder

[edit] Network Architecture

Views

Personal tools

Toolbox