Tutorial 3.4 - "Adding a component to a System"
In this tutorial we are going to learn how to add a component to a
system. The component we're going to add is the filesystem benchmarking
tool bonnie
The starting point for this tutorial is at least after tutorial 2.1 - "Creating a new system from a template". We now have:
- a functioning development environment
- our project master branch at
TROVE_ID/joebloggs/master
This time we will demonstrate by working on the project's master branch. However we will make the changes in our own system rather than modifying the base system.
According to the Adding stuff page on the baserock.org wiki the steps involved in adding a new component are:
- Get the component source
- Build the software in its normal way, without Baserock tools.
- Get the source into git, if it is not already.
- Add the program to a Baserock system using morphologies, and build that.
The source for 'bonnie' lives in an svn repository at https://code.google.com/p/bonnie-64/ and it has been lorried on the Trove for this tutorial. To get the source, we go to our system branch checkout directory and clone the git repository from the Trove
cd /src/workspace/TROVE_ID/joebloggs/master
git clone ssh://git@TROVE_HOST/delta/bonnie.git upstream:bonnie
cd upstream:bonnie
Building bonnie
is relatively straightforward, we could simply run make
.
However, to add the program to our Baserock system we need to do the following
- Create a chunk morphology to say how to build Bonnie and push that branch to the Trove
- Create a new
fstools
stratum, since there is no obvious stratum for it to join. - Add the stratum to the system
Creating a Bonnie chunk
in the
upstream:bonnie
directory, create and checkout a new projectmaster
git branchbash-4.2# git checkout -b TROVE_ID/joebloggs/master
create the morphology file
bonnie.morph
with the following content:bash-4.2# cat >bonnie.morph <<\EOF kind: chunk name: bonnie build-commands: - make install-commands: - mkdir -p "$DESTDIR/$PREFIX"/bin - install -o 0 -g 0 -m 0755 Bonnie "$DESTDIR/$PREFIX"/bin/bonnie EOF
git add
andgit
commit the new filebash-4.2# git add bonnie.morph bash-4.2# git commit -m "Morphology for building bonnie in Baserock"
git push
the new branch, so thatmorph build
can find itbash-4.2# git push -u origin HEAD
Creating the fstools
stratum
In the morphs repository, create the file
fstools.morph
with the following content:bash-4.2# cd ../baserock:baserock/morphs bash-4.2# cat >fstools.morph <<\EOF name: fstools kind: stratum build-depends: - morph: build-essential repo: baserock:baserock/morphs ref: TROVE_ID/joebloggs/master chunks: - name: bonnie repo: upstream:bonnie ref: TROVE_ID/joebloggs/master build-depends: [ ] EOF
git add
andgit
commit the new filebash-4.2# git add fstools.morph bash-4.2# git commit -m \ "Create a stratum containing Bonnie"
Adding the fstools
stratum to our System
In
baserock:baserock/morphs
, add the following lines to my-x86_64-generic.morph, before theconfiguration-extensions
line.- morph: fstools repo: baserock:baserock/morphs ref: TROVE_ID/joebloggs/master
If you've done this correctly the diff should look something like this:
bash-4.2# git diff my-x86_64-system.morph diff --git a/my-x86_64-system.morph b/my-x86_64-system.morph index 3801c1b..0564f4c 100644 --- a/my-x86_64-system.morph +++ b/my-x86_64-system.morph @@ -18,6 +18,9 @@ strata: - morph: bsp-x86_64-generic repo: baserock:baserock/morphs ref: TROVE_ID/joebloggs/master +- morph: fstools + repo: baserock:baserock/morphs + ref: TROVE_ID/joebloggs/master configuration-extensions: - set-hostname - ssh
We don't need to commit the changes yet, as
morph build
will pick them up automatically.
Building, deploying and testing the system
So now we should be able to build our system, with bonnie
added
bash-4.2# morph build my-x86_64-system
and deploy it
bash-4.2# morph deploy --no-git-update kvm my-x86_64-system \
kvm+ssh://joebloggs@192.168.56.1/bonnie/home/joebloggs/VMs/bonnie.img \
DISK_SIZE=2G HOSTNAME=bonnie AUTOSTART=yes
If we log into the deployed system we can run bonnie
# bonnie
File './Bonnie.176', size: 104857600
Writing with putc()...done
Rewriting...done
Writing intelligently...done
Reading with getc()...done
Reading intelligently...done
Seeker 3...Seeker 2...Seeker 1...start 'em...done...done...done...
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine GB M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU /sec %CPU
0 17.2 20.7 26.8 4.6 20.6 2.9 113.0 100.0 6626.9 99.4 130115 97.6
So there we are - we've added a component to our new system, so we should commit and push our changes.
# morph foreach -- git commit --amend -a -m 'Add Bonnie to my-x86_64-system'
baserock:baserock/morphs
[TROVE_ID/joebloggs/master 45f5495] Add Bonnie to my-x86_64-system
2 files changed, 5 insertions(+), 2 deletions(-)
bash-4.2# morph foreach -- sh -c 'git push -u origin HEAD 2>&1'
To ssh://git@git.baserock.org/baserock/baserock/morphs
1ede03f..f5b3192 HEAD -> TROVE_ID/joebloggs/master
Branch TROVE_ID/joebloggs/master set up to track remote branch TROVE_ID/joebloggs/master from origin.
Common pitfalls
There are some new errors we may see while doing this (as well as the ones we talked about in the earlier tutorials):
If we didn't
git commit
the newfstools.morph
file inbaserock:baserock/morphs
2013-07-11 16:19:01 baserock:baserock/morphs: Update morphologies to use build branch instead of "TROVE_ID/joebloggs/master" ERROR: Command failed: git update-index --cacheinfo 100644 f9300450c9451fc86278f9323e9f518788c144b4 fstools.morph error: fstools.morph: cannot add to the index - missing --add option? fatal: git update-index: --cacheinfo cannot add fstools.morph
If we didn't
git commit
the newbonnie.morph
file in in thebonnie
repository2013-07-11 16:23:19 Updating cached git repository upstream:bonnie ERROR: Failed to determine the build system of repo upstream:bonnie at ref 9007adb991231bd0f9242bb30c0b580f7c8f16b2: was looking for bonnie.morph
If you haven't pushed the branch of bonnie, you will get an error like the following when you try to build it.
2013-07-05 13:10:50 Updating cached git repository upstream:nasm 2013-07-05 13:10:51 Updating cached git repository upstream:syslinux 2013-07-05 13:10:52 Caching git repository upstream:bonnie ERROR: Ref TROVE_ID/joebloggs/master is an invalid reference for repo git://git.baserock.org/delta/bonnie
If we do our first build with the
--no-git-update
option:ERROR: Repository upstream:bonnie is not cached yet
(It's OK to use
--no-git-update
on subsequent builds as the source will already be cached)If we didn't add build-depends to the bonnie stratum morphology
2013-07-05 13:13:30 [Build 77/79] [bonnie] Running build-commands build failed # build # # make execv: No such file or directory ERROR: In staging area /src/tmp/staging/tmp5Dki6P: running command 'sh -c make' failed.
If the build-depends of fstools are strata that haven't been modified (e.g. it depends on build-essential and you haven't run
morph edit my-x86_64-system build-essential
)ERROR: Ref TROVE_ID/joebloggs/master is an invalid reference for repo git://git.baserock.org/baserock/baserock/morphs