"Change an existing component" tutorial

Aim

  • Show how to make a change to an existing component of a system in a feature branch using morph branch and morph edit
  • Introduce the related help and reference documentation
  • Introduce the branch naming convention

Task(s) to be completed

  • Create a new feature branch
  • Make a change in GNU Bash component
  • Build and deploy the changed system

Start state

The end state of the "Branch a system" tutorial

  • a functioning development environment
  • a populated Trove
  • a populated artifact cache
  • a built Baserock system
  • no system branches

End state

  • the chnaged system running on a deployment node

Steps

  • morph branch
  • morph edit
  • show what morph edit has done
  • edit the Bash code
  • build the changed system
  • deploy the changed system

Commentary on what is happening

Public

Posted Fri Oct 17 13:21:40 2014

Tutorial 3.0 - "Change an existing component"

In this tutorial we are going to learn how to make a change to an existing component of our system. We are going to use morph branch which we've already used, and morph edit which we haven't.

The starting point for this tutorial is the point we got to at the end of tutorial 2.1, the "Create from template" tutorial. We now have:

  • a functioning development environment
  • a built Baserock system
  • our own master branch
  • a new system on that master branch

Baserock has many parallels with git. In git it is good practice when making changes to make them on a feature branch where they can be implemented, tested, reviewed and re-worked. The changes are only merged back to master when they are ready, ensuring that master continues to work.

We're going to update our version of openssh, so we get new features and bug fixes.

So we'll go to our workspace

bash-4.2# cd /src/workspace
bash-4.2# pwd
/src/workspace

We'll start by creating a feature branch, based on our project's master branch.

This is our command:

bash-4.2# morph branch baserock:baserock/morphs \
    TROVE_ID/joebloggs/update-ssh \
    TROVE_ID/joebloggs/master

which is saying that:

  • we'll create a new branch of the baserock:baserock/morphs repository;
  • we'll call the branch TROVE_ID/joebloggs/update-ssh, identifying that it's developer joebloggs working for a company whose Trove ID is TROVE_ID, and the topic of their branch is updating ssh.
  • We will base it on our existing TROVE_ID/joebloggs/master branch.

    If we didn't have this last parameter, the new branch would be based on master, which comes from upstream and would lack all our work done thus far.

bash-4.2# morph branch baserock:baserock/morphs \
    TROVE_ID/joebloggs/update-ssh \
    TROVE_ID/joebloggs/master
2013-06-06 11:32:54 Updating git repository baserock:baserock/morphs in cache
2013-06-06 11:33:47 Updating git repository baserock:baserock/morphs in cache

If we run morph status we'll see that we now have a system branch checkout directory

bash-4.2# morph status
System branches in current workspace:
    master
    TROVE_ID/joebloggs/master
    TROVE_ID/joebloggs/update-ssh

Now for the new stuff - we're going to use the morph edit command.

Let's look at morph help edit

bash-4.2# morph help edit
Usage: morph [options] edit SYSTEM STRATUM [CHUNK]

Edit or checkout a component in a system branch.
...

The important arguments that we need are:

  • our system: my-x86_64-system
  • the stratum we are changing: foundation
  • the chunk we are changing: openssh

So our command is

morph edit my-x86_64-system foundation openssh

which is saying that we are going to make a change to the openssh chunk of the foundation stratum of the my-x86_64-system system. Let's run that now. We need to run it within the system branch checkout directory.

bash-4.2# cd TROVE_ID/joebloggs/update-ssh
bash-4.2# morph edit my-x86_64-system foundation openssh
2013-06-06 11:54:22 Updating git repository upstream:openssh in cache

Let's look in more detail at what morph edit has done.

bash-4.2# morph status
On branch TROVE_ID/joebloggs/update-ssh, root baserock:baserock/morphs
    baserock:baserock/morphs: uncommitted changes

So we'll look in the baserock:morphs directory and look at what uncommitted changes there are.

bash-4.2# cd baserock:baserock/morphs/
bash-4.2# git status
# On branch TROVE_ID/joebloggs/update-ssh
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#   modified:   my-x86_64-system.morph
#   modified:   foundation.morph
#
#no changes added to commit (use "git add" and/or "git commit -a")

So git status tells us that we have uncommitted changes to two of the .morph files in this directory.

In our morph edit command we said we were changing the foundation stratum which is part of the my-x86_64-system system so we would expect those two .morph files to have changed.

If we had changed core, we would have seen both its, and the bsp-x86_64-generic morphologies, since if a stratum is changed, any morphologies that depend on it are also changed.

First let's look at the changes in foundation.morph.

bash-4.2# git diff foundation.morph
diff --git a/foundation.morph b/foundation.morph
index e4efe89..b1d6066 100644
--- a/foundation.morph
+++ b/foundation.morph
@@ -108,10 +108,9 @@ chunks:
   unpetrify-ref: baserock/morph
 - name: openssh
   repo: upstream:openssh
-  ref: 6d43023b30937b1744703d64790f0117ac1992eb
+  ref: TROVE_ID/joebloggs/update-ssh
   build-depends:
   - groff
-  unpetrify-ref: baserock/morph
 - name: tbdiff
   repo: baserock:baserock/tbdiff
   ref: 2fdfa3149be9ae43fb23cda6459e841fb3a90698

This saying that we will use our new branch of openssh, and that we are no longer working with the version that was locked down when we ran morph petrify in our TROVE_ID/joebloggs/master branch.

Now let's look at the changes in my-x86_64-system.morph

bash-4.2# git diff my-x86_64-system.morph
diff --git a/my-x86_64-system.morph b/my-x86_64-system.morph
index 3801c1b..705b02f 100644
--- a/my-x86_64-system.morph
+++ b/my-x86_64-system.morph
@@ -14,7 +14,7 @@ strata:
   ref: TROVE_ID/joebloggs/master
 - morph: foundation
   repo: baserock:baserock/morphs
-  ref: TROVE_ID/joebloggs/master
+  ref: TROVE_ID/joebloggs/update-ssh
 - morph: bsp-x86_64-generic
   repo: baserock:baserock/morphs
   ref: TROVE_ID/joebloggs/master

We see that our system now depends on our branched version of the foundation stratum.

That's not all that happened when we ran morph edit. If we look in our branch checkout directory

bash-4.2# cd ../../
bash-4.2# pwd
/src/workspace/TROVE_ID/joebloggs/update-ssh
bash-4.2# ls
baserock:baserock  upstream:openssh

we see that, as well as baserock:baserock/morphs, our branch now contains a clone of the upstream repository for openssh, and this is where we are going to make our change.

If we go into that directory, git will confirm that we have our own version of the source code checked out, although the version used by TROVE_ID/joebloggs/master is still also available:

bash-4.2# cd upstream:openssh
bash-4.2# git branch
  baserock/morph
* TROVE_ID/joebloggs/update-ssh
bash-4.2# git status
# On branch TROVE_ID/joebloggs/update-ssh
nothing to commit (working directory clean)

We are going to merge with the V_6_2_P2 tag, since that was the newest at the time of writing, and corresponds to version 6.2p2.

We attempt this with the following command:

bash-4.2# git merge V_6_2_P2
Auto-merging sshd_config
CONFLICT (content): Merge conflict in sshd_config
Removing cipher-acss.c
Removing acss.h
Removing acss.c
Automatic merge failed; fix conflicts and then commit the result.

This has failed to merge cleanly because Baserock changed the default sshd_config to enable PermitUserEnvironment, and that is next to a setting that was changed upstream.

We can view the change by running:

bash-4.2# git diff sshd_config
diff --cc sshd_config
index ca2a509,9cd2fdd..0000000
--- a/sshd_config
+++ b/sshd_config
@@@ -97,8 -102,8 +102,13 @@@ AuthorizedKeysFile .ssh/authorized_key
  #PrintLastLog yes
  #TCPKeepAlive yes
  #UseLogin no
++<<<<<<< HEAD
 +#UsePrivilegeSeparation yes
 +PermitUserEnvironment yes
++=======
+ UsePrivilegeSeparation sandbox                # Default for new installations.
+ #PermitUserEnvironment no
++>>>>>>> V_6_2_P2
  #Compression delayed
  #ClientAliveInterval 0
  #ClientAliveCountMax 3

Resolve the merge however you like, I went with upstream's version. Once you've made the change git add it, then commit the merge.

bash-4.2# git add sshd_config
bash-4.2# git commit -m 'Merge with release 6.2p2'
[TROVE_ID/joebloggs/update-ssh a6c1599] Merge with release 6.2p2

Building a new system image to test the change is very simple. Morph requires that all changes to the source code are committed when you want to build, but morph build will handle this for you to make life easier during development by creating a temporary build ref.

NOTE: morph build detects changes in files already known to git, but if you have added extra files to a project's source code then you will need to add them with git add and commit them to the repository before building the system branch.

bash-4.2#   morph build my-x86_64-system
2013-06-07 07:00:19 Starting build 6f288ae6042147d3af0193235e20deb1
....
2013-06-07 07:08:13 Build ends successfully
2013-06-07 07:08:13 Finished build 6f288ae6042147d3af0193235e20deb1

If you were to try and build from /src/workspace now you would get an error

bash-4.2# morph build my-x86_64-system
ERROR: Can't find the system branch directory.
Morph must be built and deployed within the system branch checkout.

During the build we can see that

  • chunks which don't depend on the changed stratum (foundation) are not rebuilt
  • openssh itself is built
  • the foundation stratum and the system rootfs are rebuilt

This took about 5 minutes on an x86 Virtual Machine, if we had changed a component with many build dependencies this could have taken significantly longer.

Once the build has finished, we can deploy the built system as we learned to do in an earlier tutorial.

bash-4.2# morph deploy --no-git-update kvm \
    my-x86_64-system \
    kvm+ssh://joebloggs@192.168.122.143/update-ssh/home/joebloggs/VMs/update-ssh.img \
    DISK_SIZE=2G HOSTNAME=update-ssh AUTOSTART=yes

If we login as root we can run ssh -V to see the version.

~ # ssh -V
OpenSSH_6.2p2, OpenSSL 1.1.0-dev xx XXX xxxx

Now we know that it worked, we should commit and push our changes, as otherwise nobody else can see them. The commands to commit and push the changes are:

bash-4.2# cd ../baserock\:baserock/morphs/
bash-4.2# git commit -a -m 'Update OpenSSH to 6.2p2'
[TROVE_ID/joebloggs/update-ssh fe587c8] Update OpenSSH to 6.2p2
 2 files changed, 2 insertions(+), 3 deletions(-)
bash-4.2# morph foreach -- sh -c 'git push -u origin HEAD 2>&1'
baserock:baserock/morphs
To ssh://git@TROVE_HOST/baserock/baserock/morphs
 * [new branch]      HEAD -> TROVE_ID/joebloggs/update-ssh
Branch TROVE_ID/joebloggs/update-ssh set up to track remote branch TROVE_ID/joebloggs/update-ssh from origin.

upstream:openssh
To ssh://git@TROVE_HOST/delta/openssh
 * [new branch]      HEAD -> TROVE_ID/joebloggs/update-ssh
Branch TROVE_ID/joebloggs/update-ssh set up to track remote branch TROVE_ID/joebloggs/update-ssh from origin.
Posted Fri Oct 17 13:21:40 2014