This page aims to get you up to speed developing and modifying Baserock systems, after you have established a Baserock development environment in an x_86 virtual machine, Jetson board or Linux chroot.
Currently the simplest way to get going with Baserock on x86 VM or Jetson is to try the simple build-deploy workflow. This process doesn't work yet for chroot.
The following sections on this page describe specific actions in more detail.
A good way to test that your development system is working is to rebuild the smallest commonly useful Baserock system (the "base system"). To do this, you can checkout the current release branch of the Baserock system morphologies and build from there:
git clone git://git.baserock.org/baserock/baserock/definitions --branch baserock-16.13 cd definitions morph --verbose build systems/base-system-x86_64-generic.morph
NOTE: there used to be
morph checkoutthat you had to use. This command is deprecated now. Morph works in any Git checkout.
x86_32 to build a 32-bit system.
Morph will probably download the components from the artifact cache on git.baserock.org, rather than rebuilding them from scratch. If you want to force it to rebuild everything, you can disable remote artifact fetching with this hack:
morph --verbose --artifact-cache-server=http://example.com/ \ build systems/base-system-x86_64-generic.morph
Note: also as part of the build, Morph populates its cache; this consists of both the git source trees for each individual chunk in the image (in /src/cache/gits) and the compiled binaries (in /src/cache/artifacts). Subsequent builds will go much more quickly because most things are cached; if the upstream source tree for a chunk hasn't changed, Morph can just re-use the corresponding binary in the artifacts directory. If the source tree has changed then the artifact will be rebuilt, but at least now git only needs to fetch the the latest changes rather than the entire tree.
When the build finishes, Morph will give you a path to the built system image, which will be tar file of the root file system. To be used, this needs to be deployed to a disk image for a virtualisation environment. Deployment is a separate step from building the system as you may wish to deploy the same built system to multiple different targets. For example, the same development system we built above can be deployed to KVM, VirtualBox or even real hardware.
To deploy to a raw disk image you need to create a cluster
morphology. Here is an example; put it in a file called
clusters/base-system-rawdisk.morph in your definitions.git clone:
name: base-system-rawdisk kind: cluster systems: - morph: systems/base-system-x86_64-generic.morph deploy: my-raw-disk-image: type: extensions/rawdisk location: base-system.img DISK_SIZE: 4G
Then run the deployment command:
$ morph deploy clusters/base-system-rawdisk.morph
base-system.img will be created in the current directory.
A 4 gigabyte disk size will allow some room for upgrading a system.
Once deployment is complete, copy the newly created image out of the virtual machine and test it in a second VM. The fastest way to copy is probably using netcat, since ssh adds a fairly heavy overhead with encryption.
Alternatively, if using KVM, you can shut down your virtual machine, mount the src filesystem directly as a loopback device (with mount -o loop), and simply copy the file out. Be very sure to shut down the VM before mounting the filesystem in the host and conversely to unmount the filesystem before re-starting your VM.
To deploy to KVM you need ssh access to the host you want to deploy to.
morph deploy will create a new VM on the host. Below is an example KVM
name: base-system-kvm kind: cluster systems: - morph: systems/base-system-x86_64-generic.morph deploy: my-kvm-system: type: extensions/kvm location: kvm+ssh://user@host/my-kvm-system/path/to/vm.img DISK_SIZE: 4G RAM_SIZE: 2G VCPUS: 1 HOSTNAME: my-kvm-system
Make sure the location path is correct. It should be the name of your VM followed by a path to the VM's disk image.
To deploy, run the deployment command:
$ morph deploy clusters/base-system-kvm.morph
Morph will ssh into
user and create a file
user must have permission to create in
To deploy to VirtualBox you must be able to access the host that runs VirtualBox
ssh from within your development VM.
morph deploy creates a new virtual machine that runs
the system image you've just built, and sets up the networking to allow the VM to communicate
with the host and with the network
The parameters to
morph deploy - as shown in the example below - are as follows:
SYSTEMis the name of the system you have just built
LOCATIONis a custom URL constructed from the following
- How to log in via ssh to the host that runs VirtualBox:
email@example.com the example below.
aliceis the username, and
192.168.122.1is the address, as seen from the development VM, of the host. The address may be a private IP address provided by VirtualBox, if your development VM is running on the same physical host.
- The name of the new virtual machine:
devsysin the example. Note that this is not part of the filename on the host machine.
- The filename on the host machine for the new virtual machine's
/home/alice/devsys.vdiin the example.
- How to log in via ssh to the host that runs VirtualBox:
DISK_SIZE- the size of the disk image to create, which is 2.00 GB in this case
In addition to the
DISK_SIZE parameters, you must supply
values for the following
HOST_IPADDR- the ip address of the host-only network adapter on the VirtualBox host
NETMASK- the netmask of the host-only network adapter on the host
NETWORK_CONFIG- a string containing configuration information for the
eth1interfaces in the VM:
eth0will be connected to the host-only network adapter;
eth1will be connected to the NAT network adapter on the host and will be used by the VM to access the outside world.
Here is a cluster morphology using these parameters:
name: base-system-virtualbox kind: cluster systems: - morph: systems/base-system-x86_64-generic.morph deploy: my-base-system-virtualbox: type: extensions/virtualbox-ssh location: vbox+ssh://firstname.lastname@example.org/devsys/home/alice/devsys.vdi DISK_SIZE: 2G HOST_IPADDR: 192.168.100.1 NETMASK: 255.255.255.0 NETWORK_CONFIG: lo:loopback;eth0:static,address=192.168.100.2,netmask=255.255.255.0;eth1:dhcp,hostname=$(hostname)
Change the location above to work with your system, and then run it to create and start the VM.
When the VM has finished booting into Baserock booting up, log in as root.
Bash isn't the default shell, so you will need to run it
manually after logging in by typing
bash, after which you'll be greeted by
the prompt that you specified in your branch of the Bash source code.
To deploy to OpenStack you need to create a cluster morphology with specific parameters:
locationis the URL to the OpenStack identity service.
OPENSTACK_USERis the deployer's OpenStack username.
OPENSTACK_TENANTis the OpenStack project for this image.
OPENSTACK_IMAGENAMEis a name to identify the image in OpenStack.
DISK_SIZE- the size of the disk image to create. If you deploy an image with
cloud-init(as in the following example), the disk will be resized to fit the maximum space given in OpenStack.
The following field must also be set, but we recommend passing it on the commandline:
OPENSTACK_PASSWORDis the password of the OpenStack user.
Here is an example of a cluster morphology to deploy a system with
integrated, put it in a file called
the definitions repository checkout.
name: base-system-openstack kind: cluster systems: - morph: systems/base-system-x86_64-generic.morph deploy: openstack-image: type: extensions/openstack location: http://openstack_host:5000/v2.0/ OPENSTACK_USER: your_user OPENSTACK_TENANT: your_project OPENSTACK_IMAGENAME: the_imagename DISK_SIZE: 3G CLOUD_INIT: yes KERNEL_ARGS: console=ttyS0 console=tty0
Make sure DISK_SIZE is enough to fit the system you are trying to deploy. If you are going to deploy a development system you may need 3G.
Then run the deployment command:
$ morph deploy clusters/base-system-openstack.morph openstack-image.OPENSTACK_PASSWORD=the_user_password
Be aware that your shell could save the the full command, including your password, in your shell history file.
Once deployment is complete, it's possible to launch an instance with the new
image in OpenStack. Also, if the system deployed has
cloud-init you can pass
#cloud-config customization scripts using the "Post-creation" tab during the
launch of the instance.
WARNING: if your OpenStack authentication token expires during the image
upload, the deployment will probably fail. See:
https://bugs.launchpad.net/glance/+bug/1182536. To work around this, you or
the administrator of your OpenStack cloud may be able to increase the
expiration time of tokens. Otherwise, you will need to either reduce the size
of the system you are trying to deploy, or run
morph deploy from a machine
which has fast enough upload speed to the cloud. A
within the cloud that you're deploying to works well for this!
To update morph, see this guide: http://wiki.baserock.org/using-latest-morph/
See the 'upgrading Baserock systems' guide.
From inside your VM, you can make a change to the code of the system you are running and build a new system image to test your change. To do this, clone the definitions repo (or use your existing clone), and checkout a new feature branch to work in:
$ git clone git://git.baserock.org/baserock/baserock/definitions $ cd definitions $ git checkout baserock-16.13 -b test-branch
This Git repo contains the definitions for the Baserock 'build' or 'devel' system that you are running. The system definition will be in the systems/ subfolder. You can find out exactly what system you are running by looking in the /baserock directory, which will have a corresponding .meta file -- for example, a system built from systems/build-system-x86_32.morph will contain the file /baserock/build-system-x86_64-rootfs.meta.
You can edit the system .morph file, or the stratum .morph files that it refers to, in order to add or remove components from your new system. You can also edit code of existing components. For example:
$ morph get-repo bash $ cd ../bash $ git status HEAD detached at 3590145 nothing to commit, working directory clean
This is a normal git repository, probably cloned from
morph get-repo has checked out the ref
that is specified in
strata/core.orph, where GNU Bash was defined. You could
git clone to fetch the repo --
morph get-repo is just a shortcut.
If you want to make a change to GNU Bash and build a system with that change, you should first check out a branch:
$ git checkout -b my-bash-feature
Make your code changes. You probably want to test them locally before rebuilding
the whole system with the new version of GNU Bash. Once you are ready to test a
Baserock system with the patched version of Bash, you can edit
to look in your local clone of bash.git, by changing the list entry that starts
'name: bash' to this:
- name: bash morph: strata/core/bash.morph repo: file:///src/bash ref: my-bash-feature
This assumes that your bash.git clone is in /src.
You now need to build the new system image, which can be done using Morph. The uncommitted changes in definitions.git will be included on a temporary branch, provided the 'local-changes' setting is set to 'include'. This feature should only be used for local testing, of course! You shouldn't publish definitions containing file:/// URLs, because nobody else will be able to access the repos.
$ morph build systems/base-system-x86_64-generic.morph
After the build completes, you can use
morph deploy as above to create a
disk image to copy out of your VM and try to boot.
If you want to publish your change, you will need to host your bash.git fork and
your definitions.git fork on a public Git server (Github, Gitlab, your own Git
server, etc.), and change the
repo: field for 'bash' to point to that public
repo (e.g. git://github.com/youraccount/bash) instead of your local copy.
This is our current workflow for adding new software into a Baserock system.
- Get the source into git, if it is not already.
- Add the program as a chunk in (a stratum in) a Baserock system morph file
- Add the program's build dependencies to the system as required until it builds
Start with a working VM. Prove it's working by building something, eg base-system.
Fetch the definitions repo (or use an existing clone).
$ git clone git://git.baserock.org/baserock/baserock/definitions $ cd definitions $ git checkout -b my-feature-branch
Then retrieve the source you're working with, in another directory.
- If the source came as git, great.
- If the upstream is not git, you'll need to get it into git - either by creating your own git repo, or with Lorry, or other magic.
In the definitions clone:
- create your own system file based on one of the templates in systems/*
- add a new stratum to it
- create your new stratum file
- add new chunk(s) in that for your component(s), specifying repo as "url://to/upstream/repo.git" or "file:///src/your-git-repo"
- the ref field can be any valid Git ref (such as 'master') while prototyping. Current policy is that in definitions.git master the ref field must point to a fixed commit SHA1 so that it doesn't change unexpectedly.
Now you can try
$ morph build systems/your-new-system.morph
Unless the build system of the software you're integrating follows a known pattern, you will also need to write a 'chunk morphology' describing how to build it. You will get an error 'couldn't find morphology' if you need to do this. There are plenty of existing chunk morphologies in various subdirectories of the 'strata/' directory in definitions.git.
There is an informal specification of the definitions format.
See build-failures for help on build failures
If you are planning to develop Morph itself, see using-latest-morph.