Build server structure and workflow

What links here (2) →

Components

Build environment consists of the following components:

• Build core — responsible for building packages from source code. Core is implemented using 'client-server' approach.
• Version control System (VCS) — Git-based version control system. It servers as a storage for source code of all packages.
• Distribution build system — responsible for creation of a final product (distribution ISO image).
• Web UI — this is the main way of interaction between users and build environment.

Workflow

User — VCS — Build environment workflow looks like the following:

Fig. 2.1.1 "User — VCS — Build environment workflow"

Build environment — Repository workflow:

Fig. 2.1.2 “Build environment — Repository workflow”

Repository — Distribution workflow:

Рисунок 2.1.3 “Repository — Distribution workflow”

Note that the diagrams above gives an overview picture, some technical details are missing there.

Operations with packages

The main participant of the data flow is a package
The sequence of operations required to add a new package to repository is shown at Fig. 2.1.1:

1. New package from some outer source is loaded into the system using WebUI or Git command-line interface.
2. The next step is to build the new package.
3. Finally, the package is published in target repository.

Package building process contains the following steps (shown at Fig. 2.1.2):

• An isolated build environment is created in order to perform clean and safe build.
• Source code is complied, if necessary.
• Installation of project files is performed.
• Calculation of check sums for files.
• Finally, the package archive is created.

Adding file to repository:

• Put file into the destination folder
• Update indexes storing information about packages that exist in the repository

The main requirement for repository is data consistency. Simultaneous updates that come to the repository are put into a queue and processed one by one. This allows to avoid "broken dependencies" problem (when some packages cannot be installed because they depend on packages which are missing in the repository).

Distribution building process

Distribution building process (shown at Fig. 2.1.3):

• Creation of isolated environment
• Detection of packages to be installed and their extraction from repositories
• Installation of the base system
• Installation of additional packages
• Preliminary system setup
• Copying files to destination folders
• Create archive of the system
• Configure system bootloader
• Create a local repository
• Create distribution ISO image
• Publish ISO image on server.