ABF Console Client

From Rosalab Wiki
Revision as of 15:54, 18 February 2015 by D uragan (Talk | contribs) (+ skip-proj-cfg-update)

This is a page snapshot, showing old (but not deleted) versions of images and templates.
Jump to: navigation, search

Intro

ABF Console Client aims to simplify remote work with ABF using command line and supports most active stages of a project life cycle - modification, building and publishing.

In ROSA, OpenMandriva and derived systems ABF console client is included in the abf-console-client package and laucnhed by typing abf command.

First launch and configuration

When launched for the first time, ABF console client will ass you for necessary parameters (user name, password, etc.) and some default option (e.g., default group, build platform, etc.). All these settings will be saved to .abfcfg file in your home directory. To change your settings, edit this file or remove it and launch abf command. Note that this file also stores alias definitions which will be lost if you remove the file.


Getting Started

Typical actions with ABF console client can be performed in the following way:

  • Clone project Git repository 
abf get PROJECT

where PROJECT is a project name with owner (owner/proj_name). Owner can be omitted, default group will be used. This command is equivalent of "git clone" invoked with URL to corresponding Git repository.

  • Commit local project modifications and push them to Git repository 
abf put -m MSG

This command first scans the directory for binary files (e.g., tarballs with source code), upload them to ABF File Store and put their hash sums into .abf.yml file. Then it parses spec file and tries to detect Sources which are present in .abf.yml but are no longer used. If these sources are binary files mentioned in .abf.yml, then they are moved to the "removed sources" section of that file so they will not be fetched from File Store when package is being built. Then "abf put" invokes the "git add --all; git commit -m MSG; "git push" sequence of commands.

  • Initiate a build 
abf build

When called without arguments, this command takes a look at active Git branch, asks ABF about repositories this branch is associated with and initiates a build from the branch to that repositories. You can specify branch to be used and target repositories as additional parameters.

  • Check build status 
abf status ID

This command will provide you with a short information about a particular build task. ID of the task is printed to STDOUT by "abf build". You can omit this parameter; in this case status of the last launched project will be checked.

  • Publish build task to repository 
abf publish ID

If you package has been successfully built, you can publish it to repository. Alternatively, you can specify "--auto-publish" option when invoking "abf build". Note that the latter option will work only if automated publishing is allowed by repository settings and if there is no package with the same name, version and released already published to the target repository.

Example

Let's clone import/gcc project, modify something inside it (source tarballs and/or spec) and rebuild the package.

  • clone a git repository an enter its folder
 abf get import/gcc -b rosa2012.1
 cd gcc
  • <some actions with tarballs and spec files>
  • upload source files to File-Store, modify .abf.yml, commit changes with a given message and push everything to repositories
abf put -m "My brand new gcc version"
  • start a new build task on ABF
abf build

(this command prints IDs of initiated build tasks. These IDs are also saved in ~/.abf_projects)

  • check a status of your build task
    • This command will print statuses for the last build task:
abf status
    • or you can specify task IDs explicitly:
abf status <ID1> <ID2> ...
  • if the built has been completed successfully, you can publish it:
abf publish <ID1> <ID2> ...

Command Reference

Below is the list of all commands officially supported in ABF console client. You can also invoke abf --help and use abf help' command to get descriptions of the commands and actions supported by your version of the client.

help

If you want to get a help for some command, you can launch either abf help <command> or abf <command> -h/--help


Invocation and Options:

abf help <command_name>

  • <command_name>: A command to get help for.

add

Add project to repository.

You can't build project to repository unless you explicitly add the project to repository.

Invocation and Options:

abf add [-h] [-p PROJECT] [-v] repository

  • -p PROJECT, --project PROJECT': Project name in <owner>/<name> format.
  • repository': Repository name in <platform>/<repository> format (e.g., rosa2012.1/main).

alias

Manage aliases for console client. With aliases you can make your frequently used commands shorter.

For example, you are going to work only with packages for rosa2012.1, you have to checkout to this branch after every "abf get" (or use -b rosa2012.1). But you can just add an alias like "g: get -b rosa2012.1" and after that "abf g" will do it automatically.

Moreover, the alias name can be not only the first abf argument. For example, you can create another useful alias like "pack: rosa2012.1 build_branch -p" and run "abf copy pack" to copy contents of rosa2012.1 branch to build_branch and compress it.

Invocation and Options:

abf alias <action> param1 [param2] [...]

  • <action>: One of actions: list, add, remove.

add

Add a new alias. the first argument is an alias name, all the arguments after that are alias target. For example, abf alias add sg search groups will срфтпу every call of "abf sg rosa" to "abf search groups rosa".

list

List all the currently available aliases. By default this list looks like 

        b: build
       sp: search projects
       su: search users
       st: status
        s: store
      spl: search platforms
       sg: search groups

remove

Remove an alias. The only option is an alias name.

build

Initiate a build task on ABF. There are lots of options, but in most cases it works great with no options at all. abf-console-client tries to automatically resolve all the options needed. How does it work: read notes and examples.

Invocation and Options:

abf build [--project <project>] [--branch <branch>] [--tag <tag>] [--commit <commit>] [--save-to-repository <repository>] [--repository <repository>] [--arch <arch>] [--auto-publish] [--update-type <type>] [--skip-spec-check]

  • -p/--project <project>: project name ([group/]project). If the option is not specified and you are in a git repository directory - a project name will be resolved automatically.
  • -b/--branch <branch>: git branch name to build. Read notes.
  • -t/--tag <tag>: git tag to build. Read notes.
  • -c/--commit <commit>: git commit sha hash to build. Read notes.
  • -s/--save-to-repository <repository>: repository to save results to. If no platform part specified, it is assumed to be "<default_group>_personal". If this option is not specified at all and you are in a git repository, it will be resolved automatically (read notes).
  • -r/--repository <repository>: repositories to build with. Can be set more than once. If no platform part specified, it is assumed to be your "<default_build_platform>". If no repositories were specified at all, it will be resolved automatically (read notes).
  • -a/--arch <arch>: architectures to build for. Can be set more than once. If not set - use all the available architectures for the project given.
  • --auto-publish: enable automatic publishing. Deprecated synonym for --auto-publish-status=default.
  • --auto-publish-status {default,none,testing}: enable automatic publishing. Default is "default" - that is, use repository settings.
  • --skip-personal: do not use personal repository to resolve dependencies.
  • --testing: Include "testing" subrepository.
  • --no-extra-tests: Do not launch comprehensive tests.
  • --auto-create-container: enable automatic creation of container
  • --cached-chroot: use cached chroot for the build
  • -l/--build-list <build_list>: build list whose container should be used during the build. Can be specified more than once.
  • --save-chroot: save build chroot in case of failure
  • --update-type <type>: Update type, one of {security, bugfix, enhancement, recommended, newpackage}. Default is "security".
  • --skip-spec-check: Do not check spec file.
  • --skip-proj-cfg-update: do not update information about the project in the project cache of console client.

Notes:

Console client tries to automatically resolve all the options without taking into account anything except --branch. If it fails - use only user-given options. If succeeds, but user specified other parameters - discard everything we've resolved and use only user-given options.

Git hash resolving:

  • Only one of --branch, --tag or --commit options can be used at once.
  • If you've specified commit hash - it will be used "as is".
  • If you've specified branch or tag name - commit hash will be resolved automatically using ABF API. (the hash of top commit in the branch will be used for branch)
  • If you've specified a project name - you have to specify a branch, tag or commit manually.
  • If you've specified no branch, tag or commit hash and you've not specified a project name (you have to be in a git repository) - the top remote commit of your current branch will be used.

Other options resolving:

  • if no --arch specified, use all the available project architectures.
  • build repository can usually be resolved from a git branch name. If there is a build platform with the name of your current git branch exists - it will be used.
  • save-to repository: if some repository from a build platform can be used as save-to repository - it will be used.

You can execute abf build (with no options) when you've specified a branch name (or it's resolved automatically) and there is a platform on ABF having the same name. In this case a save-to repository will be selected from a list of available save-to repositories for this platform. Build repositories are repositories of this platform too("main" and one another repository if needed for selected save-to repository).

Examples:

  • Build a current branch of a local cloned project. Build and save-to repositories can be resolved from branch name: abf build
  • Build a project without a local git repository. Build and save-to repositories can be resolved from branch name: abf build --project import/gcc --branch rosa2012.1
  • Build a project to another platform: abf build --project import/gcc --branch rosa2012.1 --save-to-repository rosa2012lts/contrib. Build repositories will be rosa2012lts/main and rosa2012lts/contrib.

chain_build

Initiate a chain of build tasks on ABF. Accepts almost the same set of options as abf build. Note that unlike abf build, chain_build doesn't automatically detect values for different options. You have to manually specify all options necessary for the build - in particular, "save-to" repository and repositories to build with (see example).

Invocation and Options:

abf chain_build [-h] [-i INFILE] [-b BRANCH] [-t TAG] [-c COMMIT] [-u TIMEOUT] [-s SAVE_TO_REPOSITORY] [-a ARCH] [-r REPOSITORY] [-l BUILD_LIST] [--auto-publish] [--auto-publish-status {default,none,testing}] [--skip-personal] [--testing] [--no-extra-tests] [--auto-create-container] [--cached-chroot] [--save-chroot] [--update-type {security,bugfix,enhancement,recommended,newpackage}] [-v] [project [project ...]]

Extra arguments and options with respect to abf build:

  • You can specify several projects to be built one after another. You can also group projects with ":" to indicate that they can be built in parallel. For example, abf chain_build a b:c d will build project "a", then (after "a" is built) will launch builds of "b" and "c" in parallel and after both of these projects are built, the build of "d" will be initiated. If automated publishing is set, then console client waits for every build to be published before starting the next build in the chain. If automated container creation is set, then console client waits for container to be ready and when the next build is started, containers from all previous builds are used as extra repositories.
  • -i/--infile: File with project names. You can omit project names in command line and provide a file with project names instead. The file will be read line by line. All projects specified at the same line will be built in parallel; the next line will be processed only after all the build from the previous line are completed successfully. Project name in a line can be separated by colon (":") or by space symbols.
  • -u/--timeout <timeout>: number of seconds to sleep between successive checks of build status.

Example:

  • Launch build chain - first, build urpmi project, then apache and dracut in parallel and finally launch build of texinfo. Use rosa2014.1 Git branch and save results to abf_personal/main repository. Build with rosa2014.1/main and rosa2014.1/contrib repositories, disable automated publishing and enable container creation:
abf chain_build urpmi apache:dracut texinfo -b rosa2014.1 -s abf_personal/main -r rosa2014.1/main -r rosa2014.1/contrib --auto-publish-status=none --auto-create-container

Build of every successive chain will be initiated only if all builds from the previous chain were completed successfully and only after containers for all of them are created. These containers will be automatically attached to builds of all successive chains.

clean

Scan the spec and .abf.yml files for possible problems. First, it detects missing sources or patches (which are mentioned in spec using non-remote path, but are absent in current directory and .abf.yml). The check will also fail if there are some problems with the spec file itself and RPM can't parse it. These checks are automatically executed when creating a build task from the current directory. If they fail, the console client refuses to create a build task. This behavior can be turned off by --skip-spec-check option for the build command.

In addition, the 'clean' check reports warnings about files specified in .abf.yml and at the same time in the spec file with remote URL. These warnings are not considered to be crucial and don't prevent the build from being started

Invocation and Options:

abf clean [--auto-remove]

  • -a/--auto-remove: automatically remove all the unnecessary files.

copy

Copy files from one git branch to another. Be careful: all the files from destination branch will be removed, and replaced by the files from a source branch.

This command can be useful to keep sources of the project in one branch and build package in another.

Invocation and Options:

abf copy [-h] [-p] [-v] src_branch [dst_branch]

  • <src_branch>: A branch to copy files from.
  • <dst_branch>: A branch to copy files to. If not specified, it's assumed to be the current branch.
  • -p/--pack: Create a tar.gz from the <src_branch> and put this archive and spec file to <dst_branch>.

create

Create a project from SRPM file.

Invocation and Options:

abf create [-h] [-b BRANCH] [--no-def-branch] [-v] srpm [owner]

  • srpm: srpm file
  • owner: who will own the project; default_owner is used by default
  • -b BRANCH, --branch BRANCH: create additional branch; can be set more than once.
  • --no-def-branch: Do not automatically create branch set as default in user config (if it is set to something different from "master").

destroy

Destroy project. Be careful - destroyed project can't be restored!

Invocation and Options:

abf destroy [-h] [-v] project

  • project: project name. ([group/]project). If no group specified, it's assumed to be your default group.

fetch

Download all the files listed in .abf.yml from File-Store to local directory.

Invocation and Options:

abf fetch [--only <file_name>]

  • -o/--only <file_name>: Limit the list of downloaded files to this file name(s). This option can be specified more than once.

fork

Fork project.

Invocation and Options:

abf fork [-h] [-v] source_project [target_project]

  • source_project: project to be cloned, in the <group>/<name> format
  • target_project: <group>/<name> of the new project. If not specified, the source project will be cloned to a project with the same name in your default group

get

Clone a remote git repository by its group and name. For example, abf get import/gcc will clone a project gcc from group import.

Invocation and Options:

abf get <project> [--branch <branch_name>]

  • <project>: a project to clone.
  • -b/--branch <branch_name>: specify a branch to check out inside a cloned git repository.
  • --skip-proj-cfg-update: do not update information about the project in the project cache of console client.

Notes:

You can omit a group name, your default group (given from configs) will be used.

info

Get information about single instance - platform, repository or project.

Invocation and Options:

abf info {platforms,repositories,projects} [-f [FILTER [FILTER ...]]] [-o [OUTPUT [OUTPUT ...]]]

  • -f [FILTER [FILTER ...]], --filter [FILTER [FILTER ...]]: The filter may be specified by defining multiple pairs type.attribute=value or attribute=value, where type is one of the following positional arguments: ['platforms', 'repositories', 'projects'], attribute is the one of the instance fields or special attribute (page - using for pagination) and value - string, that can take asterisk (*) or anything else...
  • -o [OUTPUT [OUTPUT ...]], --output [OUTPUT [OUTPUT ...]]: output format

Example

  • Get names of all projects in rosa2012lts platform: abf info projects -f platforms.name=rosa2012lts page=*

locate

Command to maintain a database of local projects.

Console client tends to maintain a database of local projects: while any interaction with your local git repository via abf-console-client, it stores the path to this repository. After that you can call abfcd [<project_group>/]<project_name> (omit group_name to use your default) to change your current working directory to the project's path (if it had been stored before). If you just want to get a project path without changing directory, use abf locate -p <project>.

You can also add projects to this database by manually. To add only one project call abf locate update -d /path/to/project. To scan directory recursively and add all the found projects, call abf locale update-recursive -d /path/to/directory.

Invocation and Options:

abf locate [<action>] [--project <project>] [--directory <directory>]

  • <action>: one of {update, update-recursive}. update will update
  • -p/--project <project>: Project to show information for (if needed). Format: "[group/]name". If no group specified, your default group will be used.
  • -d/--directory <directory>: Directory to update locations for. It should be a git repository for "update" and any directory for "update-recursive". If not specified - the current directory will be used.

mock-urpm

Build a project locally using mock-urpm. No checkouts will be made, the current git repository state will be used.

Invocation and Options:

abf mock-urpm [-c <config>]

  • -c/--config <config>: A config template to use. Specify one of the config names from /etc/abf/mock-urpm/configs/. Directory path and extension (".cfg") should be omitted. If no config specified, "default.cfg" will be used. Autocompletion worsk for config names.

Notes:

  • Be careful, "abf fetch" will be called before building a package. It means that if you have modified tarball, but .abf.yml contains a hash of an old one - your tarball will be replaced by downloaded from server.
  • Building root can be found in /var/lib/abf/mock-urpm.

proj_alias

Create alias of existing projects. Aliases act like different ABF projects but share the same Git repository.

Invocation and Options:

abf proj_alias [-h] [-v] source_project target_project

  • source_project: project to fork ([group/]project).
  • target_project: target project ([group/]project).

publish

Publish the task that have already been built.

Invocation and Options:

abf publish <task_id> [<task_id>] [...]

  • <task_id>: ID of a build list to publish.

pullrequest

Send a pull request from SRC_BRANCH to DST_BRANCH

Invocation and Options:

abf pullrequest [-h] [-p PROJECT] [-v] from_ref to_ref title body

Positional arguments:

  • <from_ref>: source ref or branch
  • <to_ref>: destination ref or branch
  • <title>: Request title
  • <body>: Request body

Optional arguments:

  • -p PROJECT, --project PROJECT: project name (group/project). By default, the project is used in whose folder this command is launched
-v, --verbose: be verbose, display even debug messages

put

Upload binary files to File-Store and update (or create) .abf.yml file. Can also commit and push changes.

Invocation and Options:

abf put [--message <message>] [--minimal-file-size <size>] [--do-not-remove-files]

  • -m, --message <message>: With this option specified, "git add --all", "git commit -m <message>" and "git push" will be executed.
  • -s, --minimal-file-size <size>: The minimal file size to upload to File-Store. Smaller files will not be processed. Default is 0B.
  • -n, --do-not-remove-files: By default files are being removed on uploading. Override this behavior.
  • -a, --upload-all: By default, console client analyzes spec file and tries to detect which files located in the current folder are really used by the project and uploads only these files to file store. With this option, console client will upload all binary files located in the current folder.

remote

Add remote Git repository and fetch its contents.

Invocation and Options:

abf remote [-h] [-v] remote_group [remote_name]

  • remote_group: ABF group to fetch from. This value will be also used as the name of remote repository.
  • remote_name: Project to fetch (by default the same project name is used as the current one).

Example:

  • Let's first get the foo project from the openmandriva group:
 abf get openmandriva/foo
  • Now let's add remote repository of the same project in the import group:
 abf remote import
  • Voila! Now we have a remote repository named "import" which contains data from the import/foo. We can merge some branch from it, for example:
 git merge import/rosa2014.1

remove

Remove project from repository.

Once you removed project from repository, you will not be able to build it for that repository anymore.

Invocation and Options:

abf remove [-h] [-p PROJECT] [-v] repository

  • -p PROJECT, --project PROJECT': Project name in <owner>/<name> format.
  • repository': Repository name in <platform>/<repository> format (e.g., rosa2012.1/main).

rpmbuild

Build a project locally using rpmbuild. No checkouts will be made, the current git repository state will be used.

Invocation and Options:

abf rpmbuild [-h] [-b {b,s,a}] [-v]

  • -b {b,s,a}, --build {b,s,a}: Build src.rpm (s), rpm (b) or both (a)
  • -v, --verbose: be verbose, display even debug messages

Notes: No checkouts will be made,the current git repository state will be used

search

Search for something on ABF.

Invocation and Options:

abf search <target> "<query>"

  • <target>: what to search for: {users, groups, platforms, projects}.
  • <query>: A string to search for. No regular expressions, just a plain text.

show

Show some general information about the project. Bash autocomplete uses it. If you are in a git repository directory, command will show information about this project (unless -p/--project specified).

Invocation and Options:

abf show <target> [--project <project_name>]

  • <target>: What has to be displayed. Available choices: build-repos, build-platforms, save-to-repos, save-to-platforms
  • -p/--project <project_name>: Project to show information about. If you are in a git repository, the information for <project_name> will be displayed instead.

Notes:

Repository name includes its platform name. Example of repository name: "rosa2012.1/main" - repository "main" of "rosa2012.1" platform.

Build repository - repository that you can connect to a building chroot.

Target (or save-to) repository - when a build task completed successfully, packages can be published to this repository.

status

Show information about a build list. It will print something like that: 

 Buildlist ID:       944492
 User:               akirilenko
 Project:            import/mock-urpm
 Status:             build has been published
 Build for platform: rosa2012.1
 Save to repository: rosa2012.1/main
 Build repositories: [rosa2012.1/main]
 Architecture:       i586
 Created at:         2013-02-12 15:25:09
 Updated at:         2013-02-12 15:43:16

Invocation and Options:

abf status [--project <project>] [--short]

  • -p/--project <project>: Project. If last IDs for this project can be found in local database - use them. Otherwise, print error message end exit.
  • -s, --short: Show only one-line information including id, project, arch and status.

store

Upload a given file to File-Store. Prints a sha1 hash or error message (with non-zero return code).

If the file with same hash already presents on File-Store, it will not be reuploaded. In any case, file hash will be printed.

Invocation and Options:

abf store <path>

  • <path>: A path to file to upload.

test

Execute a set of internal datamodel tests.

update

Change project settings.

Invocation and Options:

abf update [-h] [-p PROJECT] [--name [NAME]] [--desc [DESC]] [--visibility {open,hidden}] [--is_pkg {true,false}] [--branch [BRANCH]] [--issues {true,false}] [--wiki {true,false}] [--biarch {true,false}] [-v]

  • -p PROJECT, --project PROJECT Project to show information for (if needed). Format: "[group/]name". If no group specified, default group will be used.
  • --name [NAME]: New project name.
  • --desc [DESC]: Project description.
  • --visibility {open,hidden}: Project visibility. Please specify "open" or "hidden".
  • --is_pkg {true,false}: Is project a package. Please specify "true" or "false".
  • --branch [BRANCH]: Default branch for the project Git repository.
  • --maintainer [ID_OR_USERNAME]: Project maintainer. You can specify either user name or his ID in ABF.
  • --issues {true,false}: Should project issue tracker be enabled. Please specify "true" or "false".
  • --wiki {true,false}: Should project wiki be enabled. Please specify "true" or "false".
  • --biarch {true,false}: Enable/disable publishing 32bit packages into 64bit repository. Please specify "true" or "false".

Additional features

Cache of local project locations

ABF client remembers locations of all repositories ever cloned using it and you can easily move to particular project directory in your local filesystem using "abfcd" command: 

abfcd import/abf-console-client

This information is stored in .abf_projects file in your home directory.

You can learn the location of any project in your filesystem by executing 

abf locate -p PROJECT

If you already have a set of repositories not known to the console client (e,g,m they were cloned using git directly or you removed your .abf_projects file), you can ask the console client to scan particular folders and add all project it finds there to its local cache: 

abf locate update-recursive -d PATH

where PATH is a directory with repositories.

Information from .abf_projects file is also used by the console client to move files among different projects.

Cache update requires some time, so if you operate with a large set of projects, it can make sense to disable it by specifying --skip-proj-cfg-update option (supported for get, build and chain_build commands).

Bash autocompletion

ABF console client support rich autocompletion in Bash. In particular, it can suggest completions for option names, git branches, build- and save-to repositories for "abf build" and a lot of other parameters. As a result, it's much easier to work with console client, because you don't have to type the long repository name or remember the list of possible variants. In some cases autocompletion may take about 1 second, but remember that its results are always cached, so the next time it will be much more faster.

Retrieved from "http://wiki.rosalab.ru/en/index.php?title=ABF_Console_Client&oldid=8613"