ABF Console Client

From Rosalab Wiki
Revision as of 13:18, 16 April 2013 by Anton.kirilenko (Talk | contribs)

This is a page snapshot, showing old (but not deleted) versions of images and templates.
Jump to: navigation, search

Intro

Let's take a look at package life-cycle. User wants to create (or get existing) project, modify it, build and publish. Let's start in order.

  1. Clone git repository. A project name is assumed to be known. User have to open ABF web page, search for a project, go to a project page, copy git URL, clone repository using git. However, it can be done easier. Just execute "abf get PROJECT", where PROJECT is a project name with owner (owner/proj_name). Owner can be omitted, default group will be used.
  2. Project modification. When you've got files in your project modified, you can execute "abf put -m MSG". It will execute "git add --all", "git commit -m MSG", "git push". Additionally, all the source archives have to be uploaded to File-Store and .abf.yml file have to modified. It's a routine work, especially when you have to build a hundred projects a day. Console client will do it for you automatically.
  3. Create a build-task. To do it, you have to open an ABF web page, click some check-boxes and click 'Start build'. It looks like a few work to do. But if you have to build hundreds packages a day, you can do it much faster using console client, especially if you write your own script executing console client.
  4. Building process. To check the current building status, you don't have to check web page anymore. Just execute "abf buildstatus ID" to get a short summary. ID can be omitted and the status for the last built project. It will be described later in detail.
  5. Publication. "abf publish ID" will do it easily.

As you can see, console client makes maintainer's work easier a bit.

The list of some features:

  • ABF client caches the locations of git-repositories in your system when it accesses them. If you have already got lots of repositories, you can cache their locations at one go. Just run "abf locate update-recursive -d PATH", where PATH is a directory with repositories. For what perpose should one know the location of every repository? For example, user can now execute "abfcd PROJECT". You can just learn the location of any project by executing "abf locate -p PROJECT". Console client will be able to move files not only among the project branches, but also among projects.
  • First version of console client had only got the basic version of bash autocomplete script. Now it autocompletes almost everything: option names, git branches, build and save-to repositories for "abf build" (the latter requires a project name to be specified, so you have to type the --project/-p before --save-to-repository/-s ). As a result, it's much easier to work with console client, because you don't have to type the long repository name or remember the list of possible variants. Sometimes autocompletion works long (about 1 second), but the results are getting cached and the process is speeding up.
  • The new command, "abf clean" appeared. It scans the spec and .abf.yml files and current directory and shows problems found. If some source or patch file can not be found (or resolved via URL or .abf.yml) - it prints an error message. It also prints warnings, for example, if a file is specified in .abf.yml and in spec file. The test is automatically executed while creating a build task from the current directory and prevents a build task from being sent. This check can be turned off by option --skip-spec-check.
  • When user creates a new build task, its ID will be stored and associated with a project. "abf buildstatus" will print the summary for the last build task. If project was specified (or you are in a project directory) - it will print information about the last build task for this project.
  • About work with API. The number of unnecessary API calls have been significantly decreased. The fact is that when you, for example, requests information about repository, API returns some particular information about platform too. The older version could only use the ID of this platform and downloaded the full information about platform by one more API call. Now this information is stored in platform object that is marked as 'stub'. When you try to access a field that is not loaded but have to present in this class - a new API call will be made and information will be retrieved. As a result, the first console client call doesn't results in dozens of API calls anymore.
  • One more interesting feature - work with (http://en.wikipedia.org/wiki/HTTP_ETag) to cache the results of API calls with autovalidation. It results in speeding the process up without a risk of working with the outdated data. ABF is not well configured now to fully support this technology, but it's going to process requests for cached data fast. It will also stop increasing a counter of API calls (every IP and login are limited to 2000 API calls per hour).

As you can see, the ABF console client development continues. All your feedback (either positive or negative), suggestions and regards are welcome.

Workflow Examples

Update package

You want to clone a project import/gcc, modify something (modify source tarballs and/or spec) and rebuild the packages. 

 # clone a git repository
 abf get import/gcc -b rosa2012.1
 cd gcc
 <some actions with tarballs and spec files>
 
 # upload source files to File-Store, 
 # modify .abf.yml and commit changes
 # with comment given and push
 abf put -m "My brand new gcc version"
 
 # now you can start a new build task on ABF.
 # all the arguments will be resolved automatically.
 abf build
 # it prints IDs of build tasks that were sent.
 # These IDs are also saved in ~/.abf_projects
 
 # check a status of your build task.
 # This command will print statuses for last 
 # build tasks:
 abf status

 # or you can specify IDs explicitly:
 abf status <ID1> <ID2> ...
 
 # if the task has been built successfully,
 # you can try to publish it:
 abf publish <IF1> <ID2> ...

Command List

help

If you want to get a help for some command - you've got two options: abf help <command> and abf <command> -h/--help

Options:

abf hel <command_name>

  • <command_name>: A command to get help for.

alias

Manage aliases for console client. With aliases you can make your frequently used commands shorter.

For example, you are going to work only with packages for rosa2012.1, you have to checkout to this branch after every "abf get" (or use -b rosa2012.1). But you can just add an alias like "g: get -b rosa2012.1" and after that "abf g" will do it automatically.

Moreover, the alias name can be not only the first abf argument. For example, you can create another useful alias like "pack: rosa2012.1 build_branch -p" and run "abf copy pack" to copy contents of rosa2012.1 branch to build_branch and compress it.

Options:

abf alias <action> param1 [param2] [...]

  • <action>: One of actions: list, add, remove.

list

List all the currently available aliases. By default this list looks like 

        b: build
       sp: search projects
       su: search users
       st: status
        s: store
      spl: search platforms
       sg: search groups

add

Add a new alias. the first argument is an alias name, all the arguments after that are alias target. For example, abf alias add sg search groups will срфтпу every call of "abf sg rosa" to "abf search groups rosa".

remove

Remove an alias. The only option is an alias name.

get

Clone a remote git repository by its group and name. For example, abf get import/gcc will clone a project gcc from group import.

Options:

abf get <project> [--branch <branch_name>]

  • <project>: a project to clone.
  • -b/--branch <branch_name>: specify a branch to check out inside a cloned git pepository.

Notes:

You can omit a group name, your default group (given from configs) will be used.

put

Upload binary files to File-Store and update (or create) .abf.yml file. Can also commit and push changes.

Options:

abf put [--message <message>] [--minimal-file-size <size>] [--do-not-remove-files]

  • -m/--message <message>: With this option specified, "git add --all", "git commit -m <message>" and "git push" will be executed.
  • -s/--minimal-file-size <size>: The minimal file size to upload to File-Store. Smaller files will not be processed. Default is 0B.
  • -n/--do-not-remove-files: By default files are being removed on uploading. Override this behavior.

store

Upload a given file to File-Store. Prints a sha1 hash or error message (with non-zero return code).

If the file with same hash already presents on File-Store, it will not be reuploaded. In any case, file hash will be printed.

Options: abf store <path>

  • <path>: A path to file to upload.


fetch

Download all the files listed in .abf.yml from File-Store to local directory.

Options:

abf fetch [--only <file_name>]

  • -o/--only <file_name>: Limit the list of downloaded files to this file name(s). This option can be specified more than once.

show

Show some general information about the project. Bash autocomplete uses it. If you are in a git repository directory, command will show information about this project (unless -p/--project specified).

Options:

abf show <target> [--project <project_name>]

  • <target>: What has to be displayed. Available choices: build-repos, build-platforms, save-to-repos, save-to-platforms
  • -p/--project <project_name>: Project to show information about. If you are in a git repository, the information for <project_name> will be displayed instead.

Notes:

Repository name includes its platform name. Example of repository name: "rosa2012.1/main" - repository "main" of "rosa2012.1" platform.

Build repository - repository that you can connect to a building chroot.

Target (or save-to) repository - when a build task completed successfully, packages can be published to this repository.

locate

Command to maintain a database of local projects.

Console client tends to maintain a database of local projects: while any interaction with your local git repository via abf-console-client, it stores the path to this repository. After that you can call abfcd [<project_group>/]<project_name> (omit group_name to use your default) to change your current working directory to the project's path (if it had been stored before). If you just want to get a project path without changing directory, use abf locate -p <project>.

You can also add projects to this database by manually. To add only one project call abf locate update -d /path/to/project. To scan directory recursively and add all the found projects, call abf locale update-recursive -d /path/to/directory.


Options:

abf locate [<action>] [--project <project>] [--directory <directory>]

  • <action>: one of {update, update-recursive}. update will update
  • -p/--project <project>: Project to show information for (if needed). Format: "[group/]name". If no group specified, your default group will be used.
  • -d/--directory <directory>: Directory to update locations for. It should be a git repository for "update" and any directory for "update-recursive". If not specified - the current directory will be used.


build

Initiate a build task on ABF. There are lots of options, but in most cases it works great with no options at all. abf-console-client tries to automatically resolve all the options needed. How does it work: read notes and examples.


Options: abf build [--project <project>] [--branch <branch>] [--tag <tag>] [--commit <commit>] [--save-to-repository <repository>] [--repository <repository>] [--arch <arch>] [--auto-publish] [--update-type <type>] [--skip-spec-check]

  • -p/--project <project>: project name ([group/]project). If the option is not specified and you are in a git repository directory - a project name will be resolved automatically.
  • -b/--branch <branch>: git branch name to build. Read notes.
  • -t/--tag <tag>: git tag to build. Read notes.
  • -c/--commit <commit>: git commit sha hash to build. Read notes.
  • -s/--save-to-repository <repository>: repository to save results to. If no platform part specified, it is assumed to be "<default_group>_personal". If this option is not specified at all and you are in a git repository, it will be resolved automatically (read notes).
  • -r/--repository <repository>: repositories to build with. Can be set more than once. If no platform part specified, it is assumed to be your "<default_build_platform>". If no repositories were specified at all, it will be resolved automatically (read notes).
  • -a/--arch <arch>: architectures to build for. Can be set more than once. If not set - use all the available architectures for the project given.
  • --auto-publish: Enable automatic publishing.
  • --update-type <type>: Update type, one of {security, bugfix, enhancement, recommended, newpackage}. Default is "security".
  • --skip-spec-check: Do not check spec file.

Notes:

Console client tries to automatically resolve all the options without taking into account anything except --branch. If it fails - use only user-given options. If succeeds, but user specified other parameters - discard everything we've resolved and use only user-given options.

Git hash resolving:

  • Only one of --branch, --tag or --commit options can be used at once.
  • If you've specified commit hash - it will be used "as is".
  • If you've specified branch or tag name - commit hash will be resolved automatically using ABF API. (the hash of top commit in the branch will be used for branch)
  • If you've specified a project name - you have to specify a branch, tag or commit manually.
  • If you've specified no branch, tag or commit hash and you've not specified a project name (you have to be in a git repository) - the top remote commit of your current branch will be used.

Other options resolving:

  • if no --arch specified, use all the available project architectures.
  • build repository can usually be resolved from a git branch name. If there is a build platform with the name of your current git branch exists - it will be used.
  • save-to repository: if some repository from a build platform can be used as save-to repository - it will be used.

You can execute abf build (with no options) when you've specified a branch name (or it's resolved automatically) and there is a platform on ABF having the same name. In this case a save-to repository will be selected from a list of available save-to repositories for this platform. Build repositories are repositories of this platform too("main" and one another repository if needed for selected save-to repository).

Examples:

  • Build a current branch of a local cloned project. Build and save-to repositories can be resolved from branch name: abf build
  • Build a project without a local git repository. Build and save-to repositories can be resolved from branch name: abf build --project import/gcc --branch rosa2012.1
  • Build a project to another platform: abf build --project import/gcc --branch rosa2012.1 --save-to-repository rosa2012lts/contrib. Build repositories will be rosa2012lts/main and rosa2012lts/contrib.


mock-urpm

Build a project locally using mock-urpm. No checkouts will be made, the current git repository state will be used.

Options:

abf mock-urpm [-c <config>]

  • -c/--config <config>: A config template to use. Specify one of the config names from /etc/abf/mock-urpm/configs/. Directory path and extension (".cfg") should be omitted. If no config specified, "default.cfg" will be used. Autocompletion worsk for config names.

Notes:

  • Be careful, "abf fetch" will be called before building a package. It means that if you have modified tarball, but .abf.yml contains a hash of an old one - your tarball will be replaced by downloaded from server.
  • Building root can be found in /var/lib/abf/mock-urpm.

rpmbuild

Build a project locally using rpmbuild. No checkouts will be made, the current git repository state will be used.

Options:

abf mock-urpm [-c <config>]

  • -c/--config <config>: A config template to use. Specify one of the config names from /etc/abf/mock-urpm/configs/. Directory path and extension (".cfg") should be omitted. If no config specified, "default.cfg" will be used. Autocompletion worsk for config names.

Notes:

  • Be careful, "abf fetch" will be called before building a package. It means that if you have modified tarball, but .abf.yml contains a hash of an old one - your tarball will be replaced by downloaded from server.
  • Building root can be found in /var/lib/abf/mock-urpm.

publish

Publish the task that have already been built.

Options:

abf publish <task_id> [<task_id>] [...]

  • <task_id>: ID of a build list to publish.


copy

Copy files from one git branch to another. Be careful: all the files from destination branch will be removed, and replaced by the files from a source branch.

This command can be useful to keep sources of the project in one branch and build package in another. Just execute abf

Options:

  • <src_branch>: A branch to copy files from.
  • <dst_branch): A branch to copy files to. If not specified, it's assumed to be the current branch.
  • -p/--pack: Create a tar.gz from the <src_branch> and put this archive and spec file to <dst_branch>.


status

Show information about a build list. It will print something like that: 

 Buildlist ID:       944492
 User:               akirilenko
 Project:            import/mock-urpm
 Status:             build has been published
 Build for platform: rosa2012.1
 Save to repository: rosa2012.1/main
 Build repositories: [rosa2012.1/main]
 Architecture:       i586
 Created at:         2013-02-12 15:25:09
 Updated at:         2013-02-12 15:43:16


Options:

abf status [--project <project>] [--short]

  • -p/--project <project>: Project. If last IDs for this project can be found in local database - use them. Otherwise, print error message end exit.
  • -s, --short: Show only one-line information including id, project, arch and status.


clean

Analyze spec file and show missing and unnecessary files from the current git repository directory.

Options:

abf clean [--auto-remove]

  • -a/--auto-remove: automatically remove all the unnecessary files.

search

Search for something on ABF.

Options: abf search <target> "<query>"

  • <target>: what to search for: {users, groups, platforms, projects}.
  • <query>: A string to search for. No regular expressions, just a plain text.

test

Execute a set of internal datamodel tests.

Retrieved from "http://wiki.rosalab.ru/en/index.php?title=ABF_Console_Client&oldid=2443"